Transparent Proxying

  • On Kubernetes kuma-dp leverages transparent proxying automatically via iptables or CNI, for all incoming and outgoing traffic that is automatically intercepted by kuma-dp without having to change the application code.
  • On Universal kuma-dp leverages the data plane proxy specification associated to it for receiving incoming requests on a pre-defined port, while it can leverage both transparent proxying and explicit outbound entries in the same data plane proxy specification for all outgoing traffic.

There are several advantages for using transparent proxying in universal mode:

  • Simpler , as the outbound section becomes obsolete and can be skipped.
  • Uiversal service naming using .mesh DNS domain.
  • Better service manageability (security, tracing).

The Kuma control plane exposes a DNS service which handles the name resolution in the .mesh DNS zone. By default it listens on port UDP/5653. For this setup we need it to listen on port UDP/53, therefore make sure that this environment variable is set before running kuma-cp: KUMA_DNS_SERVER_PORT=53.

The IP address of the host that runs Kuma Control plane will be used in the next section. Make sure to have it once, kuma-cp is started.

The host that will run the kuma-dp process in transparent proxying mode needs to be prepared with the following steps:

  1. Create a new dedicated user on the machine.
  2. Point the DNS resolving for .mesh to the kuma-cp embedded DNS server.

Kuma comes with which can help us preparing the host. Due to the wide variety of Linux setup options, these steps may vary and may need to be adjusted for the specifics of the particular deployment. However, the common steps would be to execute the following commands as root:

Where kuma-dp-user is the name of the dedicated user that will be used to run the kuma-dp process and <kuma-cp IP> is the IP address of the Kuma control plane (kuma-cp).

Please note that this command will change both the host iptables rules as well as modify , while keeping a backup copy of the original file.

The command has several other options which allow to change the default inbound and outbound redirect ports, add ports for exclusion and also disable the iptables or resolve.conf modification steps. The command’s help has enumerated and documented the available options.

The changes will persist over restarts, so this command is needed only once. Reverting back to the original state of the host can be done by issuing kumactl uninstall transparent-proxy.

In transparent proxying mode, the Dataplane resource that will be should ommit the networking.outbound section and use networking.transparentProxyingsection instead.

  1. type: Dataplane
  2. mesh: default
  3. name: {{ name }}
  4. networking:
  5.   address: {{ address }}
  6.   inbound:
  7.   - port: {{ port }}
  8.     tags:
  9.       kuma.io/service: demo-client
  10.   transparentProxying:
  11.     redirectPortInbound: 15006

The ports illustrated above are the default ones that kumactl install transparent-proxy will set. These can be changed using the relevant flags to that command.

It is important that the kuma-dp process runs with the same system user that was passed to kumactl install transparent-proxy --kuma-dp-user.

When systemd is used, this can be done with an entry User=kuma-dp in the section of the service file.

When starting kuma-dp with a script or some other automation instead, we can use runuser with the aforementioned yaml resource as follows:

There are two ways of how the service can interact with its sidecar to connect to other services.

One is explicitly defining outbounds in the Dataplane:

  1. type: Dataplane
  2. ...
  3. networking:
  4.   ...
  5.   outbound:
  6.   - port: 10000
  7.     tags:
  8.       kuma.io/service: backend

The alternative approach is Transparent Proxying. With Transparent Proxying before we start a service, we apply iptables (opens new window) that intercept all the traffic on VM/Pod and redirect it to Envoy. The main advantage of this mode is when you integrate with the current hostname resolving mechanism, you can deploy Service Mesh transparently on the platform without reconfiguring applications.

Kuma provides support for transparent proxying on both Universal and Kubernetes.

Kuma deploys iptables rules either with kuma-init Init Container or with cni when deployed with CNI mode.

By default, all the traffic is intercepted by Envoy. You can exclude which ports are intercepted by Envoy with the following annotations placed on the Pod

You can also control this value on whole Kuma deployment with the following Kuma CP configuration

  1. KUMA_RUNTIME_KUBERNETES_SIDECAR_TRAFFIC_EXCLUDE_INBOUND_PORTS=1234
  2. KUMA_RUNTIME_KUBERNETES_SIDECAR_TRAFFIC_EXCLUDE_OUTBOUND_PORTS=5678,8900

Global settings can be always overridden with annotations on the individual Pods.

When deploying Kuma with kumactl install control-plane you can set those settings with

When deploying Kuma with HELM, use controlPlane.envVar value

  1. envVar:
  2.   KUMA_RUNTIME_KUBERNETES_SIDECAR_TRAFFIC_EXCLUDE_INBOUND_PORTS: "1234"