Configuring Gateway Network Topology
Today’s networks vary widely in nature, but support for these attributes is a requirement no matter what the network topology is. This information should be preserved and forwarded whether the network uses cloud-based Load Balancers, on-premise Load Balancers, gateways that are exposed directly to the internet, gateways that serve many intermediate proxies, and other deployment topologies not specified.
While Istio provides an , given the varieties of architectures mentioned above, reasonable defaults are not able to be shipped that support the proper forwarding of client attributes to the destination workloads. This becomes ever more vital as Istio multicluster deployment models become more common.
For more information on X-Forwarded-For
, see the IETF’s RFC.
Configuration of XFF and XFCC headers can be set globally for all gateway workloads via MeshConfig
or per gateway using a pod annotation. For example, to configure globally during install or upgrade when using an IstioOperator
custom resource:
You can also configure both of these settings by adding the proxy.istio.io/config
annotation to the Pod spec of your Istio ingress gateway.
...
metadata:
annotations:
"proxy.istio.io/config": '{"gatewayTopology" : { "numTrustedProxies": <VALUE>, "forwardClientCertDetails": <ENUM_VALUE> } }'
Applications rely on reverse proxies to forward client attributes in a request, such as X-Forward-For
header. However, due to the variety of network topologies that Istio can be deployed in, you must set the numTrustedProxies
to the number of trusted proxies deployed in front of the Istio gateway proxy, so that the client address can be extracted correctly. This controls the value populated by the ingress gateway in the X-Envoy-External-Address
header which can be reliably used by the upstream services to access client’s original IP address.
For example, if you have a cloud based Load Balancer and a reverse proxy in front of your Istio gateway, set numTrustedProxies
to 2
.
Note that all proxies in front of the Istio gateway proxy must parse HTTP traffic and append to the X-Forwarded-For
header at each hop. If the number of entries in the X-Forwarded-For
header is less than the number of trusted hops configured, Envoy falls back to using the immediate downstream address as the trusted client address. Please refer to the to understand how X-Forwarded-For
headers and trusted client addresses are determined.
Example using X-Forwarded-For capability with httpbin
-
$ cat <<EOF > topology.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
defaultConfig:
gatewayTopology:
numTrustedProxies: 2
EOF
$ istioctl install -f topology.yaml
If you previously installed an Istio ingress gateway, restart all ingress gateway pods after step 1.
Create an
httpbin
namespace:Set the
istio-injection
label toenabled
for sidecar injection:Deploy
httpbin
in thehttpbin
namespace:$ kubectl apply -n httpbin -f samples/httpbin/httpbin.yaml
Deploy a gateway associated with
httpbin
:Set a local
GATEWAY_URL
environmental variable based on your Istio ingress gateway’s IP address:$ export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
Run the following
curl
command to simulate a request with proxy addresses in theX-Forwarded-For
header:$ curl -s -H 'X-Forwarded-For: 56.5.6.7, 72.9.5.6, 98.1.2.3' "$GATEWAY_URL"/get?show_env=true
{
"args": {
"show_env": "true"
},
"headers": {
"Accept": ...
"Host": ...
"User-Agent": ...
"X-B3-Parentspanid": ...
"X-B3-Sampled": ...
"X-B3-Spanid": ...
"X-B3-Traceid": ...
"X-Envoy-Attempt-Count": ...
"X-Envoy-External-Address": "72.9.5.6",
"X-Forwarded-Client-Cert": ...
"X-Forwarded-Proto": ...
"X-Request-Id": ...
},
"origin": "56.5.6.7, 72.9.5.6, 98.1.2.3,10.244.0.1",
"url": ...
}
In the above example $GATEWAY_URL
resolved to 10.244.0.1. This will not be the case in your environment.
Configuring X-Forwarded-Client-Cert Headers
From Envoy’s documentation regarding XFCC:
x-forwarded-client-cert (XFCC) is a proxy header which indicates certificate information of part or all of the clients or proxies that a request has flowed through, on its way from the client to the server. A proxy may choose to sanitize/append/forward the XFCC header before proxying the request.
To configure how XFCC headers are handled, set forwardClientCertDetails
in your IstioOperator
where ENUM_VALUE
can be of the following type.
See the for examples of using this capability.
This feature is actively in development and is considered .
The PROXY protocol allows for exchanging and preservation of client attributes across multiple proxies without relying on Layer 7 protocols.
If your external load balancer is configured to use the PROXY protocol, the Istio gateway must also be configured to accept the PROXY protocol. Enabling this requires adding the using an EnvoyFilter
applied on the gateway workload. For example:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: proxy-protocol
namespace: istio-system
spec:
configPatches:
- applyTo: LISTENER
patch:
operation: MERGE
value:
listener_filters:
- name: envoy.listener.proxy_protocol
- name: envoy.listener.tls_inspector
workloadSelector:
labels:
istio: ingressgateway