NoEncap and Hybrid Traffic Modes of Antrea

    The NoEncap and Hybrid traffic modes require AntreaProxy to support correct NetworkPolicy enforcement, which is why trying to disable AntreaProxy in these modes will normally cause the Antrea Agent to fail. It is possible to override this behavior and force AntreaProxy to be disabled by setting the ALLOW_NO_ENCAP_WITHOUT_ANTREA_PROXY environment variable to true for the Antrea Agent. For example:

    Let us start from Hybrid mode which is simpler to configure. Hybrid mode does not encapsulate Pod traffic when the source and the destination Nodes are in the same subnet. Thus it requires the Node network to allow Pod IP addresses to be sent out from the Nodes’ NICs. This requirement is not supported in all the networks and clouds, or in some cases it might require specific configuration of the Node network. For example:

    • On AWS, the source/destination checks must be disabled on the EC2 instances of the Kubernetes Nodes, as described in the .

    • On Google Compute Engine, IP forwarding must be enabled on the VM instances as described in the Google Cloud documentation.

    • On Azure, there is no way to let VNet forward unknown IPs, hence Antrea Hybrid mode cannot work on Azure.

    1.   antrea-agent.conf: |
    3.     ... ...

    After changing the config parameter, you can deploy Antrea in Hybrid mode with the usual command:

    1. kubectl apply -f antrea.yml

    NoEncap Mode

    In NoEncap mode, Antrea never encapsulates Pod traffic. Just like Hybrid mode, the Node network needs to allow Pod IP addresses sent out from Nodes. When the Nodes are not in the same subnet, NoEncap mode additionally requires the Node network be able to route the Pod traffic from the source Node to the destination Node. There are two possibilities to enable this routing by Node network:

    • Leverage Route Controller of Kubernetes Cloud Controller Manager. The Kubernetes Cloud Providers that implement Route Controller can add routes to the cloud network routers for the Pod CIDRs of Nodes, and then the cloud network is able to route Pod traffic between Nodes. This Route Controller functionality is supported by the Cloud Provider implementations of the major clouds, including: , Azure, , and vSphere (with NSX-T).

    • Run a routing protocol or even manually configure routers to add routes to the Node network routers. For example, Antrea can work with and leverage kube-router to advertise Pod CIDRs to routers using BGP. Section Using kube-router for BGP describes how to configure Antrea and kube-router to work together.

    When the Node network can support forwarding and routing of Pod traffic, Antrea can be configured to run in the NoEncap mode, by setting the trafficEncapMode config parameter of antrea-agent to noEncap. By default, Antrea performs SNAT (source network address translation) for the outbound connections from a Pod to outside of the Pod network, using the Node’s IP address as the SNAT IP. In the NoEncap mode, as the Node network knows about Pod IP addresses, the SNAT by Antrea might be unnecessary. In this case, you can disable it by setting the noSNAT config parameter to true. The trafficEncapMode and noSNAT config parameters are defined in antrea-agent.conf of the ConfigMap in the .

    We can run kube-router in advertisement-only mode to advertise Pod CIDRs to the peered routers, so the routers can know how to route Pod traffic to the Nodes. To deploy kube-router in advertisement-only mode, first download the :

    1. curl -LO https://raw.githubusercontent.com/cloudnativelabs/kube-router/v0.4.0/daemonset/generic-kuberouter-only-advertise-routes.yaml

    Then edit the yaml file and set the following kube-router arguments:

    1. - "--run-router=true"
    2. - "--run-firewall=false"
    3. - "--run-service-proxy=false"
    4. - "--enable-ibgp=false"
    5. - "--enable-overlay=false"
    6. - "--enable-pod-egress=false"
    7. - "--peer-router-ips=<CHANGE ME>"
    8. - "--peer-router-asns=<CHANGE ME>"

    The BGP peers should be configured by specifying the --peer-router-asns and --peer-router-ips parameters. Note, the ASNs and IPs must match the configuration on the peered routers. For example:

    Then you can deploy the kube-router DaemonSet with:

    1. kubectl apply -f generic-kuberouter-only-advertise-routes.yaml

    You can verify that the kube-router Pods are running on the Nodes of your Kubernetes cluster by (the cluster in the following example has only two Nodes):

    1. $ kubectl -n kube-system get pods -l k8s-app=kube-router
    2. NAME                READY     STATUS    RESTARTS   AGE