Limitations and known issues

    Calico BGP networking limitations

    If you are using a networking type that requires layer 2 reachability (such as Calico with a BGP mesh and no peering to your fabric), you can check that your network has layer 2 reachability as follows:

    On each of your nodes, check the IP network of the network adapter that you plan to use for pod networking. For example, on Linux, assuming your network adapter is eth0, you can run:

    In this case, the IPv4 is 192.168.171.136/24; which, after applying the /24 mask gives 192.168.171.0/24 for the IP network.

    Similarly, on Windows, you can run

    In this case, the IPv4 address is 172.20.41.103 and the mask is represented as bytes 255.255.224.0 rather than CIDR notation. Applying the mask, we get a network address 172.20.32.0/19.

    Because the linux node has network 192.168.171.136/24 and the Windows node has a different network, 172.20.32.0/19, they are unlikely to be on the same layer 2 network.

    Calico VXLAN networking limitations

    Because of differences between the Linux and Windows dataplane feature sets, the following Calico features are not supported on Windows.

    VXLAN limitations

    VXLAN support

    • Windows 1903 build 18317 and above

    Configuration updates

    Certain configuration changes will not be honored after the first pod is networked. This is because Windows does not currently support updating the VXLAN subnet parameters after the network is created so updating those parameters requires the node to be drained:

    One example is the VXLAN VNI setting. To change such parameters:

    • Drain the node of all pods

    • Delete the Calico HNS network:

    • Update the configuration in config.ps1, run uninstall-calico.ps1 and then install-calico.ps1 to regenerate the CNI configuration.

    Pod-to-pod connections are dropped with TCP reset packets

    • The policy that applies to a pod is updated
    • Some ingress or egress policy that applies to a pod contains selectors and the set of endpoints that those selectors match changes

    Felix must reprogram the HNS ACL policy attached to the pod. This reprogramming can cause TCP resets. Microsoft has confirmed this is a HNS issue, and they are investigating.

    Windows 1809 prior to build 17763.1432

    On Windows nodes, kube-proxy unconditionally applies source NAT to traffic from local pods to service ClusterIPs. This means that, at the destination pod, where policy is applied, the traffic appears to come from the source host rather than the source pod. In turn, this means that a network policy with a source selector matching the source pod will not match the expected traffic.

    Network policy and using selectors

    Under certain conditions, relatively simple Calico policies can require significant Windows dataplane resources, that can cause significant CPU and memory usage, and large policy programming latency.

    We recommend avoiding policies that contain rules with both a source and destination selector. The following is an example of a policy that would be inefficient. The policy applies to all workloads, and it only allows traffic from workloads labeled as clients to workloads labeled as servers:

    Because the policy applies to all workloads, it will be rendered once per workload (even if the workload is not labeled as a server), and then the selectors will be expanded into many individual dataplane rules to capture the allowed connectivity.

    Here is a much more efficient policy that still allows the same traffic:

    Next steps