Networking on Windows

Networking for Windows containers is exposed through . Windows containers function similarly to virtual machines in regards to networking. Each container has a virtual network adapter (vNIC) which is connected to a Hyper-V virtual switch (vSwitch). The Host Networking Service (HNS) and the Host Compute Service (HCS) work together to create containers and attach container vNICs to networks. HCS is responsible for the management of containers whereas HNS is responsible for the management of networking resources such as:

• Virtual networks (including creation of vSwitches)
• Endpoints / vNICs
• Namespaces
• Policies including packet encapsulations, load-balancing rules, ACLs, and NAT rules.

The Windows HNS and vSwitch implement namespacing and can create virtual NICs as needed for a pod or container. However, many configurations such as DNS, routes, and metrics are stored in the Windows registry database rather than as files inside , which is how Linux stores those configurations. The Windows registry for the container is separate from that of the host, so concepts like mapping /etc/resolv.conf from the host into a container don’t have the same effect they would on Linux. These must be configured using Windows APIs run in the context of that container. Therefore CNI implementations need to call the HNS instead of relying on file mappings to pass network details into the pod or container.

Windows supports five different networking drivers/modes: L2bridge, L2tunnel, Overlay (Beta), Transparent, and NAT. In a heterogeneous cluster with Windows and Linux worker nodes, you need to select a networking solution that is compatible on both Windows and Linux. The following table lists the out-of-tree plugins are supported on Windows, with recommendations on when to use each CNI:

As outlined above, the CNI plugin is also on Windows via the VXLAN network backend (Beta support ; delegates to win-overlay) and (stable support; delegates to win-bridge).

For Node, Pod, and Service objects, the following network flows are supported for TCP/UDP traffic:

• Pod → Pod (IP)
• Pod → Pod (Name)
• Pod → Service (Cluster IP)
• Pod → Service (FQDN)
• Pod → external (IP)
• Pod → external (DNS)
• Node → Pod
• Pod → Node

The following IPAM options are supported on Windows:

• azure-vnet-ipam (for azure-cni only)
• (fallback option if no IPAM is set)

A Kubernetes is an abstraction that defines a logical set of Pods and a means to access them over a network. In a cluster that includes Windows nodes, you can use the following types of Service:

• NodePort
• ClusterIP
• ExternalName

Windows container networking differs in some important ways from Linux networking. The Microsoft documentation for Windows Container Networking provides additional details and background.

Warning:

There are known issue with NodePort Services on overlay networking, if the destination node is running Windows Server 2022. To avoid the issue entirely, you can configure the service with externalTrafficPolicy: Local.

There are known issues with Pod to Pod connectivity on l2bridge network on Windows Server 2022 with KB5005619 or higher installed. To workaround the issue and restore Pod to Pod connectivity, you can disable the WinDSR feature in kube-proxy.

These issues require OS fixes. Please follow https://github.com/microsoft/Windows-Containers/issues/204 for updates.

• Local NodePort access from the node itself (works for other nodes or external clients)
• More than 64 backend pods (or unique destination addresses) for a single Service
• IPv6 communication between Windows pods connected to overlay networks
• Local Traffic Policy in non-DSR mode
• Outbound communication using the ICMP protocol via the win-overlay, win-bridge, or using the Azure-CNI plugin.
  Specifically, the Windows data plane (VFP) doesn’t support ICMP packet transpositions, and this means:
  • ICMP packets directed to destinations within the same network (such as pod to pod communication via ping) work as expected;
  • TCP/UDP packets work as expected;
  • ICMP packets directed to pass through a remote network (e.g. pod to external internet communication via ping) cannot be transposed and thus will not be routed back to their source;
  • Since TCP/UDP packets can still be transposed, you can substitute ping <destination> with when debugging connectivity with the outside world.

Other limitations:

• Windows reference network plugins win-bridge and win-overlay do not implement v0.4.0, due to a missing CHECK implementation.
• The Flannel VXLAN CNI plugin has the following limitations on Windows:
  • Node-pod connectivity is only possible for local pods with Flannel v0.12.0 (or higher).