IPv4/IPv6 dual-stack

    FEATURE STATE:

    IPv4/IPv6 dual-stack networking enables the allocation of both IPv4 and IPv6 addresses to Pods and .

    IPv4/IPv6 dual-stack networking is enabled by default for your Kubernetes cluster starting in 1.21, allowing the simultaneous assignment of both IPv4 and IPv6 addresses.

    IPv4/IPv6 dual-stack on your Kubernetes cluster provides the following features:

    • Dual-stack Pod networking (a single IPv4 and IPv6 address assignment per Pod)
    • IPv4 and IPv6 enabled Services
    • Pod off-cluster egress routing (eg. the Internet) via both IPv4 and IPv6 interfaces

    Prerequisites

    The following prerequisites are needed in order to utilize IPv4/IPv6 dual-stack Kubernetes clusters:

    • Kubernetes 1.20 or later

      For information about using dual-stack services with earlier Kubernetes versions, refer to the documentation for that version of Kubernetes.

    • Provider support for dual-stack networking (Cloud provider or otherwise must be able to provide Kubernetes nodes with routable IPv4/IPv6 network interfaces)

    • A network plugin that supports dual-stack networking.

    To configure IPv4/IPv6 dual-stack, set dual-stack cluster network assignments:

    • kube-apiserver:
      • --service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>
    • kube-controller-manager:
      • --cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>
      • --service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>
      • --node-cidr-mask-size-ipv4|--node-cidr-mask-size-ipv6 defaults to /24 for IPv4 and /64 for IPv6
    • kube-proxy:
      • --cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>
    • kubelet:
      • when there is no --cloud-provider the administrator can pass a comma-separated pair of IP addresses via --node-ip to manually configure dual-stack .status.addresses for that Node. If a Pod runs on that node in HostNetwork mode, the Pod reports these IP addresses in its .status.podIPs field. All podIPs in a node match the IP family preference defined by the .status.addresses field for that Node.

    Note:

    An example of an IPv4 CIDR: 10.244.0.0/16 (though you would supply your own address range)

    An example of an IPv6 CIDR: fdXY:IJKL:MNOP:15::/64 (this shows the format but is not a valid address - see RFC 4193)

    FEATURE STATE: Kubernetes v1.27 [alpha]

    When using an external cloud provider, you can pass a dual-stack --node-ip value to kubelet if you enable the CloudDualStackNodeIPs feature gate in both kubelet and the external cloud provider. This is only supported for cloud providers that support dual stack clusters.

    Services

    You can create Services which can use IPv4, IPv6, or both.

    When you define a Service you can optionally configure it as dual stack. To specify the behavior you want, you set the .spec.ipFamilyPolicy field to one of the following values:

    • SingleStack: Single-stack service. The control plane allocates a cluster IP for the Service, using the first configured service cluster IP range.
    • PreferDualStack:
      • Allocates IPv4 and IPv6 cluster IPs for the Service.
    • RequireDualStack: Allocates Service .spec.ClusterIPs from both IPv4 and IPv6 address ranges.
      • Selects the .spec.ClusterIP from the list of .spec.ClusterIPs based on the address family of the first element in the .spec.ipFamilies array.

    If you would like to define which IP family to use for single stack or define the order of IP families for dual-stack, you can choose the address families by setting an optional field, .spec.ipFamilies, on the Service.

    Note: The .spec.ipFamilies field is immutable because the .spec.ClusterIP cannot be reallocated on a Service that already exists. If you want to change .spec.ipFamilies, delete and recreate the Service.

    You can set .spec.ipFamilies to any of the following array values:

    • ["IPv4"]
    • ["IPv6"]
    • ["IPv4","IPv6"] (dual stack)
    • ["IPv6","IPv4"] (dual stack)

    The first family you list is used for the legacy .spec.ClusterIP field.

    These examples demonstrate the behavior of various dual-stack Service configuration scenarios.

    Dual-stack options on new Services

    1. This Service specification does not explicitly define .spec.ipFamilyPolicy. When you create this Service, Kubernetes assigns a cluster IP for the Service from the first configured service-cluster-ip-range and sets the .spec.ipFamilyPolicy to SingleStack. ( and headless Services with selectors will behave in this same way.)

    2. This Service specification explicitly defines PreferDualStack in .spec.ipFamilyPolicy. When you create this Service on a dual-stack cluster, Kubernetes assigns both IPv4 and IPv6 addresses for the service. The control plane updates the .spec for the Service to record the IP address assignments. The field .spec.ClusterIPs is the primary field, and contains both assigned IP addresses; .spec.ClusterIP is a secondary field with its value calculated from .spec.ClusterIPs.

      • For the .spec.ClusterIP field, the control plane records the IP address that is from the same address family as the first service cluster IP range.
      • On a cluster with dual-stack enabled, specifying RequireDualStack in .spec.ipFamilyPolicy behaves the same as PreferDualStack.

      service/networking/dual-stack-preferred-svc.yaml IPv4/IPv6 dual-stack - 图2

      1. apiVersion: v1
      2.    kind: Service
      3.    metadata:
      4.      name: my-service
      5.      labels:
      7.    spec:
      8.      ipFamilyPolicy: PreferDualStack
      9.      selector:
      10.        app.kubernetes.io/name: MyApp
      11.      ports:
      12.        - protocol: TCP
      13.          port: 80

    3. This Service specification explicitly defines IPv6 and IPv4 in .spec.ipFamilies as well as defining PreferDualStack in .spec.ipFamilyPolicy. When Kubernetes assigns an IPv6 and IPv4 address in .spec.ClusterIPs, .spec.ClusterIP is set to the IPv6 address because that is the first element in the .spec.ClusterIPs array, overriding the default.

      1. apiVersion: v1
      2.    kind: Service
      3.    metadata:
      4.      name: my-service
      5.      labels:
      6.        app.kubernetes.io/name: MyApp
      7.    spec:
      8.      ipFamilyPolicy: PreferDualStack
      9.      ipFamilies:
      10.      - IPv6
      11.      - IPv4
      12.      selector:
      13.        app.kubernetes.io/name: MyApp
      14.      ports:
      15.        - protocol: TCP
      16.          port: 80

    Dual-stack defaults on existing Services

    These examples demonstrate the default behavior when dual-stack is newly enabled on a cluster where Services already exist. (Upgrading an existing cluster to 1.21 or beyond will enable dual-stack.)

    1. When dual-stack is enabled on a cluster, existing Services (whether IPv4 or IPv6) are configured by the control plane to set .spec.ipFamilyPolicy to SingleStack and set .spec.ipFamilies to the address family of the existing Service. The existing Service cluster IP will be stored in .spec.ClusterIPs.

      IPv4/IPv6 dual-stack - 图4

      You can validate this behavior by using kubectl to inspect an existing service.

      1. kubectl get svc my-service -o yaml
      1. apiVersion: v1
      2. kind: Service
      3. metadata:
      4.   labels:
      5.     app.kubernetes.io/name: MyApp
      6.   name: my-service
      7. spec:
      8.   clusterIP: 10.0.197.123
      9.   - 10.0.197.123
      10.   ipFamilies:
      11.   - IPv4
      12.   ipFamilyPolicy: SingleStack
      13.   ports:
      14.   - port: 80
      15.     protocol: TCP
      16.     targetPort: 80
      17.   selector:
      19.   type: ClusterIP
      20. status:
      21.   loadBalancer: {}

    2. service/networking/dual-stack-default-svc.yaml

      You can validate this behavior by using kubectl to inspect an existing headless service with selectors.

      1. kubectl get svc my-service -o yaml
      1. apiVersion: v1
      2. kind: Service
      3. metadata:
      4.   labels:
      5.     app.kubernetes.io/name: MyApp
      6.   name: my-service
      7. spec:
      8.   clusterIP: None
      9.   clusterIPs:
      10.   - None
      11.   ipFamilies:
      12.   - IPv4
      13.   ipFamilyPolicy: SingleStack
      14.   ports:
      15.   - port: 80
      16.     protocol: TCP
      17.     targetPort: 80
      18.   selector:
      19.     app.kubernetes.io/name: MyApp

    Switching Services between single-stack and dual-stack

    Services can be changed from single-stack to dual-stack and from dual-stack to single-stack.

    1. To change a Service from single-stack to dual-stack, change .spec.ipFamilyPolicy from SingleStack to PreferDualStack or RequireDualStack as desired. When you change this Service from single-stack to dual-stack, Kubernetes assigns the missing address family so that the Service now has IPv4 and IPv6 addresses.

      Edit the Service specification updating the .spec.ipFamilyPolicy from SingleStack to PreferDualStack.

      Before:

      After:

      1. spec:
      2.   ipFamilyPolicy: PreferDualStack

    2. To change a Service from dual-stack to single-stack, change .spec.ipFamilyPolicy from PreferDualStack or RequireDualStack to SingleStack. When you change this Service from dual-stack to single-stack, Kubernetes retains only the first element in the .spec.ClusterIPs array, and sets .spec.ClusterIP to that IP address and sets .spec.ipFamilies to the address family of .spec.ClusterIPs.

    For and without .spec.ipFamilyPolicy explicitly set, the .spec.ipFamilyPolicy field defaults to RequireDualStack.

    To provision a dual-stack load balancer for your Service:

    • Set the .spec.type field to LoadBalancer
    • Set .spec.ipFamilyPolicy field to PreferDualStack or RequireDualStack

    Note: To use a dual-stack LoadBalancer type Service, your cloud provider must support IPv4 and IPv6 load balancers.

    If you want to enable egress traffic in order to reach off-cluster destinations (eg. the public Internet) from a Pod that uses non-publicly routable IPv6 addresses, you need to enable the Pod to use a publicly routed IPv6 address via a mechanism such as transparent proxying or IP masquerading. The ip-masq-agent project supports IP masquerading on dual-stack clusters.

    Note: Ensure your provider supports IPv6.

    Windows support

    Kubernetes on Windows does not support single-stack “IPv6-only” networking. However, dual-stack IPv4/IPv6 networking for pods and nodes with single-family services is supported.

    You can use IPv4/IPv6 dual-stack networking with l2bridge networks.

    Note: Overlay (VXLAN) networks on Windows do not support dual-stack networking.