Change IP pool block size

    Value

    Changing IP pool block size after installation requires ordered steps to minimize pod connectivity disruption.

    This how-to guide uses the following Calico features:

    • IPPool resource with field

    Concepts

    By default, Calico uses an IPAM block size of 64 addresses – /26 for IPv4, and /122 for IPv6. However, the block size can be changed depending on the IP pool address family.

    • IPv4: 20-32, inclusive
    • IPv6: 116-128, inclusive

    You can have only one default IP pool for per protocol in your installation manifest. In this example, there is one IP pool for IPv4 (/26), and one IP pool for IPv6 (/122).

    However, the following is invalid because it has two IP pools for IPv4.

    1. apiVersion: operator.tigera.io/v1
    2. kind: Installation
    3. metadata:
    4.   name: default
    5. spec:
    6.     Configures Calico networking.
    7.   calicoNetwork:
    8.      Note: The ipPools section cannot be modified post-install.
    9.     - blockSize: 26
    10.       cidr: 10.48.0.0/21
    11.       encapsulation: IPIP
    12.       natOutgoing: Enabled
    13.       nodeSelector: all()
    14.     - blockSize: 31
    15.       cidr: 10.48.8.0/21
    16.       encapsulation: IPIP
    17.       natOutgoing: Enabled
    18.       nodeSelector: all()

    Expand or shrink IP pool block sizes

    By default, the Calico IPAM block size for an IP pool is /26. To expand from the default size /26, lower the blockSize (for example, /24). To shrink the from the default /26, raise the number (for example, /28).

    Best practice: change IP pool block size before installation

    Because the blockSize field cannot be edited directly after Calico installation, it is best to change the IP pool block size before installation to minimize disruptions to pod connectivity.

    Required

    Verify that you are using Calico IPAM.

    If you are not sure which IPAM your cluster is using, the way to tell depends on install method.

    • Operator
    • Manifest

    The IPAM plugin can be queried on the default Installation resource.

    1. kubectl get installation default -o go-template --template {{.spec.cni.ipam.type}}

    If your cluster is using Calico IPAM, the above command should return a result of Calico.

    1. cat /etc/cni/net.d/10-calico.conflist

    Look for the entry:

    1.          "ipam": {
    2.               "type": "calico-ipam"
    3.           },

    If it is present, you are using the Calico IPAM. If the IPAM is not Calico, or the 10-calico.conflist file does not exist, you cannot use these features in your cluster.

    How to

    note

    Follow the steps in order to minimize pod connectivity disruption. Pods may lose connectivity when they are redeployed, and may lose external connectivity while in the temporary pool. Also, when pods are deleted, applications may be temporarily unavailable (depending on the type of application). Plan your changes accordingly.

    The high-level steps to follow are:

    1. Note: The temporary IP pool must not overlap with the existing one.
    2. Disable the existing IP poolNote: When you disable an IP pool, only new IP address allocations are prevented; networking of existing pods are not affected.
    3. This includes any new pods that may have been created with the existing IP pool prior to disabling the pool. Verify that new pods get an address from the temporary IP pool.
    4. Delete the existing IP pool
    6. Disable the temporary IP pool
    8. Delete the temporary IP pool

    In the following steps, our Kubernetes cluster has a default CIDR block size of /26. We want to shrink the block size to /28 to use the pool more efficiently.

    We add a new IPPool with the CIDR range, 10.0.0.0/16.

    Create a temporary-pool.yaml.

    1. apiVersion: projectcalico.org/v3
    2. kind: IPPool
    3. metadata:
    4.   name: temporary-pool
    5.   cidr: 10.0.0.0/16
    6.   ipipMode: Always
    7.   natOutgoing: true

    Apply the changes.

    1. calicoctl apply -f temporary-pool.yaml

    Let’s verify the temporary IP pool.

    1. NAME                  CIDR             NAT    IPIPMODE   DISABLED
    2. default-ipv4-ippool   192.168.0.0/16   true   Always     false
    3. temporary-pool        10.0.0.0/16      true   Always     false

    Disable the existing IP pool

    Disable allocations in the default pool.

    1. calicoctl patch ippool default-ipv4-ippool -p '{"spec": {"disabled": true}}'

    Verify the changes.

    1. NAME                  CIDR             NAT    IPIPMODE   DISABLED
    2. default-ipv4-ippool   192.168.0.0/16   true   Always     true
    3. temporary-pool        10.0.0.0/16      true   Always     false

    Delete pods from the existing IP pool

    1. kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp

    Restart all pods with just one command.

    Change IP pool block size - 图2caution

    The following command is disruptive and may take several minutes depending on the number of pods deployed.

    1. kubectl delete pod -A --all

    Now that you’ve verified that pods are getting IPs from the new range, you can safely delete the existing pool.

    Create a new IP pool with the desired block size

    In this step, we update the IPPool with the new block size of (/28).

    1. apiVersion: projectcalico.org/v3
    2. kind: IPPool
    3. metadata:
    4.   name: default-ipv4-ippool
    5. spec:
    6.   blockSize: 28
    7.   cidr: 192.0.0.0/16
    8.   ipipMode: Always
    9.   natOutgoing: true

    Apply the changes.

    1. calicoctl apply -f pool.yaml

    Disable the temporary IP pool

    1. calicoctl patch ippool temporary-pool -p '{"spec": {"disabled": true}}'

    In our example, coredns is our only pod; for multiple pods you would trigger a deletion for all pods in the cluster.

    1. kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp

    Restart all pods with just one command.

    caution

    The following command is disruptive and may take several minutes depending on the number of pods deployed.

    1. kubectl delete pod -A --all

    Validate your pods and block size are correct by running the following commands:

    1. kubectl get pods --all-namespaces -o wide
    2. calicoctl ipam show --show-blocks

    Delete the temporary IP pool

    Clean up the IP pools by deleting the temporary IP pool.