Resize CPU and Memory Resources assigned to Containers

    This page assumes that you are familiar with Quality of Service for Kubernetes Pods.

    This page shows how to resize CPU and memory resources assigned to containers of a running pod without restarting the pod or its containers. A Kubernetes node allocates resources for a pod based on its requests, and restricts the pod’s resource usage based on the limits specified in the pod’s containers.

    For in-place resize of pod resources:

    • Container’s resource requests and limits are mutable for CPU and memory resources.
    • allocatedResources field in containerStatuses of the Pod’s status reflects the resources allocated to the pod’s containers.
    • resources field in containerStatuses of the Pod’s status reflects the actual resource requests and limits that are configured on the running containers as reported by the container runtime.
    • resize field in the Pod’s status shows the status of the last requested pending resize. It can have the following values:
      • Proposed: This value indicates an acknowledgement of the requested resize and that the request was validated and recorded.
      • InProgress: This value indicates that the node has accepted the resize request and is in the process of applying it to the pod’s containers.
      • Deferred: This value means that the requested resize cannot be granted at this time, and the node will keep retrying. The resize may be granted when other pods leave and free up node resources.
      • Infeasible: is a signal that the node cannot accommodate the requested resize. This can happen if the requested resize exceeds the maximum resources the node can ever allocate for a pod.

    You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:

    Your Kubernetes server must be at or later than version 1.27. To check the version, enter kubectl version.

    Container Resize Policies

    Resize policies allow for a more fine-grained control over how pod’s containers are resized for CPU and memory resources. For example, the container’s application may be able to handle CPU resources resized without being restarted, but resizing memory may require that the application hence the containers be restarted.

    To enable this, the Container specification allows users to specify a resizePolicy. The following restart policies can be specified for resizing CPU and memory:

    • NotRequired: Resize the container’s resources while it is running.
    • RestartContainer: Restart the container and apply new resources upon restart.

    If resizePolicy[*].restartPolicy is not specified, it defaults to NotRequired.

    Note: If the Pod’s restartPolicy is Never, container’s resize restart policy must be set to NotRequired for all Containers in the Pod.

    Below example shows a Pod whose Container’s CPU can be resized without restart, but memory resize memory requires the container to be restarted.

    You can create a Guaranteed or Burstable class pod by specifying requests and/or limits for a pod’s containers.

    Consider the following manifest for a Pod that has one Container.

    pods/qos/qos-pod-5.yaml 

    1. apiVersion: v1
    2. kind: Pod
    3. metadata:
    4.   name: qos-demo-5
    5.   namespace: qos-example
    6. spec:
    7.   containers:
    8.   - name: qos-demo-ctr-5
    9.     image: nginx
    10.     resources:
    11.       limits:
    12.         memory: "200Mi"
    14.       requests:
    15.         memory: "200Mi"

    Create the pod in the qos-example namespace:

    1. kubectl create namespace qos-example
    2. kubectl create -f https://k8s.io/examples/pods/qos/qos-pod-5.yaml

    This pod is classified as a Guaranteed QoS class requesting 700m CPU and 200Mi memory.

    View detailed information about the pod:

    Also notice that the values of resizePolicy[*].restartPolicy defaulted to NotRequired, indicating that CPU and memory can be resized while container is running.

    1. spec:
    2.   containers:
    3.     ...
    4.     resizePolicy:
    5.     - resourceName: cpu
    6.       restartPolicy: NotRequired
    7.     - resourceName: memory
    8.       restartPolicy: NotRequired
    9.     resources:
    10.       limits:
    11.         cpu: 700m
    12.         memory: 200Mi
    13.       requests:
    14.         cpu: 700m
    15.         memory: 200Mi
    16. ...
    17.   containerStatuses:
    18. ...
    19.     name: qos-demo-ctr-5
    20.     ready: true
    21. ...
    22.     allocatedResources:
    23.       cpu: 700m
    24.       memory: 200Mi
    25.     resources:
    26.       limits:
    27.         cpu: 700m
    28.         memory: 200Mi
    30.         cpu: 700m
    31.         memory: 200Mi
    32.     restartCount: 0
    33.     started: true
    34.   qosClass: Guaranteed

    Updating the pod’s resources

    Let’s say the CPU requirements have increased, and 0.8 CPU is now desired. This is typically determined, and may be programmatically applied, by an entity such as VerticalPodAutoscaler (VPA).

    Note: While you can change a Pod’s requests and limits to express new desired resources, you cannot change the QoS class in which the Pod was created.

    Now, patch the Pod’s Container with CPU requests & limits both set to 800m:

    1. kubectl -n qos-example patch pod qos-demo-5 --patch '{"spec":{"containers":[{"name":"qos-demo-ctr-5", "resources":{"requests":{"cpu":"800m"}, "limits":{"cpu":"800m"}}}]}}'

    The Pod’s spec below reflects the updated CPU requests and limits.

    1. spec:
    2.   containers:
    3.     ...
    4.     resources:
    5.       limits:
    6.         cpu: 800m
    7.         memory: 200Mi
    8.       requests:
    9.         cpu: 800m
    10.         memory: 200Mi
    11. ...
    12.   containerStatuses:
    13. ...
    14.     allocatedResources:
    15.       cpu: 800m
    16.       memory: 200Mi
    17.     resources:
    18.       limits:
    19.         cpu: 800m
    20.         memory: 200Mi
    21.       requests:
    22.         cpu: 800m
    23.         memory: 200Mi
    24.     restartCount: 0
    25.     started: true

    Observe that the allocatedResources values have been updated to reflect the new desired CPU requests. This indicates that node was able to accommodate the increased CPU resource needs.

    In the Container’s status, updated CPU resource values shows that new CPU resources have been applied. The Container’s restartCount remains unchanged, indicating that container’s CPU resources were resized without restarting the container.

    Delete your namespace:

    1. kubectl delete namespace qos-example

    What’s next

    For cluster administrators