Viewing and listing the nodes in your OKD cluster

    When you perform node management operations, the CLI interacts with node objects that are representations of actual node hosts. The master uses the information from node objects to validate nodes with health checks.

    You can get detailed information on the nodes in the cluster.

    • The following command lists all nodes:

      The following example is a cluster with healthy nodes:

      Example output

      1. NAME                   STATUS    ROLES     AGE       VERSION
      2. master.example.com     Ready     master    7h        v1.26.0
      3. node1.example.com      Ready     worker    7h        v1.26.0
      4. node2.example.com      Ready     worker    7h        v1.26.0

      The following example is a cluster with one unhealthy node:

      1. $ oc get nodes

      Example output

      1. NAME                   STATUS                      ROLES     AGE       VERSION
      2. master.example.com     Ready                       master    7h        v1.26.0
      3. node1.example.com      NotReady,SchedulingDisabled worker    7h        v1.26.0
      4. node2.example.com      Ready                       worker    7h        v1.26.0

      The conditions that trigger a NotReady status are shown later in this section.

    • The -o wide option provides additional information on nodes.

      1. $ oc get nodes -o wide

      Example output

      1. NAME                STATUS   ROLES    AGE    VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINER-RUNTIME
      2. master.example.com  Ready    master   171m   v1.26.0   10.0.129.108   <none>        Red Hat Enterprise Linux CoreOS 48.83.202103210901-0 (Ootpa)   4.18.0-240.15.1.el8_3.x86_64   cri-o://1.26.0-30.rhaos4.10.gitf2f339d.el8-dev
      3. node1.example.com   Ready    worker   72m    v1.26.0   10.0.129.222   <none>        Red Hat Enterprise Linux CoreOS 48.83.202103210901-0 (Ootpa)   4.18.0-240.15.1.el8_3.x86_64   cri-o://1.26.0-30.rhaos4.10.gitf2f339d.el8-dev
      4. node2.example.com   Ready    worker   164m   v1.26.0   10.0.142.150   <none>        Red Hat Enterprise Linux CoreOS 48.83.202103210901-0 (Ootpa)   4.18.0-240.15.1.el8_3.x86_64   cri-o://1.26.0-30.rhaos4.10.gitf2f339d.el8-dev

    • The following command lists information about a single node:

      For example:

      1. $ oc get node node1.example.com

      Example output

      1. NAME                   STATUS    ROLES     AGE       VERSION
      2. node1.example.com      Ready     worker    7h        v1.26.0

    • The following command provides more detailed information about a specific node, including the reason for the current condition:

      1. $ oc describe node <node>

      For example:

      1. $ oc describe node node1.example.com
      1. Name:               node1.example.com (1)
      2. Roles:              worker (2)
      3. Labels:             beta.kubernetes.io/arch=amd64   (3)
      4.                     beta.kubernetes.io/instance-type=m4.large
      5.                     beta.kubernetes.io/os=linux
      6.                     failure-domain.beta.kubernetes.io/region=us-east-2
      7.                     failure-domain.beta.kubernetes.io/zone=us-east-2a
      8.                     kubernetes.io/hostname=ip-10-0-140-16
      9.                     node-role.kubernetes.io/worker=
      10. Annotations:        cluster.k8s.io/machine: openshift-machine-api/ahardin-worker-us-east-2a-q5dzc  (4)
      11.                     machineconfiguration.openshift.io/currentConfig: worker-309c228e8b3a92e2235edd544c62fea8
      12.                     machineconfiguration.openshift.io/desiredConfig: worker-309c228e8b3a92e2235edd544c62fea8
      13.                     machineconfiguration.openshift.io/state: Done
      14.                     volumes.kubernetes.io/controller-managed-attach-detach: true
      15. CreationTimestamp:  Wed, 13 Feb 2019 11:05:57 -0500
      16. Taints:             <none>  (5)
      17. Conditions:                 (6)
      18.   Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
      19.   ----             ------  -----------------                 ------------------                ------                       -------
      20.   OutOfDisk        False   Wed, 13 Feb 2019 15:09:42 -0500   Wed, 13 Feb 2019 11:05:57 -0500   KubeletHasSufficientDisk     kubelet has sufficient disk space available
      21.   MemoryPressure   False   Wed, 13 Feb 2019 15:09:42 -0500   Wed, 13 Feb 2019 11:05:57 -0500   KubeletHasSufficientMemory   kubelet has sufficient memory available
      22.   DiskPressure     False   Wed, 13 Feb 2019 15:09:42 -0500   Wed, 13 Feb 2019 11:05:57 -0500   KubeletHasNoDiskPressure     kubelet has no disk pressure
      23.   PIDPressure      False   Wed, 13 Feb 2019 15:09:42 -0500   Wed, 13 Feb 2019 11:05:57 -0500   KubeletHasSufficientPID      kubelet has sufficient PID available
      24.   Ready            True    Wed, 13 Feb 2019 15:09:42 -0500   Wed, 13 Feb 2019 11:07:09 -0500   KubeletReady                 kubelet is posting ready status
      25. Addresses:   (7)
      26.   InternalIP:   10.0.140.16
      27.   InternalDNS:  ip-10-0-140-16.us-east-2.compute.internal
      29. Capacity:    (8)
      30.  attachable-volumes-aws-ebs:  39
      31.  cpu:                         2
      32.  hugepages-1Gi:               0
      33.  hugepages-2Mi:               0
      34.  memory:                      8172516Ki
      35.  pods:                        250
      36. Allocatable:
      37.  attachable-volumes-aws-ebs:  39
      38.  cpu:                         1500m
      39.  hugepages-1Gi:               0
      40.  hugepages-2Mi:               0
      41.  memory:                      7558116Ki
      42.  pods:                        250
      43. System Info:    (9)
      44.  Machine ID:                              63787c9534c24fde9a0cde35c13f1f66
      45.  System UUID:                             EC22BF97-A006-4A58-6AF8-0A38DEEA122A
      46.  Boot ID:                                 f24ad37d-2594-46b4-8830-7f7555918325
      47.  Kernel Version:                          3.10.0-957.5.1.el7.x86_64
      48.  OS Image:                                Red Hat Enterprise Linux CoreOS 410.8.20190520.0 (Ootpa)
      49.  Operating System:                        linux
      50.  Architecture:                            amd64
      51.  Container Runtime Version:               cri-o://1.26.0-0.6.dev.rhaos4.3.git9ad059b.el8-rc2
      52.  Kubelet Version:                         v1.26.0
      53.  Kube-Proxy Version:                      v1.26.0
      54. PodCIDR:                                  10.128.4.0/24
      55. ProviderID:                               aws:///us-east-2a/i-04e87b31dc6b3e171
      56. Non-terminated Pods:                      (12 in total)  (10)
      57.   Namespace                               Name                                   CPU Requests  CPU Limits  Memory Requests  Memory Limits
      58.   ---------                               ----                                   ------------  ----------  ---------------  -------------
      59.   openshift-cluster-node-tuning-operator  tuned-hdl5q                            0 (0%)        0 (0%)      0 (0%)           0 (0%)
      60.   openshift-image-registry                node-ca-9hmcg                          0 (0%)        0 (0%)      0 (0%)           0 (0%)
      61.   openshift-ingress                       router-default-76455c45c-c5ptv         0 (0%)        0 (0%)      0 (0%)           0 (0%)
      62.   openshift-machine-config-operator       machine-config-daemon-cvqw9            20m (1%)      0 (0%)      50Mi (0%)        0 (0%)
      63.   openshift-marketplace                   community-operators-f67fh              0 (0%)        0 (0%)      0 (0%)           0 (0%)
      64.   openshift-monitoring                    alertmanager-main-0                    50m (3%)      50m (3%)    210Mi (2%)       10Mi (0%)
      65.   openshift-monitoring                    node-exporter-l7q8d                    10m (0%)      20m (1%)    20Mi (0%)        40Mi (0%)
      66.   openshift-monitoring                    prometheus-adapter-75d769c874-hvb85    0 (0%)        0 (0%)      0 (0%)           0 (0%)
      67.   openshift-multus                        multus-kw8w5                           0 (0%)        0 (0%)      0 (0%)           0 (0%)
      68.   openshift-sdn                           ovs-t4dsn                              100m (6%)     0 (0%)      300Mi (4%)       0 (0%)
      69.   openshift-sdn                           sdn-g79hg                              100m (6%)     0 (0%)      200Mi (2%)       0 (0%)
      70. Allocated resources:
      71.   (Total limits may be over 100 percent, i.e., overcommitted.)
      72.   Resource                    Requests     Limits
      73.   --------                    --------     ------
      74.   cpu                         380m (25%)   270m (18%)
      75.   memory                      880Mi (11%)  250Mi (3%)
      76.   attachable-volumes-aws-ebs  0            0
      78.   Type     Reason                   Age                From                      Message
      79.   ----     ------                   ----               ----                      -------
      80.   Normal   NodeHasSufficientPID     6d (x5 over 6d)    kubelet, m01.example.com  Node m01.example.com status is now: NodeHasSufficientPID
      81.   Normal   NodeAllocatableEnforced  6d                 kubelet, m01.example.com  Updated Node Allocatable limit across pods
      82.   Normal   NodeHasSufficientMemory  6d (x6 over 6d)    kubelet, m01.example.com  Node m01.example.com status is now: NodeHasSufficientMemory
      83.   Normal   NodeHasNoDiskPressure    6d (x6 over 6d)    kubelet, m01.example.com  Node m01.example.com status is now: NodeHasNoDiskPressure
      84.   Normal   NodeHasSufficientDisk    6d (x6 over 6d)    kubelet, m01.example.com  Node m01.example.com status is now: NodeHasSufficientDisk
      85.   Normal   NodeHasSufficientPID     6d                 kubelet, m01.example.com  Node m01.example.com status is now: NodeHasSufficientPID
      86.   Normal   Starting                 6d                 kubelet, m01.example.com  Starting kubelet.
      87.  ...

    Among the information shown for nodes, the following node conditions appear in the output of the commands shown in this section:

    Table 1. Node Conditions
    ConditionDescription

    Ready

    If true, the node is healthy and ready to accept pods. If false, the node is not healthy and is not accepting pods. If unknown, the node controller has not received a heartbeat from the node for the node-monitor-grace-period (the default is 40 seconds).

    DiskPressure

    If true, the disk capacity is low.

    MemoryPressure

    If true, the node memory is low.

    PIDPressure

    If true, there are too many processes on the node.

    OutOfDisk

    If true, the node has insufficient free space on the node for adding new pods.

    NetworkUnavailable

    If true, the network for the node is not correctly configured.

    NotReady

    If true, one of the underlying components, such as the container runtime or network, is experiencing issues or is not yet configured.

    Pods cannot be scheduled for placement on the node.

    You can list all the pods on a specific node.

    Procedure

    • To list all or selected pods on one or more nodes:

      1. $ oc describe node <node1> <node2>

      For example:

    • To list all or selected pods on selected nodes:

      1. $ oc describe --selector=<node_selector>
      1. $ oc describe node  --selector=kubernetes.io/os

      Or:

      1. $ oc describe -l=<pod_selector>
      1. $ oc describe node -l node-role.kubernetes.io/worker

    • To list all pods on a specific node, including terminated pods:

      1. $ oc get pod --all-namespaces --field-selector=spec.nodeName=<nodename>

    You can display usage statistics about nodes, which provide the runtime environments for containers. These usage statistics include CPU, memory, and storage consumption.

    Prerequisites

    • You must have cluster-reader permission to view the usage statistics.

    • Metrics must be installed to view the usage statistics.

    Procedure

    • To view the usage statistics:

      1. $ oc adm top nodes

      Example output

      1. $ oc adm top node --selector=''

      You must choose the selector (label query) to filter on. Supports =, ==, and !=.