Recommended single-node OpenShift cluster configuration for vDU application workloads

    Additional resources

    OKD enables low latency processing for applications running on commercial off-the-shelf (COTS) hardware by using several technologies and specialized hardware devices:

    Real-time kernel for RHCOS

    Ensures workloads are handled with a high degree of process determinism.

    CPU isolation

    Avoids CPU scheduling delays and ensures CPU capacity is available consistently.

    NUMA-aware topology management

    Aligns memory and huge pages with CPU and PCI devices to pin guaranteed container memory and huge pages to the non-uniform memory access (NUMA) node. Pod resources for all Quality of Service (QoS) classes stay on the same NUMA node. This decreases latency and improves performance of the node.

    Huge pages memory management

    Using huge page sizes improves system performance by reducing the amount of system resources required to access page tables.

    Precision timing synchronization using PTP

    Allows synchronization between nodes in the network with sub-microsecond accuracy.

    Running vDU application workloads requires a bare-metal host with sufficient resources to run OKD services and production workloads.

    One vCPU is equivalent to one physical core when simultaneous multithreading (SMT), or Hyper-Threading, is not enabled. When enabled, use the following formula to calculate the corresponding ratio:

    • (threads per core × cores) × sockets = vCPUs

    The server must have a Baseboard Management Controller (BMC) when booting with virtual media.

    Bare-metal hosts require the firmware to be configured before the host can be provisioned. The firmware configuration is dependent on the specific hardware and the particular requirements of your installation.

    Procedure

    1. Set the UEFI/BIOS Boot Mode to .

    2. In the host boot sequence order, set Hard drive first.

    3. Apply the specific firmware configuration for your hardware. The following table describes a representative firmware configuration for an Intel Xeon Skylake or Intel Cascade Lake server, based on the Intel FlexRAN 4G and 5G baseband PHY reference design.

      The exact firmware configuration depends on your specific hardware and network requirements. The following sample configuration is for illustrative purposes only.

      Table 2. Sample firmware configuration for an Intel Xeon Skylake or Cascade Lake server
      Firmware settingConfiguration

      CPU Power and Performance Policy

      Performance

      Uncore Frequency Scaling

      Disabled

      Performance P-limit

      Disabled

      Enhanced Intel SpeedStep ® Tech

      Enabled

      Intel Configurable TDP

      Enabled

      Configurable TDP Level

      Level 2

      Intel® Turbo Boost Technology

      Enabled

      Energy Efficient Turbo

      Disabled

      Disabled

      Package C-State

      C0/C1 state

      C1E

      Disabled

      Processor C6

      Disabled

    Enable global SR-IOV and VT-d settings in the firmware for the host. These settings are relevant to bare-metal environments.

    Connectivity prerequisites for managed cluster networks

    Before you can install and provision a managed cluster with the GitOps Zero Touch Provisioning (ZTP) pipeline, the managed cluster host must meet the following networking prerequisites:

    • There must be bi-directional connectivity between the GitOps ZTP container in the hub cluster and the Baseboard Management Controller (BMC) of the target bare-metal host.

    • The managed cluster must be able to resolve and reach the API hostname of the hub hostname and *.apps hostname. Here is an example of the API hostname of the hub and *.apps hostname:

      • api.hub-cluster.internal.domain.com

      • console-openshift-console.apps.hub-cluster.internal.domain.com

    • The hub cluster must be able to resolve and reach the API and *.apps hostname of the managed cluster. Here is an example of the API hostname of the managed cluster and *.apps hostname:

      • api.sno-managed-cluster-1.internal.domain.com

      • console-openshift-console.apps.sno-managed-cluster-1.internal.domain.com

    Workload partitioning configures OKD services, cluster management workloads, and infrastructure pods to run on a reserved number of host CPUs.

    To configure workload partitioning with GitOps Zero Touch Provisioning (ZTP), you specify cluster management CPU resources with the cpuset field of the SiteConfig custom resource (CR) and the reserved field of the group PolicyGenTemplate CR. The GitOps ZTP pipeline uses these values to populate the required fields in the workload partitioning MachineConfig CR (cpuset) and the PerformanceProfile CR (reserved) that configure the single-node OpenShift cluster.

    For maximum performance, ensure that the reserved and isolated CPU sets do not share CPU cores across NUMA zones.

    • The workload partitioning MachineConfig CR pins the OKD infrastructure pods to a defined cpuset configuration.

    • The PerformanceProfile CR pins the systemd services to the reserved CPUs.

    Additional resources

    • For the recommended single-node OpenShift workload partitioning configuration, see .

    The ZTP pipeline applies the following custom resources (CRs) during cluster installation. These configuration CRs ensure that the cluster meets the feature and performance requirements necessary for running a vDU application.

    When using the GitOps ZTP plugin and SiteConfig CRs for cluster deployment, the following MachineConfig CRs are included by default.

    Use the SiteConfig extraManifests filter to alter the CRs that are included by default. For more information, see .

    Single-node OpenShift clusters that run DU workloads require workload partitioning. This limits the cores allowed to run platform services, maximizing the CPU core for application payloads.

    Workload partitioning can only be enabled during cluster installation. You cannot disable workload partitioning post-installation. However, you can reconfigure workload partitioning by updating the cpu value that you define in the performance profile, and in the related MachineConfig custom resource (CR).

    • The base64-encoded CR that enables workload partitioning contains the CPU set that the management workloads are constrained to. Encode host-specific values for crio.conf and kubelet.conf in base64. Adjust the content to match the CPU set that is specified in the cluster performance profile. It must match the number of cores in the cluster host.

      Recommended workload partitioning configuration

    • When configured in the cluster host, the contents of /etc/crio/crio.conf.d/01-workload-partitioning should look like this:

      1. [crio.runtime.workloads.management]
      2. activation_annotation = "target.workload.openshift.io/management"
      3. annotation_prefix = "resources.workload.openshift.io"
      4. resources = { "cpushares" = 0, "cpuset" = "0-1,52-53" } (1)
      1The cpuset value varies based on the installation. If Hyper-Threading is enabled, specify both threads for each core. The cpuset value must match the reserved CPUs that you define in the spec.cpu.reserved field in the performance profile.

    • When configured in the cluster, the contents of /etc/kubernetes/openshift-workload-pinning should look like this:

      1. {
      2.   "management": {
      3.     "cpuset": "0-1,52-53" (1)
      4.   }
      5. }
      1The cpuset must match the cpuset value in /etc/crio/crio.conf.d/01-workload-partitioning.

    Verification

    Check that the applications and cluster system CPU pinning is correct. Run the following commands:

    1. Open a remote shell connection to the managed cluster:

      1. $ oc debug node/example-sno-1

    2. Check that the user applications CPU pinning is correct:

      1. sh-4.4# pgrep ovn | while read i; do taskset -cp $i; done

      Example output

      1. pid 8481's current affinity list: 0-3
      2. pid 8726's current affinity list: 0-3
      3. pid 9088's current affinity list: 0-3
      4. pid 9945's current affinity list: 0-3
      5. pid 10387's current affinity list: 0-3
      6. pid 12123's current affinity list: 0-3
      7. pid 13313's current affinity list: 0-3

    3. Check that the system applications CPU pinning is correct:

      1. sh-4.4# pgrep systemd | while read i; do taskset -cp $i; done

      Example output

      1. pid 1's current affinity list: 0-3
      2. pid 938's current affinity list: 0-3
      3. pid 962's current affinity list: 0-3
      4. pid 1197's current affinity list: 0-3

    Reduced platform management footprint

    To reduce the overall management footprint of the platform, a MachineConfig custom resource (CR) is required that places all Kubernetes-specific mount points in a new namespace separate from the host operating system. The following base64-encoded example MachineConfig CR illustrates this configuration.

    Recommended container mount namespace configuration

    SCTP

    Stream Control Transmission Protocol (SCTP) is a key protocol used in RAN applications. This MachineConfig object adds the SCTP kernel module to the node to enable this protocol.

    Recommended SCTP configuration

    1. apiVersion: machineconfiguration.openshift.io/v1
    2. kind: MachineConfig
    3. metadata:
    4.   labels:
    5.     machineconfiguration.openshift.io/role: master
    6.   name: load-sctp-module
    7. spec:
    8.   config:
    9.     ignition:
    10.       version: 2.2.0
    11.     storage:
    12.       files:
    13.         - contents:
    14.             source: data:,
    15.             verification: {}
    16.           filesystem: root
    17.             mode: 420
    18.             path: /etc/modprobe.d/sctp-blacklist.conf
    19.         - contents:
    20.             source: data:text/plain;charset=utf-8,sctp
    21.           filesystem: root
    22.             mode: 420
    23.             path: /etc/modules-load.d/sctp-load.conf

    Accelerated container startup

    The following MachineConfig CR configures core OpenShift processes and containers to use all available CPU cores during system startup and shutdown. This accelerates the system recovery during initial boot and reboots.

    1. apiVersion: machineconfiguration.openshift.io/v1
    2. kind: MachineConfig
    3. metadata:
    4.   labels:
    5.     machineconfiguration.openshift.io/role: master
    6.   name: 04-accelerated-container-startup-master
    7. spec:
    8.   config:
    9.     ignition:
    10.       version: 3.2.0
    11.     storage:
    12.       files:
    13.       - contents:
    14.           source: data:text/plain;charset=utf-8;base64,IyEvYmluL2Jhc2gKIwojIFRlbXBvcmFyaWx5IHJlc2V0IHRoZSBjb3JlIHN5c3RlbSBwcm9jZXNzZXMncyBDUFUgYWZmaW5pdHkgdG8gYmUgdW5yZXN0cmljdGVkIHRvIGFjY2VsZXJhdGUgc3RhcnR1cCBhbmQgc2h1dGRvd24KIwojIFRoZSBkZWZhdWx0cyBiZWxvdyBjYW4gYmUgb3ZlcnJpZGRlbiB2aWEgZW52aXJvbm1lbnQgdmFyaWFibGVzCiMKCiMgVGhlIGRlZmF1bHQgc2V0IG9mIGNyaXRpY2FsIHByb2Nlc3NlcyB3aG9zZSBhZmZpbml0eSBzaG91bGQgYmUgdGVtcG9yYXJpbHkgdW5ib3VuZDoKQ1JJVElDQUxfUFJPQ0VTU0VTPSR7Q1JJVElDQUxfUFJPQ0VTU0VTOi0ic3lzdGVtZCBvdnMgY3JpbyBrdWJlbGV0IE5ldHdvcmtNYW5hZ2VyIGNvbm1vbiBkYnVzIn0KCiMgRGVmYXVsdCB3YWl0IHRpbWUgaXMgNjAwcyA9IDEwbToKTUFYSU1VTV9XQUlUX1RJTUU9JHtNQVhJTVVNX1dBSVRfVElNRTotNjAwfQoKIyBEZWZhdWx0IHN0ZWFkeS1zdGF0ZSB0aHJlc2hvbGQgPSAyJQojIEFsbG93ZWQgdmFsdWVzOgojICA0ICAtIGFic29sdXRlIHBvZCBjb3VudCAoKy8tKQojICA0JSAtIHBlcmNlbnQgY2hhbmdlICgrLy0pCiMgIC0xIC0gZGlzYWJsZSB0aGUgc3RlYWR5LXN0YXRlIGNoZWNrClNURUFEWV9TVEFURV9USFJFU0hPTEQ9JHtTVEVBRFlfU1RBVEVfVEhSRVNIT0xEOi0yJX0KCiMgRGVmYXVsdCBzdGVhZHktc3RhdGUgd2luZG93ID0gNjBzCiMgSWYgdGhlIHJ1bm5pbmcgcG9kIGNvdW50IHN0YXlzIHdpdGhpbiB0aGUgZ2l2ZW4gdGhyZXNob2xkIGZvciB0aGlzIHRpbWUKIyBwZXJpb2QsIHJldHVybiBDUFUgdXRpbGl6YXRpb24gdG8gbm9ybWFsIGJlZm9yZSB0aGUgbWF4aW11bSB3YWl0IHRpbWUgaGFzCiMgZXhwaXJlcwpTVEVBRFlfU1RBVEVfV0lORE9XPSR7U1RFQURZX1NUQVRFX1dJTkRPVzotNjB9CgojIERlZmF1bHQgc3RlYWR5LXN0YXRlIGFsbG93cyBhbnkgcG9kIGNvdW50IHRvIGJlICJzdGVhZHkgc3RhdGUiCiMgSW5jcmVhc2luZyB0aGlzIHdpbGwgc2tpcCBhbnkgc3RlYWR5LXN0YXRlIGNoZWNrcyB1bnRpbCB0aGUgY291bnQgcmlzZXMgYWJvdmUKIyB0aGlzIG51bWJlciB0byBhdm9pZCBmYWxzZSBwb3NpdGl2ZXMgaWYgdGhlcmUgYXJlIHNvbWUgcGVyaW9kcyB3aGVyZSB0aGUKIyBjb3VudCBkb2Vzbid0IGluY3JlYXNlIGJ1dCB3ZSBrbm93IHdlIGNhbid0IGJlIGF0IHN0ZWFkeS1zdGF0ZSB5ZXQuClNURUFEWV9TVEFURV9NSU5JTVVNPSR7U1RFQURZX1NUQVRFX01JTklNVU06LTB9CgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgpLVUJFTEVUX0NQVV9TVEFURT0vdmFyL2xpYi9rdWJlbGV0L2NwdV9tYW5hZ2VyX3N0YXRlCkZVTExfQ1BVX1NUQVRFPS9zeXMvZnMvY2dyb3VwL2NwdXNldC9jcHVzZXQuY3B1cwp1bnJlc3RyaWN0ZWRDcHVzZXQoKSB7CiAgbG9jYWwgY3B1cwogIGlmIFtbIC1lICRLVUJFTEVUX0NQVV9TVEFURSBdXTsgdGhlbgogICAgICBjcHVzPSQoanEgLXIgJy5kZWZhdWx0Q3B1U2V0JyA8JEtVQkVMRVRfQ1BVX1NUQVRFKQogIGZpCiAgaWYgW1sgLXogJGNwdXMgXV07IHRoZW4KICAgICMgZmFsbCBiYWNrIHRvIHVzaW5nIGFsbCBjcHVzIGlmIHRoZSBrdWJlbGV0IHN0YXRlIGlzIG5vdCBjb25maWd1cmVkIHlldAogICAgW1sgLWUgJEZVTExfQ1BVX1NUQVRFIF1dIHx8IHJldHVybiAxCiAgICBjcHVzPSQoPCRGVUxMX0NQVV9TVEFURSkKICBmaQogIGVjaG8gJGNwdXMKfQoKcmVzdHJpY3RlZENwdXNldCgpIHsKICBmb3IgYXJnIGluICQoPC9wcm9jL2NtZGxpbmUpOyBkbwogICAgaWYgW1sgJGFyZyA9fiBec3lzdGVtZC5jcHVfYWZmaW5pdHk9IF1dOyB0aGVuCiAgICAgIGVjaG8gJHthcmcjKj19CiAgICAgIHJldHVybiAwCiAgICBmaQogIGRvbmUKICByZXR1cm4gMQp9CgpnZXRDUFVDb3VudCAoKSB7CiAgbG9jYWwgY3B1c2V0PSIkMSIKICBsb2NhbCBjcHVsaXN0PSgpCiAgbG9jYWwgY3B1cz0wCiAgbG9jYWwgbWluY3B1cz0yCgogIGlmIFtbIC16ICRjcHVzZXQgfHwgJGNwdXNldCA9fiBbXjAtOSwtXSBdXTsgdGhlbgogICAgZWNobyAkbWluY3B1cwogICAgcmV0dXJuIDEKICBmaQoKICBJRlM9JywnIHJlYWQgLXJhIGNwdWxpc3QgPDw8ICRjcHVzZXQKCiAgZm9yIGVsbSBpbiAiJHtjcHVsaXN0W0BdfSI7IGRvCiAgICBpZiBbWyAkZWxtID1+IF5bMC05XSskIF1dOyB0aGVuCiAgICAgICgoIGNwdXMrKyApKQogICAgZWxpZiBbWyAkZWxtID1+IF5bMC05XSstWzAtOV0rJCBdXTsgdGhlbgogICAgICBsb2NhbCBsb3c9MCBoaWdoPTAKICAgICAgSUZTPSctJyByZWFkIGxvdyBoaWdoIDw8PCAkZWxtCiAgICAgICgoIGNwdXMgKz0gaGlnaCAtIGxvdyArIDEgKSkKICAgIGVsc2UKICAgICAgZWNobyAkbWluY3B1cwogICAgICByZXR1cm4gMQogICAgZmkKICBkb25lCgogICMgUmV0dXJuIGEgbWluaW11bSBvZiAyIGNwdXMKICBlY2hvICQoKCBjcHVzID4gJG1pbmNwdXMgPyBjcHVzIDogJG1pbmNwdXMgKSkKICByZXR1cm4gMAp9CgpyZXNldE9WU3RocmVhZHMgKCkgewogIGxvY2FsIGNwdWNvdW50PSIkMSIKICBsb2NhbCBjdXJSZXZhbGlkYXRvcnM9MAogIGxvY2FsIGN1ckhhbmRsZXJzPTAKICBsb2NhbCBkZXNpcmVkUmV2YWxpZGF0b3JzPTAKICBsb2NhbCBkZXNpcmVkSGFuZGxlcnM9MAogIGxvY2FsIHJjPTAKCiAgY3VyUmV2YWxpZGF0b3JzPSQocHMgLVRlbyBwaWQsdGlkLGNvbW0sY21kIHwgZ3JlcCAtZSByZXZhbGlkYXRvciB8IGdyZXAgLWMgb3ZzLXZzd2l0Y2hkKQogIGN1ckhhbmRsZXJzPSQocHMgLVRlbyBwaWQsdGlkLGNvbW0sY21kIHwgZ3JlcCAtZSBoYW5kbGVyIHwgZ3JlcCAtYyBvdnMtdnN3aXRjaGQpCgogICMgQ2FsY3VsYXRlIHRoZSBkZXNpcmVkIG51bWJlciBvZiB0aHJlYWRzIHRoZSBzYW1lIHdheSBPVlMgZG9lcy4KICAjIE9WUyB3aWxsIHNldCB0aGVzZSB0aHJlYWQgY291bnQgYXMgYSBvbmUgc2hvdCBwcm9jZXNzIG9uIHN0YXJ0dXAsIHNvIHdlCiAgIyBoYXZlIHRvIGFkanVzdCB1cCBvciBkb3duIGR1cmluZyB0aGUgYm9vdCB1cCBwcm9jZXNzLiBUaGUgZGVzaXJlZCBvdXRjb21lIGlzCiAgIyB0byBub3QgcmVzdHJpY3QgdGhlIG51bWJlciBvZiB0aHJlYWQgYXQgc3RhcnR1cCB1bnRpbCB3ZSByZWFjaCBhIHN0ZWFkeQogICMgc3RhdGUuICBBdCB3aGljaCBwb2ludCB3ZSBuZWVkIHRvIHJlc2V0IHRoZXNlIGJhc2VkIG9uIG91ciByZXN0cmljdGVkICBzZXQKICAjIG9mIGNvcmVzLgogICMgU2VlIE9WUyBmdW5jdGlvbiB0aGF0IGNhbGN1bGF0ZXMgdGhlc2UgdGhyZWFkIGNvdW50czoKICAjIGh0dHBzOi8vZ2l0aHViLmNvbS9vcGVudnN3aXRjaC9vdnMvYmxvYi9tYXN0ZXIvb2Zwcm90by9vZnByb3RvLWRwaWYtdXBjYWxsLmMjTDYzNQogICgoIGRlc2lyZWRSZXZhbGlkYXRvcnM9JGNwdWNvdW50IC8gNCArIDEgKSkKICAoKCBkZXNpcmVkSGFuZGxlcnM9JGNwdWNvdW50IC0gJGRlc2lyZWRSZXZhbGlkYXRvcnMgKSkKCgogIGlmIFtbICRjdXJSZXZhbGlkYXRvcnMgLW5lICRkZXNpcmVkUmV2YWxpZGF0b3JzIHx8ICRjdXJIYW5kbGVycyAtbmUgJGRlc2lyZWRIYW5kbGVycyBdXTsgdGhlbgoKICAgIGxvZ2dlciAiUmVjb3Zlcnk6IFJlLXNldHRpbmcgT1ZTIHJldmFsaWRhdG9yIHRocmVhZHM6ICR7Y3VyUmV2YWxpZGF0b3JzfSAtPiAke2Rlc2lyZWRSZXZhbGlkYXRvcnN9IgogICAgbG9nZ2VyICJSZWNvdmVyeTogUmUtc2V0dGluZyBPVlMgaGFuZGxlciB0aHJlYWRzOiAke2N1ckhhbmRsZXJzfSAtPiAke2Rlc2lyZWRIYW5kbGVyc30iCgogICAgb3ZzLXZzY3RsIHNldCBcCiAgICAgIE9wZW5fdlN3aXRjaCAuIFwKICAgICAgb3RoZXItY29uZmlnOm4taGFuZGxlci10aHJlYWRzPSR7ZGVzaXJlZEhhbmRsZXJzfSBcCiAgICAgIG90aGVyLWNvbmZpZzpuLXJldmFsaWRhdG9yLXRocmVhZHM9JHtkZXNpcmVkUmV2YWxpZGF0b3JzfQogICAgcmM9JD8KICBmaQoKICByZXR1cm4gJHJjCn0KCnJlc2V0QWZmaW5pdHkoKSB7CiAgbG9jYWwgY3B1c2V0PSIkMSIKICBsb2NhbCBmYWlsY291bnQ9MAogIGxvY2FsIHN1Y2Nlc3Njb3VudD0wCiAgbG9nZ2VyICJSZWNvdmVyeTogU2V0dGluZyBDUFUgYWZmaW5pdHkgZm9yIGNyaXRpY2FsIHByb2Nlc3NlcyBcIiRDUklUSUNBTF9QUk9DRVNTRVNcIiB0byAkY3B1c2V0IgogIGZvciBwcm9jIGluICRDUklUSUNBTF9QUk9DRVNTRVM7IGRvCiAgICBsb2NhbCBwaWRzPSIkKHBncmVwICRwcm9jKSIKICAgIGZvciBwaWQgaW4gJHBpZHM7IGRvCiAgICAgIGxvY2FsIHRhc2tzZXRPdXRwdXQKICAgICAgdGFza3NldE91dHB1dD0iJCh0YXNrc2V0IC1hcGMgIiRjcHVzZXQiICRwaWQgMj4mMSkiCiAgICAgIGlmIFtbICQ/IC1uZSAwIF1dOyB0aGVuCiAgICAgICAgZWNobyAiRVJST1I6ICR0YXNrc2V0T3V0cHV0IgogICAgICAgICgoZmFpbGNvdW50KyspKQogICAgICBlbHNlCiAgICAgICAgKChzdWNjZXNzY291bnQrKykpCiAgICAgIGZpCiAgICBkb25lCiAgZG9uZQoKICByZXNldE9WU3RocmVhZHMgIiQoZ2V0Q1BVQ291bnQgJHtjcHVzZXR9KSIKICBpZiBbWyAkPyAtbmUgMCBdXTsgdGhlbgogICAgKChmYWlsY291bnQrKykpCiAgZWxzZQogICAgKChzdWNjZXNzY291bnQrKykpCiAgZmkKCiAgbG9nZ2VyICJSZWNvdmVyeTogUmUtYWZmaW5lZCAkc3VjY2Vzc2NvdW50IHBpZHMgc3VjY2Vzc2Z1bGx5IgogIGlmIFtbICRmYWlsY291bnQgLWd0IDAgXV07IHRoZW4KICAgIGxvZ2dlciAiUmVjb3Zlcnk6IEZhaWxlZCB0byByZS1hZmZpbmUgJGZhaWxjb3VudCBwcm9jZXNzZXMiCiAgICByZXR1cm4gMQogIGZpCn0KCnNldFVucmVzdHJpY3RlZCgpIHsKICBsb2dnZXIgIlJlY292ZXJ5OiBTZXR0aW5nIGNyaXRpY2FsIHN5c3RlbSBwcm9jZXNzZXMgdG8gaGF2ZSB1bnJlc3RyaWN0ZWQgQ1BVIGFjY2VzcyIKICByZXNldEFmZmluaXR5ICIkKHVucmVzdHJpY3RlZENwdXNldCkiCn0KCnNldFJlc3RyaWN0ZWQoKSB7CiAgbG9nZ2VyICJSZWNvdmVyeTogUmVzZXR0aW5nIGNyaXRpY2FsIHN5c3RlbSBwcm9jZXNzZXMgYmFjayB0byBub3JtYWxseSByZXN0cmljdGVkIGFjY2VzcyIKICByZXNldEFmZmluaXR5ICIkKHJlc3RyaWN0ZWRDcHVzZXQpIgp9CgpjdXJyZW50QWZmaW5pdHkoKSB7CiAgbG9jYWwgcGlkPSIkMSIKICB0YXNrc2V0IC1wYyAkcGlkIHwgYXdrIC1GJzogJyAne3ByaW50ICQyfScKfQoKd2l0aGluKCkgewogIGxvY2FsIGxhc3Q9JDEgY3VycmVudD0kMiB0aHJlc2hvbGQ9JDMKICBsb2NhbCBkZWx0YT0wIHBjaGFuZ2UKICBkZWx0YT0kKCggY3VycmVudCAtIGxhc3QgKSkKICBpZiBbWyAkY3VycmVudCAtZXEgJGxhc3QgXV07IHRoZW4KICAgIHBjaGFuZ2U9MAogIGVsaWYgW1sgJGxhc3QgLWVxIDAgXV07IHRoZW4KICAgIHBjaGFuZ2U9MTAwMDAwMAogIGVsc2UKICAgIHBjaGFuZ2U9JCgoICggJGRlbHRhICogMTAwKSAvIGxhc3QgKSkKICBmaQogIGVjaG8gLW4gImxhc3Q6JGxhc3QgY3VycmVudDokY3VycmVudCBkZWx0YTokZGVsdGEgcGNoYW5nZToke3BjaGFuZ2V9JTogIgogIGxvY2FsIGFic29sdXRlIGxpbWl0CiAgY2FzZSAkdGhyZXNob2xkIGluCiAgICAqJSkKICAgICAgYWJzb2x1dGU9JHtwY2hhbmdlIyMtfSAjIGFic29sdXRlIHZhbHVlCiAgICAgIGxpbWl0PSR7dGhyZXNob2xkJSUlfQogICAgICA7OwogICAgKikKICAgICAgYWJzb2x1dGU9JHtkZWx0YSMjLX0gIyBhYnNvbHV0ZSB2YWx1ZQogICAgICBsaW1pdD0kdGhyZXNob2xkCiAgICAgIDs7CiAgZXNhYwogIGlmIFtbICRhYnNvbHV0ZSAtbGUgJGxpbWl0IF1dOyB0aGVuCiAgICBlY2hvICJ3aXRoaW4gKCsvLSkkdGhyZXNob2xkIgogICAgcmV0dXJuIDAKICBlbHNlCiAgICBlY2hvICJvdXRzaWRlICgrLy0pJHRocmVzaG9sZCIKICAgIHJldHVybiAxCiAgZmkKfQoKc3RlYWR5c3RhdGUoKSB7CiAgbG9jYWwgbGFzdD0kMSBjdXJyZW50PSQyCiAgaWYgW1sgJGxhc3QgLWx0ICRTVEVBRFlfU1RBVEVfTUlOSU1VTSBdXTsgdGhlbgogICAgZWNobyAibGFzdDokbGFzdCBjdXJyZW50OiRjdXJyZW50IFdhaXRpbmcgdG8gcmVhY2ggJFNURUFEWV9TVEFURV9NSU5JTVVNIGJlZm9yZSBjaGVja2luZyBmb3Igc3RlYWR5LXN0YXRlIgogICAgcmV0dXJuIDEKICBmaQogIHdpdGhpbiAkbGFzdCAkY3VycmVudCAkU1RFQURZX1NUQVRFX1RIUkVTSE9MRAp9Cgp3YWl0Rm9yUmVhZHkoKSB7CiAgbG9nZ2VyICJSZWNvdmVyeTogV2FpdGluZyAke01BWElNVU1fV0FJVF9USU1FfXMgZm9yIHRoZSBpbml0aWFsaXphdGlvbiB0byBjb21wbGV0ZSIKICBsb2NhbCBsYXN0U3lzdGVtZENwdXNldD0iJChjdXJyZW50QWZmaW5pdHkgMSkiCiAgbG9jYWwgbGFzdERlc2lyZWRDcHVzZXQ9IiQodW5yZXN0cmljdGVkQ3B1c2V0KSIKICBsb2NhbCB0PTAgcz0xMAogIGxvY2FsIGxhc3RDY291bnQ9MCBjY291bnQ9MCBzdGVhZHlTdGF0ZVRpbWU9MAogIHdoaWxlIFtbICR0IC1sdCAkTUFYSU1VTV9XQUlUX1RJTUUgXV07IGRvCiAgICBzbGVlcCAkcwogICAgKCh0ICs9IHMpKQogICAgIyBSZS1jaGVjayB0aGUgY3VycmVudCBhZmZpbml0eSBvZiBzeXN0ZW1kLCBpbiBjYXNlIHNvbWUgb3RoZXIgcHJvY2VzcyBoYXMgY2hhbmdlZCBpdAogICAgbG9jYWwgc3lzdGVtZENwdXNldD0iJChjdXJyZW50QWZmaW5pdHkgMSkiCiAgICAjIFJlLWNoZWNrIHRoZSB1bnJlc3RyaWN0ZWQgQ3B1c2V0LCBhcyB0aGUgYWxsb3dlZCBzZXQgb2YgdW5yZXNlcnZlZCBjb3JlcyBtYXkgY2hhbmdlIGFzIHBvZHMgYXJlIGFzc2lnbmVkIHRvIGNvcmVzCiAgICBsb2NhbCBkZXNpcmVkQ3B1c2V0PSIkKHVucmVzdHJpY3RlZENwdXNldCkiCiAgICBpZiBbWyAkc3lzdGVtZENwdXNldCAhPSAkbGFzdFN5c3RlbWRDcHVzZXQgfHwgJGxhc3REZXNpcmVkQ3B1c2V0ICE9ICRkZXNpcmVkQ3B1c2V0IF1dOyB0aGVuCiAgICAgIHJlc2V0QWZmaW5pdHkgIiRkZXNpcmVkQ3B1c2V0IgogICAgICBsYXN0U3lzdGVtZENwdXNldD0iJChjdXJyZW50QWZmaW5pdHkgMSkiCiAgICAgIGxhc3REZXNpcmVkQ3B1c2V0PSIkZGVzaXJlZENwdXNldCIKICAgIGZpCgogICAgIyBEZXRlY3Qgc3RlYWR5LXN0YXRlIHBvZCBjb3VudAogICAgY2NvdW50PSQoY3JpY3RsIHBzIHwgd2MgLWwpCiAgICBpZiBzdGVhZHlzdGF0ZSAkbGFzdENjb3VudCAkY2NvdW50OyB0aGVuCiAgICAgICgoc3RlYWR5U3RhdGVUaW1lICs9IHMpKQogICAgICBlY2hvICJTdGVhZHktc3RhdGUgZm9yICR7c3RlYWR5U3RhdGVUaW1lfXMvJHtTVEVBRFlfU1RBVEVfV0lORE9XfXMiCiAgICAgIGlmIFtbICRzdGVhZHlTdGF0ZVRpbWUgLWdlICRTVEVBRFlfU1RBVEVfV0lORE9XIF1dOyB0aGVuCiAgICAgICAgbG9nZ2VyICJSZWNvdmVyeTogU3RlYWR5LXN0YXRlICgrLy0gJFNURUFEWV9TVEFURV9USFJFU0hPTEQpIGZvciAke1NURUFEWV9TVEFURV9XSU5ET1d9czogRG9uZSIKICAgICAgICByZXR1cm4gMAogICAgICBmaQogICAgZWxzZQogICAgICBpZiBbWyAkc3RlYWR5U3RhdGVUaW1lIC1ndCAwIF1dOyB0aGVuCiAgICAgICAgZWNobyAiUmVzZXR0aW5nIHN0ZWFkeS1zdGF0ZSB0aW1lciIKICAgICAgICBzdGVhZHlTdGF0ZVRpbWU9MAogICAgICBmaQogICAgZmkKICAgIGxhc3RDY291bnQ9JGNjb3VudAogIGRvbmUKICBsb2dnZXIgIlJlY292ZXJ5OiBSZWNvdmVyeSBDb21wbGV0ZSBUaW1lb3V0Igp9CgptYWluKCkgewogIGlmICEgdW5yZXN0cmljdGVkQ3B1c2V0ID4mL2Rldi9udWxsOyB0aGVuCiAgICBsb2dnZXIgIlJlY292ZXJ5OiBObyB1bnJlc3RyaWN0ZWQgQ3B1c2V0IGNvdWxkIGJlIGRldGVjdGVkIgogICAgcmV0dXJuIDEKICBmaQoKICBpZiAhIHJlc3RyaWN0ZWRDcHVzZXQgPiYvZGV2L251bGw7IHRoZW4KICAgIGxvZ2dlciAiUmVjb3Zlcnk6IE5vIHJlc3RyaWN0ZWQgQ3B1c2V0IGhhcyBiZWVuIGNvbmZpZ3VyZWQuICBXZSBhcmUgYWxyZWFkeSBydW5uaW5nIHVucmVzdHJpY3RlZC4iCiAgICByZXR1cm4gMAogIGZpCgogICMgRW5zdXJlIHdlIHJlc2V0IHRoZSBDUFUgYWZmaW5pdHkgd2hlbiB3ZSBleGl0IHRoaXMgc2NyaXB0IGZvciBhbnkgcmVhc29uCiAgIyBUaGlzIHdheSBlaXRoZXIgYWZ0ZXIgdGhlIHRpbWVyIGV4cGlyZXMgb3IgYWZ0ZXIgdGhlIHByb2Nlc3MgaXMgaW50ZXJydXB0ZWQKICAjIHZpYSBeQyBvciBTSUdURVJNLCB3ZSByZXR1cm4gdGhpbmdzIGJhY2sgdG8gdGhlIHdheSB0aGV5IHNob3VsZCBiZS4KICB0cmFwIHNldFJlc3RyaWN0ZWQgRVhJVAoKICBsb2dnZXIgIlJlY292ZXJ5OiBSZWNvdmVyeSBNb2RlIFN0YXJ0aW5nIgogIHNldFVucmVzdHJpY3RlZAogIHdhaXRGb3JSZWFkeQp9CgppZiBbWyAiJHtCQVNIX1NPVVJDRVswXX0iID0gIiR7MH0iIF1dOyB0aGVuCiAgbWFpbiAiJHtAfSIKICBleGl0ICQ/CmZpCg==
    15.         mode: 493
    16.         path: /usr/local/bin/accelerated-container-startup.sh
    17.     systemd:
    18.       units:
    19.       - contents: |
    20.           [Unit]
    21.           Description=Unlocks more CPUs for critical system processes during container startup
    23.           [Service]
    24.           Type=simple
    25.           ExecStart=/usr/local/bin/accelerated-container-startup.sh
    27.           # Maximum wait time is 600s = 10m:
    28.           Environment=MAXIMUM_WAIT_TIME=600
    30.           # Steady-state threshold = 2%
    31.           # Allowed values:
    32.           #  4  - absolute pod count (+/-)
    33.           #  4% - percent change (+/-)
    34.           #  -1 - disable the steady-state check
    35.           # Note: '%' must be escaped as '%%' in systemd unit files
    36.           Environment=STEADY_STATE_THRESHOLD=2%%
    38.           # Steady-state window = 120s
    39.           # If the running pod count stays within the given threshold for this time
    40.           # period, return CPU utilization to normal before the maximum wait time has
    41.           # expires
    42.           Environment=STEADY_STATE_WINDOW=120
    44.           # Steady-state minimum = 40
    45.           # Increasing this will skip any steady-state checks until the count rises above
    46.           # this number to avoid false positives if there are some periods where the
    47.           # count doesn't increase but we know we can't be at steady-state yet.
    48.           Environment=STEADY_STATE_MINIMUM=40
    50.           [Install]
    51.           WantedBy=multi-user.target
    52.         enabled: true
    53.         name: accelerated-container-startup.service
    54.       - contents: |
    55.           [Unit]
    56.           Description=Unlocks more CPUs for critical system processes during container shutdown
    57.           DefaultDependencies=no
    59.           [Service]
    60.           Type=simple
    61.           ExecStart=/usr/local/bin/accelerated-container-startup.sh
    63.           # Maximum wait time is 600s = 10m:
    64.           Environment=MAXIMUM_WAIT_TIME=600
    66.           # Steady-state threshold
    67.           # Allowed values:
    68.           #  4  - absolute pod count (+/-)
    69.           #  4% - percent change (+/-)
    70.           #  -1 - disable the steady-state check
    71.           # Note: '%' must be escaped as '%%' in systemd unit files
    72.           Environment=STEADY_STATE_THRESHOLD=-1
    74.           # Steady-state window = 60s
    75.           # If the running pod count stays within the given threshold for this time
    76.           # period, return CPU utilization to normal before the maximum wait time has
    77.           # expires
    78.           Environment=STEADY_STATE_WINDOW=60
    80.           [Install]
    81.           WantedBy=shutdown.target reboot.target halt.target
    82.         enabled: true
    83.         name: accelerated-container-shutdown.service

    Automatic kernel crash dumps with kdump

    kdump is a Linux kernel feature that creates a kernel crash dump when the kernel crashes. kdump is enabled with the following MachineConfig CR:

    Recommended kdump configuration

    1. apiVersion: machineconfiguration.openshift.io/v1
    2. kind: MachineConfig
    3. metadata:
    4.   labels:
    5.     machineconfiguration.openshift.io/role: master
    6.   name: 06-kdump-enable-master
    7. spec:
    8.   config:
    9.     ignition:
    10.       version: 3.2.0
    11.     systemd:
    12.       units:
    13.       - enabled: true
    14.         name: kdump.service
    15.   kernelArguments:
    16.     - crashkernel=512M

    The following ContainerRuntimeConfig custom resources (CRs) configure crun as the default OCI container runtime for control plane and worker nodes. The crun container runtime is fast and lightweight and has a low memory footprint.

    For optimal performance, enable crun for master and worker nodes in single-node OpenShift, three-node OpenShift, and standard clusters. To avoid the cluster rebooting when the CR is applied, apply the change as a GitOps ZTP additional day-0 install-time manifest.

    Recommended ContainerRuntimeConfig CR for control plane nodes

    1. apiVersion: machineconfiguration.openshift.io/v1
    2. kind: ContainerRuntimeConfig
    3. metadata:
    4.  name: enable-crun-master
    5. spec:
    6.  machineConfigPoolSelector:
    7.    matchLabels:
    8.      pools.operator.machineconfiguration.openshift.io/master: ""
    9.  containerRuntimeConfig:
    10.    defaultRuntime: crun

    Recommended ContainerRuntimeConfig CR for worker nodes

    1. apiVersion: machineconfiguration.openshift.io/v1
    2. metadata:
    3.  name: enable-crun-worker
    4. spec:
    6.    matchLabels:
    7.      pools.operator.machineconfiguration.openshift.io/worker: ""
    8.  containerRuntimeConfig:
    9.    defaultRuntime: crun

    When the cluster installation is complete, the ZTP pipeline applies the following custom resources (CRs) that are required to run DU workloads.

    In GitOps ZTP v4.10 and earlier, you configure UEFI secure boot with a MachineConfig CR. This is no longer required in GitOps ZTP v4.11 and later. In v4.11, you configure UEFI secure boot for single-node OpenShift clusters using Performance profile CRs. For more information, see .

    Operator namespaces and Operator groups

    Single-node OpenShift clusters that run DU workloads require the following OperatorGroup and Namespace custom resources (CRs):

    • Local Storage Operator

    • Logging Operator

    • PTP Operator

    • SR-IOV Network Operator

    The following YAML summarizes these CRs:

    Recommended Operator Namespace and OperatorGroup configuration

    1. apiVersion: v1
    2. kind: Namespace
    3. metadata:
    4.   annotations:
    5.     workload.openshift.io/allowed: management
    6.   name: openshift-local-storage
    7. ---
    8. apiVersion: operators.coreos.com/v1
    9. kind: OperatorGroup
    10. metadata:
    11.   name: openshift-local-storage
    12.   namespace: openshift-local-storage
    13. spec:
    14.   targetNamespaces:
    15.     - openshift-local-storage
    16. ---
    17. apiVersion: v1
    18. kind: Namespace
    19. metadata:
    20.   annotations:
    21.     workload.openshift.io/allowed: management
    22.   name: openshift-logging
    23. ---
    24. apiVersion: operators.coreos.com/v1
    25. kind: OperatorGroup
    26. metadata:
    27.   name: cluster-logging
    28.   namespace: openshift-logging
    29. spec:
    30.   targetNamespaces:
    31.     - openshift-logging
    32. ---
    33. apiVersion: v1
    34. kind: Namespace
    35. metadata:
    36.   annotations:
    37.     workload.openshift.io/allowed: management
    38.   labels:
    39.     openshift.io/cluster-monitoring: "true"
    40.   name: openshift-ptp
    41. ---
    42. apiVersion: operators.coreos.com/v1
    43. kind: OperatorGroup
    44. metadata:
    45.   name: ptp-operators
    46.   namespace: openshift-ptp
    47. spec:
    48.   targetNamespaces:
    49.     - openshift-ptp
    50. ---
    51. apiVersion: v1
    52. kind: Namespace
    53. metadata:
    54.   annotations:
    55.     workload.openshift.io/allowed: management
    56.     name: openshift-sriov-network-operator
    57. ---
    58. apiVersion: operators.coreos.com/v1
    59. kind: OperatorGroup
    60. metadata:
    61.   name: sriov-network-operators
    62.   namespace: openshift-sriov-network-operator
    63. spec:
    64.   targetNamespaces:
    65.     - openshift-sriov-network-operator

    Operator subscriptions

    Single-node OpenShift clusters that run DU workloads require the following Subscription CRs. The subscription provides the location to download the following Operators:

    • Local Storage Operator

    • Logging Operator

    • PTP Operator

    • SR-IOV Network Operator

    Recommended Operator subscriptions

    1. apiVersion: operators.coreos.com/v1alpha1
    2. kind: Subscription
    3. metadata:
    4.   name: cluster-logging
    5.   namespace: openshift-logging
    6. spec:
    7.   channel: "stable" (1)
    8.   name: cluster-logging
    9.   source: redhat-operators
    10.   sourceNamespace: openshift-marketplace
    11.   installPlanApproval: Manual (2)
    12. ---
    13. apiVersion: operators.coreos.com/v1alpha1
    14. kind: Subscription
    15. metadata:
    16.   name: local-storage-operator
    17.   namespace: openshift-local-storage
    18. spec:
    19.   channel: "stable"
    20.   installPlanApproval: Automatic
    21.   name: local-storage-operator
    22.   source: redhat-operators
    23.   sourceNamespace: openshift-marketplace
    24.   installPlanApproval: Manual
    25. ---
    26. apiVersion: operators.coreos.com/v1alpha1
    27. kind: Subscription
    28. metadata:
    29.     name: ptp-operator-subscription
    30.     namespace: openshift-ptp
    31. spec:
    32.   channel: "stable"
    33.   name: ptp-operator
    34.   source: redhat-operators
    35.   sourceNamespace: openshift-marketplace
    36.   installPlanApproval: Manual
    37. ---
    38. apiVersion: operators.coreos.com/v1alpha1
    39. kind: Subscription
    40. metadata:
    41.   name: sriov-network-operator-subscription
    42.   namespace: openshift-sriov-network-operator
    43. spec:
    44.   channel: "stable"
    45.   name: sriov-network-operator
    46.   source: redhat-operators
    47.   sourceNamespace: openshift-marketplace
    48.   installPlanApproval: Manual

    Cluster logging and log forwarding

    Single-node OpenShift clusters that run DU workloads require logging and log forwarding for debugging. The following example YAML illustrates the required ClusterLogging and ClusterLogForwarder CRs.

    Recommended cluster logging and log forwarding configuration

    1Updates the existing ClusterLogging instance or creates the instance if it does not exist.
    2Updates the existing ClusterLogForwarder instance or creates the instance if it does not exist.
    3Specifies the URL of the Kafka server where the logs are forwarded to.

    Performance profile

    Single-node OpenShift clusters that run DU workloads require a Node Tuning Operator performance profile to use real-time host capabilities and services.

    In earlier versions of OKD, the Performance Addon Operator was used to implement automatic tuning to achieve low latency performance for OpenShift applications. In OKD 4.11 and later, this functionality is part of the Node Tuning Operator.

    The following example PerformanceProfile CR illustrates the required cluster configuration.

    Recommended performance profile configuration

    1. apiVersion: performance.openshift.io/v2
    2. kind: PerformanceProfile
    3. metadata:
    4.   name: openshift-node-performance-profile (1)
    5. spec:
    6.   additionalKernelArgs:
    7.   - "rcupdate.rcu_normal_after_boot=0"
    8.   - "efi=runtime" (2)
    9.   cpu:
    10.     isolated: 2-51,54-103 (3)
    11.     reserved: 0-1,52-53   (4)
    12.   hugepages:
    13.     defaultHugepagesSize: 1G
    14.     pages:
    15.       - count: 32 (5)
    16.         size: 1G  (6)
    17.         node: 0 (7)
    18.   machineConfigPoolSelector:
    19.     pools.operator.machineconfiguration.openshift.io/master: ""
    20.   nodeSelector:
    21.     node-role.kubernetes.io/master: ""
    22.   numa:
    23.     topologyPolicy: "restricted"
    24.   realTimeKernel:
    25.     enabled: true    (8)
    1Ensure that the value for name matches that specified in the spec.profile.data field of TunedPerformancePatch.yaml and the status.configuration.source.name field of validatorCRs/informDuValidator.yaml.
    2Configures UEFI secure boot for the cluster host.
    3Set the isolated CPUs. Ensure all of the Hyper-Threading pairs match.

    The reserved and isolated CPU pools must not overlap and together must span all available cores. CPU cores that are not accounted for cause an undefined behaviour in the system.

    4Set the reserved CPUs. When workload partitioning is enabled, system processes, kernel threads, and system container threads are restricted to these CPUs. All CPUs that are not isolated should be reserved.
    5Set the number of huge pages.
    6Set the huge page size.
    7Set node to the NUMA node where the hugepages are allocated.
    8Set enabled to true to install the real-time Linux kernel.

    Single-node OpenShift clusters use Precision Time Protocol (PTP) for network time synchronization. The following example PtpConfig CR illustrates the required PTP slave configuration.

    Recommended PTP configuration

    1. apiVersion: ptp.openshift.io/v1
    2. kind: PtpConfig
    3. metadata:
    4.   name: du-ptp-slave
    5.   namespace: openshift-ptp
    6. spec:
    7.   profile:
    8.     - interface: ens5f0     (1)
    9.       name: slave
    10.       phc2sysOpts: -a -r -n 24
    11.       ptp4lConf: |
    12.         [global]
    13.         #
    14.         # Default Data Set
    15.         #
    16.         twoStepFlag 1
    17.         slaveOnly 0
    18.         priority1 128
    19.         priority2 128
    20.         domainNumber 24
    21.         #utc_offset 37
    22.         clockClass 248
    23.         clockAccuracy 0xFE
    24.         offsetScaledLogVariance 0xFFFF
    25.         free_running 0
    26.         freq_est_interval 1
    27.         dscp_event 0
    28.         dscp_general 0
    29.         dataset_comparison ieee1588
    30.         G.8275.defaultDS.localPriority 128
    31.         #
    32.         # Port Data Set
    33.         #
    34.         logAnnounceInterval -3
    35.         logSyncInterval -4
    36.         logMinDelayReqInterval -4
    37.         logMinPdelayReqInterval -4
    38.         announceReceiptTimeout 3
    39.         syncReceiptTimeout 0
    40.         delayAsymmetry 0
    41.         fault_reset_interval 4
    42.         neighborPropDelayThresh 20000000
    43.         masterOnly 0
    44.         G.8275.portDS.localPriority 128
    45.         #
    47.         #
    48.         assume_two_step 0
    49.         path_trace_enabled 0
    50.         follow_up_info 0
    51.         hybrid_e2e 0
    52.         inhibit_multicast_service 0
    53.         net_sync_monitor 0
    54.         tc_spanning_tree 0
    55.         tx_timestamp_timeout 1
    56.         unicast_listen 0
    57.         unicast_master_table 0
    58.         unicast_req_duration 3600
    59.         use_syslog 1
    60.         verbose 0
    61.         summary_interval 0
    62.         kernel_leap 1
    63.         check_fup_sync 0
    64.         #
    65.         # Servo Options
    66.         #
    67.         pi_proportional_const 0.0
    68.         pi_integral_const 0.0
    69.         pi_proportional_scale 0.0
    70.         pi_proportional_exponent -0.3
    71.         pi_proportional_norm_max 0.7
    72.         pi_integral_scale 0.0
    73.         pi_integral_exponent 0.4
    74.         pi_integral_norm_max 0.3
    75.         step_threshold 2.0
    76.         first_step_threshold 0.00002
    77.         max_frequency 900000000
    78.         clock_servo pi
    79.         sanity_freq_limit 200000000
    80.         ntpshm_segment 0
    81.         #
    82.         # Transport options
    83.         #
    84.         transportSpecific 0x0
    85.         ptp_dst_mac 01:1B:19:00:00:00
    86.         p2p_dst_mac 01:80:C2:00:00:0E
    87.         udp_ttl 1
    88.         udp6_scope 0x0E
    89.         uds_address /var/run/ptp4l
    90.         #
    91.         # Default interface options
    92.         #
    93.         clock_type OC
    94.         network_transport L2
    95.         delay_mechanism E2E
    96.         time_stamping hardware
    97.         tsproc_mode filter
    98.         delay_filter moving_median
    99.         delay_filter_length 10
    100.         egressLatency 0
    101.         ingressLatency 0
    102.         boundary_clock_jbod 0
    103.         #
    104.         # Clock description
    105.         #
    106.         productDescription ;;
    107.         revisionData ;;
    108.         manufacturerIdentity 00:00:00
    109.         userDescription ;
    110.         timeSource 0xA0
    111.       ptp4lOpts: -2 -s --summary_interval -4
    112. recommend:
    113.   - match:
    114.       - nodeLabel: node-role.kubernetes.io/master
    115.     priority: 4
    116.     profile: slave
    1Sets the interface used to receive the PTP clock signal.

    Extended Tuned profile

    Single-node OpenShift clusters that run DU workloads require additional performance tuning configurations necessary for high-performance workloads. The following example Tuned CR extends the Tuned profile:

    Recommended extended Tuned profile configuration

    1. apiVersion: tuned.openshift.io/v1
    2. kind: Tuned
    3. metadata:
    4.   name: performance-patch
    5.   namespace: openshift-cluster-node-tuning-operator
    6. spec:
    7.   profile:
    8.     - data: |
    9.         [main]
    10.         summary=Configuration changes profile inherited from performance created tuned
    11.         include=openshift-node-performance-openshift-node-performance-profile
    12.         [bootloader]
    13.         cmdline_crash=nohz_full=2-51,54-103
    14.         [sysctl]
    15.         kernel.timer_migration=1
    16.         [scheduler]
    17.         group.ice-ptp=0:f:10:*:ice-ptp.*
    18.         [service]
    19.         service.stalld=start,enable
    20.         service.chronyd=stop,disable
    21.       name: performance-patch
    22.   recommend:
    23.     - machineConfigLabels:
    24.         machineconfiguration.openshift.io/role: master
    25.       priority: 19
    26.       profile: performance-patch

    SR-IOV

    Single root I/O virtualization (SR-IOV) is commonly used to enable the fronthaul and the midhaul networks. The following YAML example configures SR-IOV for a single-node OpenShift cluster.

    Recommended SR-IOV configuration

    1. apiVersion: sriovnetwork.openshift.io/v1
    2. kind: SriovOperatorConfig
    3. metadata:
    4.   name: default
    5.   namespace: openshift-sriov-network-operator
    6. spec:
    7.   configDaemonNodeSelector:
    8.     node-role.kubernetes.io/master: ""
    9.   disableDrain: true
    10.   enableInjector: true
    11.   enableOperatorWebhook: true
    12. ---
    13. apiVersion: sriovnetwork.openshift.io/v1
    14. kind: SriovNetwork
    15. metadata:
    16.   name: sriov-nw-du-mh
    17.   namespace: openshift-sriov-network-operator
    18. spec:
    19.   networkNamespace: openshift-sriov-network-operator
    20.   resourceName: du_mh
    21.   vlan: 150 (1)
    22. ---
    23. apiVersion: sriovnetwork.openshift.io/v1
    24. kind: SriovNetworkNodePolicy
    25. metadata:
    26.   name: sriov-nnp-du-mh
    27.   namespace: openshift-sriov-network-operator
    28. spec:
    29.   deviceType: vfio-pci (2)
    30.   isRdma: false
    31.   nicSelector:
    32.     pfNames:
    33.       - ens7f0 (3)
    34.   nodeSelector:
    35.     node-role.kubernetes.io/master: ""
    36.   numVfs: 8 (4)
    37.   priority: 10
    38.   resourceName: du_mh
    39. ---
    40. apiVersion: sriovnetwork.openshift.io/v1
    41. kind: SriovNetwork
    42. metadata:
    43.   name: sriov-nw-du-fh
    44.   namespace: openshift-sriov-network-operator
    45. spec:
    46.   networkNamespace: openshift-sriov-network-operator
    47.   resourceName: du_fh
    48.   vlan: 140 (5)
    49. ---
    50. apiVersion: sriovnetwork.openshift.io/v1
    51. kind: SriovNetworkNodePolicy
    52. metadata:
    53.   name: sriov-nnp-du-fh
    54.   namespace: openshift-sriov-network-operator
    55. spec:
    56.   deviceType: netdevice (6)
    57.   isRdma: true
    58.   nicSelector:
    59.     pfNames:
    60.       - ens5f0 (7)
    61.   nodeSelector:
    62.     node-role.kubernetes.io/master: ""
    63.   numVfs: 8 (8)
    64.   priority: 10
    65.   resourceName: du_fh
    1Specifies the VLAN for the midhaul network.
    2Select either vfio-pci or netdevice, as needed.
    3Specifies the interface connected to the midhaul network.
    4Specifies the number of VFs for the midhaul network.
    5The VLAN for the fronthaul network.
    6Select either vfio-pci or netdevice, as needed.
    7Specifies the interface connected to the fronthaul network.
    8Specifies the number of VFs for the fronthaul network.

    Console Operator

    The console-operator installs and maintains the web console on a cluster. When the node is centrally managed the Operator is not needed and makes space for application workloads. The following Console custom resource (CR) example disables the console.

    Recommended console configuration

    1. apiVersion: operator.openshift.io/v1
    2. kind: Console
    3. metadata:
    4.   annotations:
    5.     include.release.openshift.io/ibm-cloud-managed: "false"
    6.     include.release.openshift.io/self-managed-high-availability: "false"
    7.     include.release.openshift.io/single-node-developer: "false"
    8.     release.openshift.io/create-only: "true"
    9.   name: cluster
    10. spec:
    11.   logLevel: Normal
    12.   managementState: Removed
    13.   operatorLogLevel: Normal

    Grafana and Alertmanager

    Single-node OpenShift clusters that run DU workloads require reduced CPU resources consumed by the OKD monitoring components. The following ConfigMap custom resource (CR) disables Grafana and Alertmanager.

    Recommended cluster monitoring configuration

    1. apiVersion: v1
    2. kind: ConfigMap
    3. metadata:
    4.   name: cluster-monitoring-config
    5.   namespace: openshift-monitoring
    6. data:
    7.   config.yaml: |
    8.     grafana:
    9.       enabled: false
    10.     alertmanagerMain:
    11.       enabled: false
    12.     prometheusK8s:
    13.        retention: 24h

    You can dynamically provision local storage on single-node OpenShift clusters with Logical volume manager storage (LVM Storage).

    The following YAML example configures the storage of the node to be available to OKD applications.

    Recommended LVMCluster configuration

    1. apiVersion: lvm.topolvm.io/v1alpha1
    2. kind: LVMCluster
    3. metadata:
    4.   name: odf-lvmcluster
    5.   namespace: openshift-storage
    6. spec:
    7.   storage:
    8.     deviceClasses:
    9.     - name: vg1
    10.       deviceSelector: (1)
    11.         paths:
    12.         - /usr/disk/by-path/pci-0000:11:00.0-nvme-1
    13.       thinPoolConfig:
    14.         name: thin-pool-1
    15.         overprovisionRatio: 10
    1If no disks are specified in the deviceSelector.paths field, the LVM Storage uses all the unused disks in the specified thin pool.

    Network diagnostics

    Single-node OpenShift clusters that run DU workloads require less inter-pod network connectivity checks to reduce the additional load created by these pods. The following custom resource (CR) disables these checks.

    Recommended network diagnostics configuration