Post-installation network configuration

    The configuration for the cluster network is specified as part of the Cluster Network Operator (CNO) configuration and stored in a custom resource (CR) object that is named . The CR specifies the fields for the Network API in the operator.openshift.io API group.

    The CNO configuration inherits the following fields during cluster installation from the Network API in the Network.config.openshift.io API group and these fields cannot be changed:

    clusterNetwork

    IP address pools from which pod IP addresses are allocated.

    serviceNetwork

    IP address pool for services.

    defaultNetwork.type

    Cluster network provider, such as OpenShift SDN or OVN-Kubernetes.

    Enabling the cluster-wide proxy

    The Proxy object is used to manage the cluster-wide egress proxy. When a cluster is installed or upgraded without the proxy configured, a Proxy object is still generated but it will have a nil spec. For example:

    A cluster administrator can configure the proxy for OKD by modifying this cluster Proxy object.

    Only the Proxy object named cluster is supported, and no additional proxies can be created.

    Prerequisites

    • Cluster administrator permissions

    • OKD oc CLI tool installed

    Procedure

    1. Create a ConfigMap that contains any additional CA certificates required for proxying HTTPS connections.

      You can skip this step if the proxy’s identity certificate is signed by an authority from the RHCOS trust bundle.

      1. Create a file called user-ca-bundle.yaml with the following contents, and provide the values of your PEM-encoded certificates:

        1. apiVersion: v1
        2. data:
        3. ca-bundle.crt: | (1)
        4. <MY_PEM_ENCODED_CERTS> (2)
        5. kind: ConfigMap
        6. metadata:
        7. name: user-ca-bundle (3)
        8. namespace: openshift-config (4)
        1This data key must be named ca-bundle.crt.
        2One or more PEM-encoded X.509 certificates used to sign the proxy’s identity certificate.
        3The ConfigMap name that will be referenced from the Proxy object.
        4The ConfigMap must be in the openshift-config namespace.
      2. Create the ConfigMap from this file:

        1. $ oc create -f user-ca-bundle.yaml
    2. Use the oc edit command to modify the Proxy object:

      1. $ oc edit proxy/cluster
    3. Configure the necessary fields for the proxy:

      1. apiVersion: config.openshift.io/v1
      2. kind: Proxy
      3. metadata:
      4. name: cluster
      5. spec:
      6. httpProxy: http://<username>:<pswd>@<ip>:<port> (1)
      7. httpsProxy: http://<username>:<pswd>@<ip>:<port> (2)
      8. noProxy: example.com (3)
      9. readinessEndpoints:
      10. - http://www.google.com (4)
      11. - https://www.google.com
      12. trustedCA:
      13. name: user-ca-bundle (5)
      1A proxy URL to use for creating HTTP connections outside the cluster. The URL scheme must be http.
      2A proxy URL to use for creating HTTPS connections outside the cluster.
      3A comma-separated list of destination domain names, domains, IP addresses or other network CIDRs to exclude proxying.

      Preface a domain with . to match subdomains only. For example, .y.com matches x.y.com, but not y.com. Use * to bypass proxy for all destinations. If you scale up workers that are not included in the network defined by the networking.machineNetwork[].cidr field from the installation configuration, you must add them to this list to prevent connection issues.

      This field is ignored if neither the httpProxy or httpsProxy fields are set.

      4One or more URLs external to the cluster to use to perform a readiness check before writing the httpProxy and httpsProxy values to status.
      5A reference to the ConfigMap in the openshift-config namespace that contains additional CA certificates required for proxying HTTPS connections. Note that the ConfigMap must already exist before referencing it here. This field is required unless the proxy’s identity certificate is signed by an authority from the RHCOS trust bundle.
    4. Save the file to apply the changes.

    The URL scheme must be http. The https scheme is currently not supported.

    Setting DNS to private

    After you deploy a cluster, you can modify its DNS to use only a private zone.

    Procedure

    1. Review the DNS custom resource for your cluster:

      1. $ oc get dnses.config.openshift.io/cluster -o yaml

      Example output

      1. apiVersion: config.openshift.io/v1
      2. kind: DNS
      3. metadata:
      4. creationTimestamp: "2019-10-25T18:27:09Z"
      5. generation: 2
      6. name: cluster
      7. resourceVersion: "37966"
      8. selfLink: /apis/config.openshift.io/v1/dnses/cluster
      9. uid: 0e714746-f755-11f9-9cb1-02ff55d8f976
      10. spec:
      11. baseDomain: <base_domain>
      12. privateZone:
      13. tags:
      14. Name: <infrastructure_id>-int
      15. kubernetes.io/cluster/<infrastructure_id>: owned
      16. publicZone:
      17. id: Z2XXXXXXXXXXA4
      18. status: {}

      Note that the spec section contains both a private and a public zone.

    2. Patch the DNS custom resource to remove the public zone:

      1. $ oc patch dnses.config.openshift.io/cluster --type=merge --patch='{"spec": {"publicZone": null}}'
      2. dns.config.openshift.io/cluster patched

      Because the Ingress Controller consults the DNS definition when it creates Ingress objects, when you create or modify Ingress objects, only private records are created.

      DNS records for the existing Ingress objects are not modified when you remove the public zone.

    3. Optional: Review the DNS custom resource for your cluster and confirm that the public zone was removed:

      1. $ oc get dnses.config.openshift.io/cluster -o yaml

      Example output

      1. apiVersion: config.openshift.io/v1
      2. kind: DNS
      3. metadata:
      4. creationTimestamp: "2019-10-25T18:27:09Z"
      5. generation: 2
      6. name: cluster
      7. resourceVersion: "37966"
      8. selfLink: /apis/config.openshift.io/v1/dnses/cluster
      9. uid: 0e714746-f755-11f9-9cb1-02ff55d8f976
      10. spec:
      11. baseDomain: <base_domain>
      12. privateZone:
      13. tags:
      14. Name: <infrastructure_id>-int
      15. kubernetes.io/cluster/<infrastructure_id>-wfpg4: owned
      16. status: {}

    OKD provides the following methods for communicating from outside the cluster with services running in the cluster:

    • If you have HTTP/HTTPS, use an Ingress Controller.

    • If you have a TLS-encrypted protocol other than HTTPS, such as TLS with the SNI header, use an Ingress Controller.

    • Otherwise, use a load balancer, an external IP, or a node port.

    Configuring the node port service range

    As a cluster administrator, you can expand the available node port range. If your cluster uses of a large number of node ports, you might need to increase the number of available ports.

    The default port range is 30000-32767. You can never reduce the port range, even if you first expand it beyond the default range.

    • Your cluster infrastructure must allow access to the ports that you specify within the expanded range. For example, if you expand the node port range to 30000-32900, the inclusive port range of 32768-32900 must be allowed by your firewall or packet filtering configuration.

    Expanding the node port range

    You can expand the node port range for the cluster.

    Prerequisites

    • Install the OpenShift CLI (oc).

    • Log in to the cluster with a user with cluster-admin privileges.

    Procedure

    1. To expand the node port range, enter the following command. Replace <port> with the largest port number in the new range.

      1. $ oc patch network.config.openshift.io cluster --type=merge -p \
      2. '{
      3. "spec":
      4. { "serviceNodePortRange": "30000-<port>" }
      5. }'

      You can alternatively apply the following YAML to update the node port range:

      1. apiVersion: config.openshift.io/v1
      2. kind: Network
      3. metadata:
      4. name: cluster
      5. spec:
      6. serviceNodePortRange: 30000-<port>”

      Example output

      1. network.config.openshift.io/cluster patched
    2. To confirm that the configuration is active, enter the following command. It can take several minutes for the update to apply.

      1. $ oc get configmaps -n openshift-kube-apiserver config \
      2. -o jsonpath="{.data['config\.yaml']}" | \
      3. grep -Eo '"service-node-port-range":["[[:digit:]]+-[[:digit:]]+"]'

      Example output

      1. "service-node-port-range":["30000-33000"]

    Configuring network policy

    As a cluster administrator or project administrator, you can configure network policies for a project.

    About network policy

    In a cluster using a Kubernetes Container Network Interface (CNI) plug-in that supports Kubernetes network policy, network isolation is controlled entirely by NetworkPolicy objects. In OKD 4.8, OpenShift SDN supports using network policy in its default network isolation mode.

    When using the OpenShift SDN cluster network provider, the following limitations apply regarding network policies:

    • Egress network policy as specified by the egress field is not supported.

    • IPBlock is supported by network policy, but without support for except clauses. If you create a policy with an IPBlock section that includes an except clause, the SDN pods log warnings and the entire IPBlock section of that policy is ignored.

    Network policy does not apply to the host network namespace. Pods with host networking enabled are unaffected by network policy rules.

    By default, all pods in a project are accessible from other pods and network endpoints. To isolate one or more pods in a project, you can create NetworkPolicy objects in that project to indicate the allowed incoming connections. Project administrators can create and delete NetworkPolicy objects within their own project.

    If a pod is matched by selectors in one or more NetworkPolicy objects, then the pod will accept only connections that are allowed by at least one of those NetworkPolicy objects. A pod that is not selected by any NetworkPolicy objects is fully accessible.

    The following example NetworkPolicy objects demonstrate supporting different scenarios:

    • Deny all traffic:

      To make a project deny by default, add a NetworkPolicy object that matches all pods but accepts no traffic:

    • Only allow connections from the OKD Ingress Controller:

      To make a project allow only connections from the OKD Ingress Controller, add the following NetworkPolicy object.

      1. apiVersion: networking.k8s.io/v1
      2. kind: NetworkPolicy
      3. metadata:
      4. name: allow-from-openshift-ingress
      5. spec:
      6. - namespaceSelector:
      7. matchLabels:
      8. network.openshift.io/policy-group: ingress
      9. podSelector: {}
      10. policyTypes:
      11. - Ingress
    • Only accept connections from pods within a project:

      To make pods accept connections from other pods in the same project, but reject all other connections from pods in other projects, add the following NetworkPolicy object:

      1. kind: NetworkPolicy
      2. apiVersion: networking.k8s.io/v1
      3. metadata:
      4. name: allow-same-namespace
      5. spec:
      6. podSelector:
      7. ingress:
      8. - from:
      9. - podSelector: {}
    • Only allow HTTP and HTTPS traffic based on pod labels:

      To enable only HTTP and HTTPS access to the pods with a specific label (role=frontend in following example), add a NetworkPolicy object similar to the following:

      1. kind: NetworkPolicy
      2. apiVersion: networking.k8s.io/v1
      3. metadata:
      4. name: allow-http-and-https
      5. spec:
      6. podSelector:
      7. matchLabels:
      8. role: frontend
      9. ingress:
      10. - ports:
      11. - protocol: TCP
      12. port: 80
      13. - protocol: TCP
      14. port: 443
    • Accept connections by using both namespace and pod selectors:

      To match network traffic by combining namespace and pod selectors, you can use a NetworkPolicy object similar to the following:

      1. kind: NetworkPolicy
      2. apiVersion: networking.k8s.io/v1
      3. metadata:
      4. name: allow-pod-and-namespace-both
      5. spec:
      6. podSelector:
      7. matchLabels:
      8. name: test-pods
      9. ingress:
      10. - from:
      11. - namespaceSelector:
      12. matchLabels:
      13. project: project_name
      14. podSelector:
      15. matchLabels:
      16. name: test-pods

    NetworkPolicy objects are additive, which means you can combine multiple NetworkPolicy objects together to satisfy complex network requirements.

    For example, for the NetworkPolicy objects defined in previous samples, you can define both allow-same-namespace and allow-http-and-https policies within the same project. Thus allowing the pods with the label role=frontend, to accept any connection allowed by each policy. That is, connections on any port from pods in the same namespace, and connections on ports 80 and 443 from pods in any namespace.

    Example NetworkPolicy object

    The following annotates an example NetworkPolicy object:

    1. kind: NetworkPolicy
    2. apiVersion: networking.k8s.io/v1
    3. metadata:
    4. name: allow-27107 (1)
    5. spec:
    6. podSelector: (2)
    7. matchLabels:
    8. app: mongodb
    9. ingress:
    10. - from:
    11. - podSelector: (3)
    12. matchLabels:
    13. app: app
    14. ports: (4)
    15. - protocol: TCP
    16. port: 27017
    1The name of the NetworkPolicy object.
    2A selector describing the pods the policy applies to. The policy object can only select pods in the project that the NetworkPolicy object is defined.
    3A selector matching the pods that the policy object allows ingress traffic from. The selector will match pods in any project.
    4A list of one or more destination ports to accept traffic on.

    Creating a network policy

    If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

    Prerequisites

    • Your cluster uses a cluster network provider that supports NetworkPolicy objects, such as the OVN-Kubernetes network provider or the OpenShift SDN network provider with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

    • You installed the OpenShift CLI (oc).

    • You are logged in to the cluster with a user with admin privileges.

    • You are working in the namespace that the network policy applies to.

    • Your cluster is using a cluster network provider that supports NetworkPolicy objects, such as the OpenShift SDN network provider with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

    Procedure

    1. Create a policy rule:

      1. Create a <policy_name>.yaml file:

        1. $ touch <policy_name>.yaml

        where:

        <policy_name>

        Specifies the network policy file name.

      2. Define a network policy in the file that you just created, such as in the following examples:

        Deny ingress from all pods in all namespaces

        1. kind: NetworkPolicy
        2. apiVersion: networking.k8s.io/v1
        3. metadata:
        4. name: deny-by-default
        5. spec:
        6. podSelector:
        7. ingress: []

      Allow ingress from all pods in the same namespace

      1. kind: NetworkPolicy
      2. apiVersion: networking.k8s.io/v1
      3. metadata:
      4. name: allow-same-namespace
      5. spec:
      6. podSelector:
      7. ingress:
      8. - from:
      9. - podSelector: {}
    2. To create the network policy object, enter the following command:

      1. $ oc apply -f <policy_name>.yaml -n <namespace>

      where:

      <policy_name>

      Specifies the network policy file name.

      <namespace>

      Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

      Example output

      1. networkpolicy.networking.k8s.io/default-deny created

    Configuring multitenant isolation by using network policy

    You can configure your project to isolate it from pods and services in other project namespaces.

    Prerequisites

    • Your cluster uses a cluster network provider that supports NetworkPolicy objects, such as the OVN-Kubernetes network provider or the OpenShift SDN network provider with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

    • You installed the OpenShift CLI (oc).

    • You are logged in to the cluster with a user with admin privileges.

    Procedure

    1. Create the following NetworkPolicy objects:

      1. A policy named allow-from-openshift-ingress.

        1. $ cat << EOF| oc create -f -
        2. apiVersion: networking.k8s.io/v1
        3. kind: NetworkPolicy
        4. metadata:
        5. name: allow-from-openshift-ingress
        6. spec:
        7. ingress:
        8. - from:
        9. - namespaceSelector:
        10. matchLabels:
        11. network.openshift.io/policy-group: ingress
        12. podSelector: {}
        13. policyTypes:
        14. - Ingress
        15. EOF
      2. A policy named allow-from-openshift-monitoring:

        1. $ cat << EOF| oc create -f -
        2. apiVersion: networking.k8s.io/v1
        3. kind: NetworkPolicy
        4. metadata:
        5. name: allow-from-openshift-monitoring
        6. spec:
        7. ingress:
        8. - from:
        9. - namespaceSelector:
        10. matchLabels:
        11. network.openshift.io/policy-group: monitoring
        12. podSelector: {}
        13. - Ingress
        14. EOF
      3. A policy named allow-same-namespace:

        1. $ cat << EOF| oc create -f -
        2. kind: NetworkPolicy
        3. apiVersion: networking.k8s.io/v1
        4. name: allow-same-namespace
        5. spec:
        6. podSelector:
        7. ingress:
        8. - from:
        9. - podSelector: {}
        10. EOF
    2. Optional: To confirm that the network policies exist in your current project, enter the following command:

      1. $ oc describe networkpolicy

      Example output

      ``` Name: allow-from-openshift-ingress Namespace: example1 Created on: 2020-06-09 00:28:17 -0400 EDT Labels: Annotations: Spec: PodSelector: (Allowing the specific traffic to all pods in this namespace) Allowing ingress traffic:

      Not affecting egress traffic Policy Types: Ingress

    1. Name: allow-from-openshift-monitoring
    2. Namespace: example1
    3. Created on: 2020-06-09 00:29:57 -0400 EDT
    4. Labels: <none>
    5. Annotations: <none>
    6. Spec:
    7. PodSelector: <none> (Allowing the specific traffic to all pods in this namespace)
    8. Allowing ingress traffic:
    9. To Port: <any> (traffic allowed to all ports)
    10. From:
    11. NamespaceSelector: network.openshift.io/policy-group: monitoring
    12. Not affecting egress traffic
    13. Policy Types: Ingress
    14. ```

    As a cluster administrator, you can modify the new project template to automatically include NetworkPolicy objects when you create a new project.

    Modifying the template for new projects

    As a cluster administrator, you can modify the default project template so that new projects are created using your custom requirements.

    To create your own custom project template:

    Procedure

    1. Log in as a user with cluster-admin privileges.

    2. Generate the default project template:

      1. $ oc adm create-bootstrap-project-template -o yaml > template.yaml
    3. Use a text editor to modify the generated template.yaml file by adding objects or modifying existing objects.

    4. The project template must be created in the openshift-config namespace. Load your modified template:

      1. $ oc create -f template.yaml -n openshift-config
    5. Edit the project configuration resource using the web console or CLI.

      • Using the web console:

        1. Navigate to the AdministrationCluster Settings page.

        2. Click Global Configuration to view all configuration resources.

        3. Find the entry for Project and click Edit YAML.

      • Using the CLI:

        1. Edit the project.config.openshift.io/cluster resource:

          1. $ oc edit project.config.openshift.io/cluster
    1. Update the spec section to include the projectRequestTemplate and name parameters, and set the name of your uploaded project template. The default name is project-request.

      Project configuration resource with custom project template

      1. apiVersion: config.openshift.io/v1
      2. kind: Project
      3. metadata:
      4. ...
      5. spec:
      6. projectRequestTemplate:
      7. name: <template_name>
    2. After you save your changes, create a new project to verify that your changes were successfully applied.

    Adding network policies to the new project template

    As a cluster administrator, you can add network policies to the default template for new projects. OKD will automatically create all the NetworkPolicy objects specified in the template in the project.

    Prerequisites

    • Your cluster uses a default CNI network provider that supports NetworkPolicy objects, such as the OpenShift SDN network provider with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

    • You installed the OpenShift CLI (oc).

    • You must log in to the cluster with a user with cluster-admin privileges.

    • You must have created a custom default project template for new projects.

    Procedure

    1. Edit the default template for a new project by running the following command:

      1. $ oc edit template <project_template> -n openshift-config

      Replace <project_template> with the name of the default template that you configured for your cluster. The default template name is project-request.

    2. In the template, add each NetworkPolicy object as an element to the objects parameter. The objects parameter accepts a collection of one or more objects.

      In the following example, the objects parameter collection includes several NetworkPolicy objects.

      For the OVN-Kubernetes network provider plug-in, when the Ingress Controller is configured to use the HostNetwork endpoint publishing strategy, there is no supported way to apply network policy so that ingress traffic is allowed and all other traffic is denied.

      1. objects:
      2. - apiVersion: networking.k8s.io/v1
      3. kind: NetworkPolicy
      4. metadata:
      5. name: allow-from-same-namespace
      6. spec:
      7. podSelector:
      8. ingress:
      9. - from:
      10. - podSelector: {}
      11. - apiVersion: networking.k8s.io/v1
      12. kind: NetworkPolicy
      13. metadata:
      14. name: allow-from-openshift-ingress
      15. spec:
      16. ingress:
      17. - from:
      18. - namespaceSelector:
      19. matchLabels:
      20. network.openshift.io/policy-group: ingress
      21. podSelector: {}
      22. policyTypes:
      23. - Ingress
      24. ...
    3. Optional: Create a new project to confirm that your network policy objects are created successfully by running the following commands:

      1. Create a new project:

        1. $ oc new-project <project> (1)
      2. Confirm that the network policy objects in the new project template exist in the new project:

        1. $ oc get networkpolicy
        2. NAME POD-SELECTOR AGE
        3. allow-from-openshift-ingress <none> 7s
        4. allow-from-same-namespace <none> 7s

    The following configurations are supported for the current release of Red Hat OpenShift Service Mesh:

    • Red Hat OKD version 4.x.

    • Red Hat Red Hat OpenShift Dedicated version 4.

    • Azure Red Hat OpenShift version 4.

    Red Hat OpenShift Online is not supported for Red Hat OpenShift Service Mesh.

    • This release of Red Hat OpenShift Service Mesh is only available on OKD x86_64, IBM Z, and IBM Power Systems.

      • IBM Z is only supported on OKD 4.6 and later.

      • IBM Power Systems is only supported on OKD 4.6 and later.

    • Configurations where all Service Mesh components are contained within a single OKD cluster. Red Hat OpenShift Service Mesh does not support management of microservices that reside outside of the cluster within which Service Mesh is running.

    • Configurations that do not integrate external services such as virtual machines.

    For additional information about Red Hat OpenShift Service Mesh lifecycle and supported configurations, refer to the .

    Supported network configurations

    Red Hat OpenShift Service Mesh supports the following network configurations.

    • OVN-Kubernetes is supported as a technology preview in OKD version 4.7.

    Supported configurations for Kiali

    • The Kiali observability console is only supported on the two most recent releases of the Chrome, Edge, Firefox, or Safari browsers.

    Supported configurations for Distributed Tracing

    • Jaeger agent as a sidecar is the only supported configuration for Jaeger. Jaeger as a daemonset is not supported for multitenant installations or OpenShift Dedicated.
    • This release only supports the following Mixer adapter:

      • 3scale Istio Adapter

    Operator overview

    Red Hat OpenShift Service Mesh requires the following four Operators:

    • OpenShift Elasticsearch - (Optional) Provides database storage for tracing and logging with Jaeger. It is based on the open source Elasticsearch project.

    • Jaeger - Provides tracing to monitor and troubleshoot transactions in complex distributed systems. It is based on the open source project.

    • Kiali - Provides observability for your service mesh. Allows you to view configurations, monitor traffic, and analyze traces in a single console. It is based on the open source Kiali project.

    • Red Hat OpenShift Service Mesh - Allows you to connect, secure, control, and observe the microservices that comprise your applications. The Service Mesh Operator defines and monitors the ServiceMeshControlPlane resources that manage the deployment, updating, and deletion of the Service Mesh components. It is based on the open source project.

    Next steps

    Optimizing routing

    The OKD HAProxy router scales to optimize performance.

    Baseline Ingress Controller (router) performance

    The OKD Ingress Controller, or router, is the Ingress point for all external traffic destined for OKD services.

    When evaluating a single HAProxy router performance in terms of HTTP requests handled per second, the performance varies depending on many factors. In particular:

    • HTTP keep-alive/close mode

    • Route type

    • TLS session resumption client support

    • Number of concurrent connections per target route

    • Number of target routes

    • Back end server page size

    • Underlying infrastructure (network/SDN solution, CPU, and so on)

    While performance in your specific environment will vary, Red Hat lab tests on a public cloud instance of size 4 vCPU/16GB RAM. A single HAProxy router handling 100 routes terminated by backends serving 1kB static pages is able to handle the following number of transactions per second.

    In HTTP keep-alive mode scenarios:

    EncryptionLoadBalancerServiceHostNetwork

    none

    21515

    29622

    edge

    16743

    22913

    passthrough

    36786

    53295

    re-encrypt

    21583

    25198

    In HTTP close (no keep-alive) scenarios:

    EncryptionLoadBalancerServiceHostNetwork

    none

    5719

    8273

    edge

    2729

    4069

    passthrough

    4121

    5344

    re-encrypt

    2320

    2941

    Default Ingress Controller configuration with ROUTER_THREADS=4 was used and two different endpoint publishing strategies (LoadBalancerService/HostNetwork) were tested. TLS session resumption was used for encrypted routes. With HTTP keep-alive, a single HAProxy router is capable of saturating 1 Gbit NIC at page sizes as small as 8 kB.

    When running on bare metal with modern processors, you can expect roughly twice the performance of the public cloud instance above. This overhead is introduced by the virtualization layer in place on public clouds and holds mostly true for private cloud-based virtualization as well. The following table is a guide to how many applications to use behind the router:

    Number of applicationsApplication type

    5-10

    static file/web server or caching proxy

    100-1000

    applications generating dynamic content

    In general, HAProxy can support routes for 5 to 1000 applications, depending on the technology in use. Ingress Controller performance might be limited by the capabilities and performance of the applications behind it, such as language or static versus dynamic content.

    Ingress, or router, sharding should be used to serve more routes towards applications and help horizontally scale the routing tier.

    Ingress Controller (router) performance optimizations

    OKD no longer supports modifying Ingress Controller deployments by setting environment variables such as ROUTER_THREADS, ROUTER_DEFAULT_TUNNEL_TIMEOUT, ROUTER_DEFAULT_CLIENT_TIMEOUT, ROUTER_DEFAULT_SERVER_TIMEOUT, and RELOAD_INTERVAL.

    You can modify the Ingress Controller deployment, but if the Ingress Operator is enabled, the configuration is overwritten.

    Post-installation RHOSP network configuration

    You can configure some aspects of a OKD on Red Hat OpenStack Platform (RHOSP) cluster after installation.

    Configuring application access with floating IP addresses

    After you install OKD, configure Red Hat OpenStack Platform (RHOSP) to allow application network traffic.

    You do not need to perform this procedure if you provided values for platform.openstack.apiFloatingIP and platform.openstack.ingressFloatingIP in the install-config.yaml file, or os_api_fip and os_ingress_fip in the inventory.yaml playbook, during installation. The floating IP addresses are already set.

    Prerequisites

    • OKD cluster must be installed

    • Floating IP addresses are enabled as described in the OKD on RHOSP installation documentation.

    Procedure

    After you install the OKD cluster, attach a floating IP address to the ingress port:

    1. Show the port:

      1. $ openstack port show <cluster_name>-<cluster_ID>-ingress-port
    2. Attach the port to the IP address:

      1. $ openstack floating ip set --port <ingress_port_ID> <apps_FIP>
    3. Add a wildcard A record for *apps. to your DNS file:

      1. *.apps.<cluster_name>.<base_domain> IN A <apps_FIP>

    If you do not control the DNS server but want to enable application access for non-production purposes, you can add these hostnames to /etc/hosts:

    1. <apps_FIP> console-openshift-console.apps.<cluster name>.<base domain>
    2. <apps_FIP> integrated-oauth-server-openshift-authentication.apps.<cluster name>.<base domain>
    3. <apps_FIP> oauth-openshift.apps.<cluster name>.<base domain>
    4. <apps_FIP> prometheus-k8s-openshift-monitoring.apps.<cluster name>.<base domain>
    5. <apps_FIP> grafana-openshift-monitoring.apps.<cluster name>.<base domain>
    6. <apps_FIP> <app name>.apps.<cluster name>.<base domain>

    A Kuryr ports pool maintains a number of ports on standby for pod creation.

    Keeping ports on standby minimizes pod creation time. Without ports pools, Kuryr must explicitly request port creation or deletion whenever a pod is created or deleted.

    The Neutron ports that Kuryr uses are created in subnets that are tied to namespaces. These pod ports are also added as subports to the primary port of OKD cluster nodes.

    Because Kuryr keeps each namespace in a separate subnet, a separate ports pool is maintained for each namespace-worker pair.

    Prior to installing a cluster, you can set the following parameters in the cluster-network-03-config.yml manifest file to configure ports pool behavior:

    • The enablePortPoolsPrepopulation parameter controls pool prepopulation, which forces Kuryr to add ports to the pool when it is created, such as when a new host is added, or a new namespace is created. The default value is false.

    • The poolMinPorts parameter is the minimum number of free ports that are kept in the pool. The default value is 1.

    • The poolMaxPorts parameter is the maximum number of free ports that are kept in the pool. A value of 0 disables that upper bound. This is the default setting.

      If your OpenStack port quota is low, or you have a limited number of IP addresses on the pod network, consider setting this option to ensure that unneeded ports are deleted.

    • The poolBatchPorts parameter defines the maximum number of Neutron ports that can be created at once. The default value is 3.

    Adjusting Kuryr ports pool settings in active deployments on RHOSP

    You can use a custom resource (CR) to configure how Kuryr manages Red Hat OpenStack Platform (RHOSP) Neutron ports to control the speed and efficiency of pod creation on a deployed cluster.

    Procedure

    1. From a command line, open the Cluster Network Operator (CNO) CR for editing:

      1. $ oc edit networks.operator.openshift.io cluster
    2. Edit the settings to meet your requirements. The following file is provided as an example:

    3. Save your changes and quit the text editor to commit your changes.