Configuring ExternalIPs for services

This functionality is generally most useful for clusters installed on bare-metal hardware.

• Your network infrastructure must route traffic for the external IP addresses to your cluster.

For non-cloud environments, OKD supports the assignment of external IP addresses to a object spec.externalIPs[] field through the ExternalIP facility. By setting this field, OKD assigns an additional virtual IP address to the service. The IP address can be outside the service network defined for the cluster. A service configured with an ExternalIP functions similarly to a service with type=NodePort, allowing you to direct traffic to a local node for load balancing.

You must configure your networking infrastructure to ensure that the external IP address blocks that you define are routed to the cluster.

OKD extends the ExternalIP functionality in Kubernetes by adding the following capabilities:

• Restrictions on the use of external IP addresses by users through a configurable policy

• Allocation of an external IP address automatically to a service upon request

You can assign an external IP address in the following ways:

Automatic assignment of an external IP

OKD automatically assigns an IP address from the autoAssignCIDRs CIDR block to the spec.externalIPs[] array when you create a Service object with spec.type=LoadBalancer set. In this case, OKD implements a non-cloud version of the load balancer service type and assigns IP addresses to the services. Automatic assignment is disabled by default and must be configured by a cluster administrator as described in the following section.

Manual assignment of an external IP

OKD uses the IP addresses assigned to the spec.externalIPs[] array when you create a Service object. You cannot specify an IP address that is already in use by another service.

Use of an external IP address in OKD is governed by the following fields in the Network.config.openshift.io CR named cluster:

• spec.externalIP.autoAssignCIDRs defines an IP address block used by the load balancer when choosing an external IP address for the service. OKD supports only a single IP address block for automatic assignment. This can be simpler than having to manage the port space of a limited number of shared IP addresses when manually assigning ExternalIPs to services. If automatic assignment is enabled, a Service object with spec.type=LoadBalancer is allocated an external IP address.

• spec.externalIP.policy defines the permissible IP address blocks when manually specifying an IP address. OKD does not apply policy rules to IP address blocks defined by spec.externalIP.autoAssignCIDRs.

If routed correctly, external traffic from the configured external IP address block can reach service endpoints through any TCP or UDP port that the service exposes.

OKD supports both the automatic and manual assignment of IP addresses, and each address is guaranteed to be assigned to a maximum of one service. This ensures that each service can expose its chosen ports regardless of the ports exposed by other services.

The following YAML describes a service with an external IP address configured:

Example Service object with spec.externalIPs[] set

As a cluster administrator, you can specify IP address blocks to allow and to reject.

Restrictions apply only to users without cluster-admin privileges. A cluster administrator can always set the service spec.externalIPs[] field to any IP address.

You configure IP address policy with a policy object defined by specifying the spec.ExternalIP.policy field. The policy object has the following shape:

{
  "policy": {
    "allowedCIDRs": [],
    "rejectedCIDRs": []
  }
}

When configuring policy restrictions, the following rules apply:

• If policy={} is set, then creating a Service object with spec.ExternalIPs[] set will fail. This is the default for OKD. The behavior when policy=null is set is identical.

• If policy is set and either policy.allowedCIDRs[] or policy.rejectedCIDRs[] is set, the following rules apply:

  • If allowedCIDRs[] and rejectedCIDRs[] are both set, then rejectedCIDRs[] has precedence over allowedCIDRs[].

  • If allowedCIDRs[] is set, creating a object with spec.ExternalIPs[] will succeed only if the specified IP addresses are allowed.

  • If rejectedCIDRs[] is set, creating a Service object with spec.ExternalIPs[] will succeed only if the specified IP addresses are not rejected.

The examples that follow demonstrate several different policy configurations.

• In the following example, the policy prevents OKD from creating any service with an external IP address specified:

  Example policy to reject any value specified for Service object spec.externalIPs[]

  apiVersion: config.openshift.io/v1
  kind: Network
  metadata:
  spec:
    externalIP:
      policy: {}
    ...

• In the following example, both the allowedCIDRs and rejectedCIDRs fields are set.

  Example policy that includes both allowed and rejected CIDR blocks

  apiVersion: config.openshift.io/v1
  kind: Network
  metadata:
    name: cluster
  spec:
    externalIP:
      policy:
        allowedCIDRs:
        - 172.16.66.10/23
        rejectedCIDRs:
        - 172.16.66.10/24
    ...

• In the following example, policy is set to null. If set to null, when inspecting the configuration object by entering oc get networks.config.openshift.io -o yaml, the policy field will not appear in the output.

  Example policy to allow any value specified for Service object spec.externalIPs[]

The configuration for ExternalIP address blocks is defined by a Network custom resource (CR) named cluster. The Network CR is part of the config.openshift.io API group.

The following YAML describes the ExternalIP configuration:

Network.config.openshift.io CR named cluster

apiVersion: config.openshift.io/v1
metadata:
  name: cluster
spec:
  externalIP:
    autoAssignCIDRs: [] (1)
    policy: (2)
      ...

The following YAML describes the fields for the policy stanza:

Network.config.openshift.io policy stanza

policy:
  allowedCIDRs: [] (1)

Several possible configurations for external IP address pools are displayed in the following examples:

• The following YAML describes a configuration that enables automatically assigned external IP addresses:

  Example configuration with spec.externalIP.autoAssignCIDRs set

  apiVersion: config.openshift.io/v1
  kind: Network
  metadata:
    name: cluster
  spec:
    ...
    externalIP:
      autoAssignCIDRs:
      - 192.168.132.254/29

• The following YAML configures policy rules for the allowed and rejected CIDR ranges:

  Example configuration with spec.externalIP.policy set

As a cluster administrator, you can configure the following ExternalIP settings:

• An ExternalIP address block used by OKD to automatically populate the spec.clusterIP field for a Service object.

• A policy object to restrict what IP addresses may be manually assigned to the spec.clusterIP array of a Service object.

Prerequisites

• Install the OpenShift CLI (oc).

• Access to the cluster as a user with the cluster-admin role.

Procedure

1. Optional: To display the current external IP configuration, enter the following command:

   $ oc describe networks.config cluster

2. To edit the configuration, enter the following command:

   $ oc edit networks.config cluster

3. Modify the ExternalIP configuration, as in the following example:

   apiVersion: config.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     ...
     externalIP: (1)
   

   1	Specify the configuration for the externalIP stanza.