Consul Servers Outside of Kubernetes

    The below file shows how to configure the Helm chart to install Consul clients that will join an existing cluster.

    The global.enabled value first disables all chart components by default so that each component is opt-in. This allows us to only setup the client agents. We then opt-in to the client agents by setting client.enabled to true.

    Next, client.exposeGossipPorts can be set to true or false depending on if you want the clients to be exposed on the Kubernetes internal node IPs (true) or their pod IPs (false).

    Finally, client.join is set to an array of valid -retry-join values. In the example above, a fake value is specified. This should be set to resolve to the proper addresses of your existing Consul cluster.

    config.yaml

    Networking: Note that for the Kubernetes nodes to join an existing cluster, the nodes (and specifically the agent pods) must be able to connect to all other server and client agents inside and outside of Kubernetes over LAN. If this isn’t possible, consider running a separate Consul cluster inside Kubernetes and federating it with your cluster outside Kubernetes. You may also consider adopting Consul Enterprise for .

    Note: Consul on Kubernetes currently does not support external servers that require mutual authentication for the HTTPS clients of the Consul servers, that is when servers have either verify_incoming or verify_incoming_https set to true. As noted in the Security Model, that setting isn’t strictly necessary to support Consul’s threat model as it is recommended that all requests contain a valid ACL token.

    1. global:
    2.   tls:
    3.     enabled: true
    4.     enableAutoEncrypt: true
    6.   enabled: true
    7.   hosts:
    8.     - 'provider=my-cloud config=val ...'

    Consul Servers Outside Kubernetes - 图2

    config.yaml

    In most cases, externalServers.hosts will be the same as client.join, however, both keys must be set because they are used for different purposes: one for Serf LAN and the other for HTTPS connections. Please see the for more info. If your HTTPS port is different from Consul’s default 8501, you must also set externalServers.httpsPort.

    Configuring ACLs

    If you are running external servers with ACLs enabled, there are a couple of ways to configure the Helm chart to help initialize ACL tokens for Consul clients and consul-k8s components for you.

    If you would like to call the yourself or if your cluster has already been bootstrapped with ACLs, you can provide the bootstrap token to the Helm chart. The Helm chart will then use this token to configure ACLs for Consul clients and any consul-k8s components you are enabling.

    First, create a Kubernetes secret containing your bootstrap token:

    Then provide that secret to the Helm chart:

    1.   acls:
    2.     manageSystemACLs: true
    3.     bootstrapToken:
    4.       secretName: bootstrap-token
    5.       secretKey: token

    The bootstrap token requires the following minimal permissions:

    Next, configure external servers. The Helm chart will use this configuration to talk to the Consul server’s API to create policies, tokens, and an auth method. If you are , k8sAuthMethodHost should be set to the address of your Kubernetes API server so that the Consul servers can validate a Kubernetes service account token when using the Kubernetes auth method with consul login.

    Consul Servers Outside Kubernetes - 图4

    config.yaml

    Your resulting Helm configuration will end up looking similar to this:

    1. global:
    3.   acls:
    4.     manageSystemACLs: true
    5.       secretName: bootstrap-token
    6.       secretKey: token
    7. client:
    8.   enabled: true
    9.   # Set this to true to expose the Consul clients using the Kubernetes node
    10.   # IPs. If false, the pod IPs must be routable from the external servers.
    11.   exposeGossipPorts: true
    12.   join:
    13.     - 'provider=my-cloud config=val ...'
    14. externalServers:
    15.   enabled: true
    16.   hosts:

    config.yaml

    Bootstrapping ACLs via the Helm chart

    If you would like the Helm chart to call the bootstrapping API and set the server tokens for you, then the steps are similar. The only difference is that you don’t need to set the bootstrap token. The Helm chart will save the bootstrap token as a Kubernetes secret.

    config.yaml