Architecture

Refer to the standard production deployment guide for important information, regardless of the deployment platform.

The server agents are deployed as a and use persistent volume claims to store the server state. This also ensures that the is persisted so that servers can be rescheduled onto new IP addresses without causing issues. The server agents are configured with anti-affinity rules so that they are placed on different nodes. A readiness probe is configured that marks the pod as ready only when it has established a leader.

Additionally, a PodDisruptionBudget is configured so the Consul server cluster maintains quorum during voluntary operational events. The maximum unavailable is where is the number of server agents.

Note: Kubernetes and Helm do not delete Persistent Volumes or Persistent Volume Claims when a , so this must done manually when removing servers.

Client Agents

We do not use a NodePort Kubernetes service because requests to node ports get randomly routed to any pod in the service and we need to be able to route directly to the Consul client running on our node.

Note: There is no way to bind to a local-only host port. Therefore, any other node can connect to the agent. This should be considered for security. For a properly production-secured agent with TLS and ACLs, this is safe.

Note: Due to a limitation of anti-affinity rules with DaemonSets, a client-mode agent runs alongside server-mode agents in Kubernetes. This duplication wastes some resources, but otherwise functions perfectly fine.