Cluster manager

Upstream clusters and hosts are abstracted from the network/HTTP filter stack given that upstream clusters and hosts may be used for any number of different proxy tasks. The cluster manager exposes APIs to the filter stack that allow filters to obtain a L3/L4 connection to an upstream cluster, or a handle to an abstract HTTP connection pool to an upstream cluster (whether the upstream host supports HTTP/1.1, HTTP/2, or HTTP/3 is hidden). A filter stage determines whether it needs an L3/L4 connection or a new HTTP stream and the cluster manager handles all of the complexity of knowing which hosts are available and healthy, load balancing, thread local storage of upstream connection data (since most Envoy code is written to be single threaded), upstream connection type (TCP/IP, UDS), upstream protocol where applicable (HTTP/1.1, HTTP/2, HTTP/3), etc.

Clusters known to the cluster manager can be configured either statically, or fetched dynamically via the cluster discovery service (CDS) API. Dynamic cluster fetches allow more configuration to be stored in a central configuration server and thus requires fewer Envoy restarts and configuration distribution.

When clusters are initialized both at server boot as well as via CDS, they are “warmed.” This means that clusters do not become available until the following operations have taken place.

• Initial active health check pass if active health checking is configured. Envoy will send a health check request to each discovered host to determine its initial health status.

When discussing cluster warming, the cluster “becoming available” means:

• For updated clusters, the old cluster will continue to exist and serve traffic. When the new cluster has been warmed, it will be atomically swapped with the old cluster such that no traffic interruptions take place.