Install Multi-Primary on different networks

Before proceeding, be sure to complete the steps under before you begin.

If you are testing multicluster setup on kind you can use to make use of EXTERNAL-IP for LoadBalancer services.

In this configuration, both cluster1 and cluster2 observe the API Servers in each cluster for endpoints.

Service workloads across cluster boundaries communicate indirectly, via dedicated gateways for east-west traffic. The gateway in each cluster must be reachable from the other cluster.

Multiple primary clusters on separate networks

If the istio-system namespace is already created, we need to set the cluster’s network there:

Configure cluster1 as a primary

Create the Istio configuration for cluster1:

$ cat <<EOF > cluster1.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      multiCluster:
        clusterName: cluster1
      network: network1
EOF

$ istioctl install --context="${CTX_CLUSTER1}" -f cluster1.yaml

Install the east-west gateway in cluster1

Install a gateway in cluster1 that is dedicated to traffic. By default, this gateway will be public on the Internet. Production systems may require additional access restrictions (e.g. via firewall rules) to prevent external attacks. Check with your cloud vendor to see what options are available.

Zip

$ @samples/multicluster/gen-eastwest-gateway.sh@ \
    --mesh mesh1 --cluster cluster1 --network network1 | \
    istioctl --context="${CTX_CLUSTER1}" install -y -f -

If the control-plane was installed with a revision, add the --revision rev flag to the gen-eastwest-gateway.sh command.

Wait for the east-west gateway to be assigned an external IP address:

Since the clusters are on separate networks, we need to expose all services (*.local) on the east-west gateway in both clusters. While this gateway is public on the Internet, services behind it can only be accessed by services with a trusted mTLS certificate and workload ID, just as if they were on the same network.

Zip

$ kubectl --context="${CTX_CLUSTER1}" apply -n istio-system -f \
    @samples/multicluster/expose-services.yaml@

Set the default network for cluster2

If the istio-system namespace is already created, we need to set the cluster’s network there:

$ kubectl --context="${CTX_CLUSTER2}" get namespace istio-system && \
  kubectl --context="${CTX_CLUSTER2}" label namespace istio-system topology.istio.io/network=network2

Configure cluster2 as a primary

Create the Istio configuration for cluster2:

$ cat <<EOF > cluster2.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
    global:
      multiCluster:
        clusterName: cluster2
      network: network2
EOF

As we did with cluster1 above, install a gateway in cluster2 that is dedicated to east-west traffic.

Zip

$ @samples/multicluster/gen-eastwest-gateway.sh@ \
    --mesh mesh1 --cluster cluster2 --network network2 | \
    istioctl --context="${CTX_CLUSTER2}" install -y -f -

Wait for the east-west gateway to be assigned an external IP address:

$ kubectl --context="${CTX_CLUSTER2}" get svc istio-eastwestgateway -n istio-system
NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)   AGE
istio-eastwestgateway   LoadBalancer   10.0.12.121   34.122.91.98   ...       51s

Expose services in cluster2

As we did with cluster1 above, expose services via the east-west gateway.

Zip

$ kubectl --context="${CTX_CLUSTER2}" apply -n istio-system -f \
    @samples/multicluster/expose-services.yaml@

Enable Endpoint Discovery

Install a remote secret in cluster2 that provides access to cluster1’s API server.

Install a remote secret in cluster1 that provides access to cluster2’s API server.

$ istioctl x create-remote-secret \
  --context="${CTX_CLUSTER2}" \
  kubectl apply -f - --context="${CTX_CLUSTER1}"

Congratulations! You successfully installed an Istio mesh across multiple primary clusters on different networks!