Customize Calico configuration

    Concepts

    Calico is installed by an operator which manages the installation, upgrade, and general lifecycle of a Calico cluster. The operator is installed directly on the cluster as a Deployment, and is configured through one or more custom Kubernetes API resources.

    Calico manifests

    Calico can also be installed using raw manifests as an alternative to the operator. The manifests contain the necessary resources for installing Calico on each node in your Kubernetes cluster. Using manifests is not recommended as they cannot automatically manage the lifecycle of the Calico as the operator does. However, manifests may be useful for clusters that require highly specific modifications to the underlying Kubernetes resources.

    How to

    • Operator
    • Manifest

    Operator installations read their configuration from a specific set of Kubernetes APIs. These APIs are installed on the cluster as part of in the operator.tigera.io/v1 API group.

    • Installation: a singleton resource with name “default” that configures common installation parameters for a Calico cluster.
    • : a singleton resource with name “default” that configures installation of the Calico API server extension.

    Configure the pod IP range

    For many environments, Calico will auto-detect the correct pod IP range to use, or select an unused range on the cluster.

    You can select a specific pod IP range by modifying the spec.calicoNetwork.ipPools array in the Installation API resource.

    note

    The the ipPools array can take at most one IPv4 and one IPv6 CIDR, and only takes effect when installing Calico for the first time on a given cluster. To add additional pools, see .

    You can enable VXLAN in a cluster by setting the option on your IPv4 pool. You can also disable BGP via the spec.calicoNetwork.bgp field.

    1. kind: Installation
    2. apiVersion: operator.tigera.io/v1
    3. metadata:
    4.   name: default
    5. spec:
    6.   calicoNetwork:
    7.     bgp: Disabled
    8.     ipPools:
    9.       - cidr: 198.51.100.0/24
    10.         encapsulation: VXLAN

    We provide a number of manifests to make deployment of Calico easy. You can optionally modify the manifests before applying them. Or you can modify the manifest and reapply it to change settings as needed.

    About customizing Calico manifests

    Each manifest contains all the necessary resources for installing Calico on each node in your Kubernetes cluster.

    It installs the following Kubernetes resources:

    • Installs the calico/node container on each host using a DaemonSet.
    • Installs the Calico CNI binaries and network config on each host using a DaemonSet.
    • Runs calico/kube-controllers as a deployment.
    • The calico-etcd-secrets secret, which optionally allows for providing etcd TLS assets.

    The sections that follow discuss the configurable parameters in greater depth.

    Calico IPAM assigns IP addresses from .

    To change the default IP range used for pods, modify the CALICO_IPV4POOL_CIDR section of the calico.yaml manifest. For more information, see Configuring calico/node.

    • Their cluster is running in a properly configured AWS VPC.
    • All their Kubernetes nodes are connected to the same layer 2 network.
    • They intend to use BGP peering to make their underlying infrastructure aware of pod IP addresses.

    To disable IP-in-IP encapsulation, modify the CALICO_IPV4POOL_IPIP section of the manifest. For more information, see .

    Switching from IP-in-IP to VXLAN

    By default, the Calico manifests enable IP-in-IP encapsulation. If you are on a network that blocks IP-in-IP, such as Azure, you may wish to switch to . To do this at install time (so that Calico creates the default IP pool with VXLAN and no IP-in-IP configuration has to be undone):

    • Start with one of the Calico for policy and networking manifests.
    • Replace environment variable name CALICO_IPV4POOL_IPIP withCALICO_IPV4POOL_VXLAN. Leave the value of the new variable as “Always”.
    • Optionally, (to save some resources if you’re running a VXLAN-only cluster) completely disable Calico’s BGP-based networking:
      • Replace calico_backend: "bird" with calico_backend: "vxlan". This disables BIRD.
      • Comment out the line - -bird-ready and - -bird-live from the calico/node readiness/liveness check (otherwise disabling BIRD will cause the readiness/liveness check to fail on every node):
    1. livenessProbe:
    2.   exec:
    3.     command:
    4.       - /bin/calico-node
    5.       - -felix-live
    6.       - -bird-live
    7. readinessProbe:
    8.   exec:
    9.     command:
    10.       - /bin/calico-node
    11.         - -bird-ready
    12.       - -felix-ready

    For more information on calico/node’s configuration variables, including additional VXLAN settings, see .

    Customize Calico configuration - 图2note

    The CALICO_IPV4POOL_VXLAN environment variable only takes effect when the first calico/node to start creates the default IP pool. It has no effect after the pool has already been created. To switch to VXLAN mode after installation time, use calicoctl to modify the IPPool resource.

    Configuring etcd

    By default, these manifests do not configure secure access to etcd and assume an etcd proxy is running on each host. The following configuration options let you specify custom etcd cluster endpoints as well as TLS.

    The following table outlines the supported options for etcd:

    To use these manifests with a TLS-enabled etcd cluster you must do the following:

    1. Download the v3.24 manifest that corresponds to your installation method.

      Calico for policy and networking

      1. curl https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/calico-etcd.yaml -O

      Calico for policy and flannel for networking

    2. Within the ConfigMap section, uncomment the etcd_ca, etcd_key, and etcd_cert lines so that they look as follows.

      1. etcd_ca: '/calico-secrets/etcd-ca'
      2. etcd_cert: '/calico-secrets/etcd-cert'
      3. etcd_key: '/calico-secrets/etcd-key'

    3. Ensure that you have three files, one containing the etcd_ca value, another containing the etcd_key value, and a third containing the etcd_cert value.

    4. Using a command like the following to strip the newlines from the files and base64-encode their contents.

      1. cat <file> | base64 -w 0
      1. apiVersion: v1
      2. kind: Secret
      3. type: Opaque
      4. metadata:
      5.   name: calico-etcd-secrets
      6. data:
      7.    Populate the following files with etcd TLS configuration if desired, but leave blank if
      8.    not using TLS for etcd.
      9.    This self-hosted install expects three files with the following names.  The values
      10.    should be base64 encoded strings of the entire contents of each file.
      11.   etcd-key: LS0tLS1CRUdJTiB...VZBVEUgS0VZLS0tLS0=
      12.   etcd-cert: LS0tLS1...ElGSUNBVEUtLS0tLQ==
      13.   etcd-ca: LS0tLS1CRUdJTiBD...JRklDQVRFLS0tLS0=

    6. Apply the manifest.

      Calico for policy and networking

      Calico for policy and flannel for networking

      1. kubectl apply -f canal.yaml

    Calico’s manifests assign its components one of two service accounts. Depending on your cluster’s authorization mode, you’ll want to back these service accounts with the necessary permissions.

    Other configuration options

    The following table outlines the remaining supported ConfigMap options.

    CNI network configuration template

    The cni_network_config configuration option supports the following template fields, which will be filled in automatically by the calico/cni container:

    Instead of installing from our pre-modified Istio manifests, you may wish to customize your Istio install or use a different Istio version. This section walks you through the necessary changes to a generic Istio install manifest to allow application layer policy to operate.

    The standard Istio manifests for the sidecar injector include a ConfigMap that contains the template used when adding pods to the cluster. The template adds an init container and the Envoy sidecar. Application layer policy requires an additional lightweight sidecar called Dikastes which receives Calico policy from Felix and applies it to incoming connections and requests.

    If you haven’t already done so, download an Istio release and untar it to a working directory.

    Open the install/kubernetes/istio-demo-auth.yaml file in an editor, and locate the istio-sidecar-injector ConfigMap. In the existing istio-proxy container, add a new volumeMount.

    1. - mountPath: /var/run/dikastes
    2.   name: dikastes-sock

    Add a new container to the template.

    1.       - name: dikastes
    2.         image: calico/dikastes:v3.24.5
    3.         args: ["server", "-l", "/var/run/dikastes/dikastes.sock", "-d", "/var/run/felix/nodeagent/socket"]
    4.         securityContext:
    5.           allowPrivilegeEscalation: false
    6.         livenessProbe:
    7.           exec:
    8.             command:
    9.             - /healthz
    10.             - liveness
    11.           initialDelaySeconds: 3
    12.           periodSeconds: 3
    13.         readinessProbe:
    14.           exec:
    15.             command:
    16.             - /healthz
    17.             - readiness
    18.           initialDelaySeconds: 3
    19.           periodSeconds: 3
    20.         volumeMounts:
    21.         - mountPath: /var/run/dikastes
    22.           name: dikastes-sock
    23.         - mountPath: /var/run/felix

    Add two new volumes.

    The volumes you added are used to create Unix domain sockets that allow communication between Envoy and Dikastes and between Dikastes and Felix. Once created, a Unix domain socket is an in-memory communications channel. The volumes are not used for any kind of stateful storage on disk.

    Refer to the for an example with the above changes.