Install Calico for Windows

    Extend your Kubernetes deployment to Windows environments.

    Required

    • Install and configure

    • Linux and Windows nodes meet requirements

    • If using Calico networking, copy the kubeconfig file (used by kubelet) to each Windows node to the file, .

    • Download Calico for Windows and Kubernetes binaries to each Windows nodes to prepare for install:

      On each of your Windows nodes, download and run Calico for Windows installation scripts:

      cd into C:\CalicoWindows, you will see the calico-node.exe binary, install scripts, and other files.

    Because the Kubernetes and Calico control components do not run on Windows yet, a hybrid Linux/Windows cluster is required. First you create a Linux cluster for Calico components, then you join Windows nodes to the Linux cluster.

    The geeky details of what you get by default:

    Kubernetes

    1. Create a Linux cluster
    3. Prepare Windows nodes to join the Linux cluster

    Calico

    2. Install Calico and Kubernetes on Windows nodes

    There are many ways to create a Linux Kubernetes cluster. We regularly test Calico for Windows with kubeadm.

    A primary issue of running a hybrid Kubernetes cluster is that many Kubernetes manifests do not specify a node selector to restrict where their pods can run. For example, kubeadm installs kube-proxy (Kubernetes per-host NAT daemon) using a DaemonSet that does not include a node selector. This means that the kube-proxy pod, which only supports Linux, will be scheduled to both Linux and Windows nodes. Services/pods that should run only on Linux nodes (such as the kube-proxy DaemonSet) should be started with a node selector to avoid attempting to schedule them to Windows nodes.

    To get around this for kube-proxy:

    1. Use kubectl to retrieve the DaemonSet.

      1.  kubectl get ds kube-proxy -n kube-system -o yaml > kube-proxy.yaml

    2. Modify the kube-proxy.yaml file to include a node selector that selects only Linux nodes:

      1. spec:
      2.   template:
      3.   ...
      4.     spec:
      5.       nodeSelector:
      6.         beta.kubernetes.io/os: linux
      7.       containers:

    A similar change may be needed for other Kubernetes services (such as kube-dns or core-dns).

    On each Windows node, follow the steps below to configure kubelet and kube-proxy service.

    Step 1: Configure kubelet

    For Docker:

    • --network-plugin=cni
    • --cni-bin-dir=<directory for CNI binaries>

    For containerd:

    • --container-runtime=remote
    • --container-runtime-endpoint=npipe:////.//pipe//containerd-containerd

    The CNI bin and conf dir settings are required by the Calico installer to install the CNI binaries and configuration file.

    note

    Among other parameters, the containerd configuration file includes options to configure the CNI bin and conf dirs.

    The following kubelet settings are also important:

    • --hostname-override can be set to $(hostname) to match Calico’s default. kubelet and Calico must agree on the host/nodename; if your network environment results in hostnames that vary over time you should set the hostname override to a static value per host and update Calico’s nodename accordingly.

    • --node-ip should be used to explicitly set the IP that kubelet reports to the API server for the node. We recommend setting this to the host’s main network adapter’s IP since we’ve seen kubelet incorrectly use an IP assigned to a HNS bridge device rather than the host’s network adapter.

    • Because of a Windows networking limitation, if using Calico IPAM, —max-pods should be set to, at most, the IPAM block size of the IP pool in use minus 4:

    In addition, it’s important that kubelet is started after the vSwitch has been created, which happens when Calico initializes the dataplane. Otherwise, kubelet can be disconnected for the API server when the vSwitch is created.

    AWS users: If using the AWS cloud provider, you should add the following argument to the kubelet:

    --hostname-override=<aws instance private DNS name> (and set the Calico nodename variable to match). In addition, you should add KubernetesCluster=<cluster-name> as a tag when creating your Windows instance.

    As a quickstart, the Calico package includes a sample script at C:\CalicoWindows\kubernetes\kubelet-service.ps1 that:

    • Waits for Calico to initialise the vSwitch
    • Starts kubelet with
      • If containerd service is running, the following flags are set:
        • --container-runtime set to remote
        • --container-runtime-endpoint set to npipe:////.//pipe//containerd-containerd
      • Otherwise, the following flags are set for Docker:
        • --network-plugin set to cni
        • --cni-bin-dir set to c:\k\cni
        • --cni-conf-dir set to c:\k\cni\config
        • --pod-infra-container-image set to kubeletwin/pause
      • --kubeconfig set to the path of node kubeconfig file
      • --hostname-override set to match Calico’s nodename
      • --node-ip set to the IP of the default vEthernet device
      • --cluster-dns set to the IPs of the dns name servers

    See the README in the same directory for more details. Feel free to modify the script to adjust other kubelet parameters.

    Install Calico for Windows - 图2note

    The script will pause at the first stage until Calico is installed by following the instructions in the next section.

    Step 2: Configure kube-proxy

    kube-proxy must be configured as follows:

    • With the correct HNS network name used by the active CNI plugin. kube-proxy reads the HNS network name from an environment variable KUBE_NETWORK
      • With default configuration, Calico uses network name “Calico”
    • For VXLAN, with the source VIP for the pod subnet allocated to the node. This is the IP that kube-proxy uses when it does SNAT for a NodePort. For Calico, the source VIP should be the second IP address in the subnet chosen for the host. For example, if Calico chooses an IP block 10.0.0.0/26 then the source VIP should be 10.0.0.2. The script below will automatically wait for the block to be chosen and configure kube-proxy accordingly.
    • For Calico policy to function correctly with Kubernetes services, the WinDSR feature gate must be enabled. This requires Windows Server build 17763.1432 or greater and Kubernetes v1.14 or greater. Calico will automatically enable the WinDSR feature gate if kubernetes services are managed by Calico for Windows.

    kube-proxy should be started via a script that waits for the Calico HNS network to be provisioned. The Calico package contains a suitable script for use with Calico networking at C:\CalicoWindows\kubernetes\kube-proxy-service.ps1. The script:

    • Waits for Calico to initialise the vSwitch.
    • Calculates the correct source VIP for the local subnet.

    See the README in the same directory for more details. Feel free to modify the script to adjust other kube-proxy parameters.

    The script will pause at the first stage until Calico is installed by following the instructions in the next section.

    1. Disable the default Calico IP-in-IP networking (which is not compatible with Windows), by modifying the Calico manifest, and setting the CALICO_IPV4POOL_IPIP environment variable to “Never” before applying the manifest.

      If you do apply the manifest with the incorrect value, changing the manifest and re-applying will have no effect. To adjust the already-created IP pool:

      1. calicoctl get ippool -o yaml > ippool.yaml

      Then, modify ippool.yaml by setting the ipipMode to and then apply the updated manifest:

    If using Calico VXLAN networking

    1. Modify VXLAN as described in guide. Note the following:

      • Windows can support only a single type of IP pool so it is important that you use only a single VXLAN IP pool in this mode.
      • Windows supports only VXLAN on port 4789 and VSID >=4096. Calico’s default (on Linux and Windows) is to use port 4789 and VSID 4096.

    2. Apply the manifest using calicoctl, and verify that you have a single pool with VXLANMODE Always.

      1. $ calicoctl get ippool -o wide

    3. For Linux control nodes using Calico networking, strict affinity must be set to true. This is required to prevent Linux nodes from borrowing IP addresses from Windows nodes:

      1. kubectl patch ipamconfigurations default --type merge --patch='{"spec": {"strictAffinity": true}}'

    Follow the steps below on each Windows node to install Kubernetes and Calico:

    If using Calico BGP

    Install the RemoteAccess service using the following PowerShell commands:

    1. Install-WindowsFeature RemoteAccess
    2. Install-WindowsFeature RSAT-RemoteAccess-PowerShell
    3. Install-WindowsFeature Routing

    Then restart the computer:

    1. Restart-Computer -Force

    before running:

    Sometimes the remote access service fails to start automatically after install. To make sure it is running, execute the following command:

    1. Start-Service RemoteAccess

    1. If using a non-Calico network plugin for networking, install and verify it now.

    2. Edit the install configuration file, config.ps1 as follows:

    3. Run the installer.

      • Change directory to the location that you unpacked the archive. For example:
    1. cd C:\CalicoWindows
    • Run the install script:
    1. .\install-calico.ps1

    note

    The installer initializes the Windows vSwitch, which can cause a short connectivity outage as the networking stack is reconfigured. After running that command, you may need to:

    • Reconnect to your remote desktop session.
    • Restart kubelet and kube-proxy if they were already running.
    • If you haven’t started kubelet and kube-proxy already, you should do so now. The quickstart scripts provided in the Calico package provide an easy way to do this. Calico requires kubelet to be running to complete its per-node configuration (since Kubelet creates the Kubernetes Node resource).

    Install Calico for Windows - 图4note

    After you run the installer, do not move the directory because the service registration refers to the path of the directory.

    1. Verify that the Calico services are running.

      1. Get-Service -Name CalicoNode
      2. Get-Service -Name CalicoFelix