K8s DNS stack by Kubespray

Other nodes in the inventory, like external storage nodes or a separate etcd clusternode group, considered non-cluster and left up to the user to configure DNS resolve.

DNS variables

There are several global variables which can be used to modify DNS settings:

ndots

ndots value to be used in /etc/resolv.conf

It is important to note that multiple search domains combined with high ndotsvalues lead to poor performance of DNS stack, so please choose it wisely.

searchdomains

Custom search domains to be added in addition to the cluster search domains (default.svc.{{ dns_domain }}, svc.{{ dns_domain }}).

Most Linux systems limit the total number of search domains to 6 and the total length of all search domainsto 256 characters. Depending on the length of dns_domain, you’re limitted to less then the total limit.

Please note that resolvconf_mode: docker_dns will automatically add your systems search domains asadditional search domains. Please take this into the accounts for the limits.

nameservers

This variable is only used by resolvconf_mode: host_resolvconf. These nameservers are added to the hosts/etc/resolv.conf after upstream_dns_servers and thus serve as backup nameservers. If this variableis not set, a default resolver is chosen (depending on cloud provider or 8.8.8.8 when no cloud provider is specified).

upstream_dns_servers

DNS servers to be added after the cluster DNS. Used by all resolvconf_mode modes. These serve as backupDNS servers in early cluster deployment when no cluster DNS is available yet.

DNS modes supported by Kubespray

You can modify how Kubespray sets up DNS for your cluster with the variables and resolvconf_mode.

coredns (default)

This installs CoreDNS as the default cluster DNS for all queries.

coredns_dual

This installs CoreDNS as the default cluster DNS for all queries, plus a secondary CoreDNS stack.

manual

This does not install coredns, but allows you to specifymanual_dns_server, which will be configured on nodes for handling Pod DNS.Use this method if you plan to install your own DNS server in the cluster afterinitial deployment.

none

This does not install any of DNS solution at all. This basically disables cluster DNS completely andleaves you with a non functional cluster.

resolvconf_mode configures how Kubespray will setup DNS for hostNetwork: true PODs and non-k8s containers.There are three modes available:

docker_dns (default)

This sets up the docker daemon with additional —dns/—dns-search/—dns-opt flags.

The following nameservers are added to the docker daemon (in the same order as listed here):

  • cluster nameserver (depends on dns_mode)
  • content of optional upstream_dns_servers variable

The following search domains are added to the docker daemon (in the same order as listed here):

  • cluster domains (default.svc.{{ dns_domain }}, svc.{{ dns_domain }})
  • content of optional searchdomains variable
  • host system search domains (read from hosts /etc/resolv.conf)

The following dns options are added to the docker daemon

  • ndots:{{ ndots }}
  • timeout:2
  • attempts:2

For normal PODs, k8s will ignore these options and setup its own DNS settings for the PODs, takingthe —cluster_dns (either coredns or coredns_dual, depending on dns_mode) kubelet option into account.For hostNetwork: true PODs however, k8s will let docker setup DNS settings. Docker containers whichare not started/managed by k8s will also use these docker options.

host_resolvconf

This activates the classic Kubespray behaviour that modifies the hosts /etc/resolv.conf file and dhclientconfiguration to point to the cluster dns server (either coredns or coredns_dual, depending on dns_mode).

As cluster DNS is not available on early deployment stage, this mode is split into 2 stages. In the firststage (dns_early: true), /etc/resolv.conf is configured to use the DNS servers found in and nameservers. Later, /etc/resolv.conf is reconfigured to use the cluster DNS server first, leavingthe other nameservers as backups.

Also note, existing records will be purged from the /etc/resolv.conf,including resolvconf’s base/head/cloud-init config files and those that come from dhclient.

none

Does nothing regarding /etc/resolv.conf. This leaves you with a cluster that works as expected in most cases.The only exception is that hostNetwork: true PODs and non-k8s managed containers will not be able to resolvecluster service names.

Setting enable_nodelocaldns to true will make pods reach out to the dns (core-dns) caching agent running on the same node, thereby avoiding iptables DNAT rules and connection tracking. The local caching agent will query kube-dns / core-dns (depending on what main DNS plugin is configured in your cluster) for cache misses of cluster hostnames(cluster.local suffix by default).

More information on the rationale behind this implementation can be found .

  • Kubespray has yet ways to configure Kubedns addon to forward requests SkyDns cannot answer with authority to arbitrary recursive resolvers. This task is leftfor future. See official SkyDns docsfor details.

  • There isfor the SkyDNS ndots param via anoption for KubeDNSadd-on, while SkyDNS supports it though.

  • the searchdomains have a limitation of a 6 names and 256 charslength. Due to default svc, default.svc subdomains, the actuallimits are a 4 names and 239 chars respectively.