TLS bootstrapping

    The API server should be configured with an authenticator that can authenticate tokens as a user in the group.

    This group will later be used in the controller-manager configuration to scope approvals in the default approval controller. As this feature matures, you should ensure tokens are bound to a Role-Based Access Control (RBAC) policy which limits requests (using the bootstrap token) strictly to client requests related to certificate provisioning. With RBAC in place, scoping the tokens to a group allows for great flexibility (e.g. you could disable a particular bootstrap group’s access when you are done provisioning the nodes).

    While any authentication strategy can be used for the kubelet’s initial bootstrap credentials, the following two authenticators are recommended for ease of provisioning.

    1. - alpha
    2. Token authentication file

    Using bootstrap tokens is currently alpha and will simplify the management of bootstrap token management especially in a HA scenario.

    Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example:

    head -c 16 /dev/urandom | od -An -t x | tr -d ' '

    will generate tokens that look like 02b50b05283e98dd0fd71db496ef01e8

    The token file should look like the following example, where the first three values can be anything and the quoted group name should be as depicted:

    Add the --client-ca-file=FILENAME flag to the kube-apiserver command to enable client certificate authentication, referencing a certificate authority bundle containing the signing certificate (e.g. --client-ca-file=/var/lib/kubernetes/ca.pem).

    The API for requesting certificates adds a certificate-issuing control loop to the Kubernetes Controller Manager. This takes the form of a local signer using assets on disk. Currently, all certificates issued have one year validity and a default set of key usages.

    You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates. This CA should be trusted by kube-apiserver for authentication with the --client-ca-file=FILENAME flag. The management of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes. Both certificate and key are assumed to be PEM-encoded.

    The kube-controller-manager flags are:

    1. --cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"

    In 1.7 the experimental “group auto approver” controller is dropped in favor of the new csrapproving controller that ships as part of kube-controller-manager and is enabled by default. The controller uses the to determine if a given user is authorized to request a CSR, then approves based on the authorization outcome. To prevent conflicts with other approvers, the builtin approver doesn’t explicitly deny CSRs, only ignoring unauthorized requests.

    The controller categorizes CSRs into three subresources:

    1. nodeclient - a request by a user for a client certificate with O=system:nodes and CN=system:node:(node name).
    2. selfnodeclient - a node renewing a client certificate with the same O and CN.
    3. selfnodeserver - a node renewing a serving certificate. (ALPHA, requires feature gate)

    The checks to determine if a CSR is a selfnodeserver request is currently tied to the kubelet’s credential rotation implementation, an alpha feature. As such, the definition of selfnodeserver will likely change in a future and requires the RotateKubeletServerCertificate feature gate on the controller manager. The feature progress can be tracked at kubernetes/features#267.

    The following RBAC ClusterRoles represent the nodeclient, selfnodeclient, and selfnodeserver capabilities. Similar roles may be automatically created in future releases.

    1. # REMOVED: This flag no longer works as of 1.7.

    An admin would create a ClusterRoleBinding targeting that group.

    1. # Approve all CSRs for the group "system:bootstrappers"
    2. kind: ClusterRoleBinding
    3. apiVersion: rbac.authorization.k8s.io/v1
    4. metadata:
    5.   name: auto-approve-csrs-for-group
    6. subjects:
    7. - kind: Group
    8.   name: system:bootstrappers
    9.   apiGroup: rbac.authorization.k8s.io
    10. roleRef:
    11.   kind: ClusterRole
    12.   name: approve-node-client-csr

    To let a node renew its own credentials, an admin can construct a ClusterRoleBinding targeting that node’s credentials:

    Deleting the binding will prevent the node from renewing its client credentials, effectively removing it from the cluster once its certificate expires.

    To request a client certificate from kube-apiserver, the kubelet first needs a path to a kubeconfig file that contains the bootstrap authentication token. You can use kubectl config set-cluster, set-credentials, and set-context to build this kubeconfig. Provide the name kubelet-bootstrap to kubectl config set-credentials and include --token=<token-value> as follows:

      When starting the kubelet, if the file specified by --kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On approval of the certificate request and receipt back by the kubelet, a kubeconfig file referencing the generated key and obtained certificate is written to the path specified by --kubeconfig. The certificate and key file will be placed in the directory specified by --cert-dir.

      Additionally, in 1.7 the kubelet implements alpha features for enabling rotation of both its client and/or serving certs. These can be enabled through the respective RotateKubeletClientCertificate and RotateKubeletServerCertificate feature flags on the kubelet, but may change in backward incompatible ways in future releases.

      The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an “Approved” status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl. An administrator can list CSRs with kubectl get csr and describe one in detail with kubectl describe csr <name>. Before the 1.6 release there were so an approver had to update the Status field directly (rough how-to). Later versions of Kubernetes offer kubectl certificate approve <name> and commands.