Prerequisites

    Kubernetes v1.10 or higher is supported by Rook.

    Privileges and RBAC

    Rook requires privileges to manage the storage in your cluster. See the details for setting up Rook in a Kubernetes cluster with Pod Security Policies enabled.

    Flexvolume Configuration

    The Rook agent requires setup as a Flex volume plugin to manage the storage attachments in your cluster. See the Flex Volume Configuration topic to configure your Kubernetes deployment to load the Rook volume plugin.

    Rook Ceph requires a Linux kernel built with the RBD module. Many distributions of Linux have this module but some don’t, e.g. the GKE Container-Optimised OS (COS) does not have RBD. You can test your Kubernetes nodes by running . If it says ‘not found’, you may have to or choose a different Linux distribution.

    Kernel modules directory configuration

    Normally, on Linux, kernel modules can be found in /lib/modules. However, there are some distributions that put them elsewhere. In that case the environment variable LIB_MODULES_DIR_PATH can be used to override the default. Also see the documentation in helm-operator on the parameter agent.libModulesDirPath. One notable distribution where this setting is useful would be .

    Extra agent mounts

    On certain distributions it may be necessary to mount additional directories into the agent container. That is what the environment variable AGENT_MOUNTS is for. Also see the documentation in helm-operator on the parameter agent.mounts. The format of the variable content should be mountname1=/host/path1:/container/path1,mountname2=/host/path2:/container/path2.

    Some Linux distributions do not ship with the lvm2 package. This package is required on all storage nodes in your k8s cluster. Please install it using your Linux distribution’s package manager; for example:

    Bootstrapping Kubernetes

    Rook will run wherever Kubernetes is running. Here are some simple environments to help you get started with Rook.

    1. $ minikube start
    2. Starting local Kubernetes cluster...
    3. Starting VM...
    4. SSH-ing files into VM...
    5. Setting up certs...
    6. Starting cluster components...
    7. Connecting to cluster...
    8. Setting up kubeconfig...

    After these steps, your minikube cluster is ready to install Rook on.

    You can easily spin up Rook on top of a kubeadm cluster. You can find the instructions on how to install kubeadm in the page.

    By using kubeadm, you can use Rook in just a few minutes!

    For a quick start with a new local cluster, use the Rook fork of coreos-kubernetes. This will bring up a multi-node Kubernetes cluster with vagrant and CoreOS virtual machines ready to use Rook immediately.

    Then wait for the cluster to come up and verify that kubernetes is done initializing (be patient, it takes a bit):

    1. kubectl cluster-info

    Once you see a url response, your cluster is .

    Support for authenticated docker registries

    If you want to use an image from authenticated docker registry (e.g. for image cache/mirror), you’ll need to add an imagePullSecret to all relevant service accounts. This way all pods created by the operator (for service account: rook-ceph-system) or all new pods in the namespace (for service account: default) will have the imagePullSecret added to their spec.

    The whole process is described in the official kubernetes documentation.

    First, we’ll create the secret for our registry as described :

    Next we’ll add the following snippet to all relevant service accounts as described here:

    1. - name: my-registry-secret

    The service accounts are:

    • rook-ceph-system (namespace: rook-ceph): Will affect all pods created by the rook operator in the rook-ceph namespace.
    • default (namespace: rook-ceph): Will affect most pods in the rook-ceph namespace.
    • (namespace: rook-ceph): Will affect the MGR pods in the rook-ceph namespace.
    • rook-ceph-osd (namespace: rook-ceph): Will affect the OSD pods in the rook-ceph namespace.

    You can do it either via e.g. kubectl -n <namespace> edit serviceaccount default or by modifying the and cluster.yaml before deploying them.

    Since it’s the same procedure for all service accounts, here is just one example:

    1. apiVersion: v1
    2. kind: ServiceAccount
    3. metadata:
    4.   name: default
    5.   namespace: rook-ceph
    6. secrets:
    7. - name: default-token-12345
    8. - name: my-registry-secret       # parts

    After doing this for all service accounts all pods should be able to pull the image from your registry.

    Now that you have a Kubernetes cluster running, you can start using Rook with .

    Using Rook on Tectonic Bare Metal

    Follow these instructions to run Rook on Tectonic Kubernetes