Operator SDK tutorial for Ansible-based Operators

    • Create a Memcached deployment

    • Ensure that the deployment size is the same as specified by the custom resource (CR) spec

    • Update the Memcached CR status using the status writer with the names of the memcached pods

    This process is accomplished by using two centerpieces of the Operator Framework:

    Operator SDK

    The operator-sdk CLI tool and controller-runtime library API

    Operator Lifecycle Manager (OLM)

    Installation, upgrade, and role-based access control (RBAC) of Operators on a cluster

    • Operator SDK CLI installed

    • Ansible version v2.9.0+

    • version v1.1.0+

    • Ansible Runner HTTP Event Emitter plug-in version v1.0.0+

    • version v0.11.2+

    • Logged into an OKD 4.8 cluster with oc with an account that has cluster-admin permissions

    • To allow the cluster pull the image, the repository where you push your image must be set as public, or you must configure an image pull secret

    Creating a project

    Use the Operator SDK CLI to create a project called memcached-operator.

    Procedure

    1. Create a directory for the project:

    2. Change to the directory:

      1. $ cd $HOME/projects/memcached-operator

    3. Run the operator-sdk init command with the ansible plug-in to initialize the project:

      1. $ operator-sdk init \
      2.     --plugins=ansible \
      3.     --domain=example.com

    Among the files generated by the operator-sdk init command is a Kubebuilder PROJECT file. Subsequent operator-sdk commands, as well as help output, that are run from the project root read this file and are aware that the project type is Ansible. For example:

    1. domain: example.com
    2. layout: ansible.sdk.operatorframework.io/v1
    3. projectName: memcached-operator
    4. version: 3-alpha

    Use the Operator SDK CLI to create a Memcached API.

    Procedure

    • Run the following command to create an API with group cache, version, v1, and kind Memcached:

      1. $ operator-sdk create api \
      2.     --group cache \
      3.     --version v1 \
      4.     --kind Memcached \
      5.     --generate-role (1)

    After creating the API, your Operator project updates with the following structure:

    Memcached CRD

    Includes a sample Memcached resource

    Manager

    Program that reconciles the state of the cluster to the desired state by using:

    • A reconciler, either an Ansible role or playbook

    • A watches.yaml file, which connects the Memcached resource to the memcached Ansible role

    Modifying the manager

    Update your Operator project to provide the reconcile logic, in the form of an Ansible role, which runs every time a Memcached resource is created, updated, or deleted.

    Procedure

    1. Update the roles/memcached/tasks/main.yml file with the following structure:

      1. ---
      2. - name: start memcached
      3.   community.kubernetes.k8s:
      4.     definition:
      5.       kind: Deployment
      6.       apiVersion: apps/v1
      7.       metadata:
      8.         name: '{{ ansible_operator_meta.name }}-memcached'
      9.         namespace: '{{ ansible_operator_meta.namespace }}'
      10.       spec:
      11.         replicas: "{{size}}"
      12.         selector:
      13.           matchLabels:
      14.             app: memcached
      15.         template:
      16.           metadata:
      17.             labels:
      18.               app: memcached
      19.           spec:
      20.             - name: memcached
      21.               command:
      22.               - memcached
      23.               - -m=64
      24.               - -o
      25.               - modern
      27.               image: "docker.io/memcached:1.4.36-alpine"
      28.               ports:
      29.                 - containerPort: 11211

      This memcached role ensures a memcached deployment exist and sets the deployment size.

      1. ---
      2. # defaults file for Memcached
      3. size: 1

    3. Update the Memcached sample resource in the config/samples/cache_v1_memcached.yaml file with the following structure:

      1. apiVersion: cache.example.com/v1
      2. kind: Memcached
      3. metadata:
      4.   name: memcached-sample
      5. spec:
      6.   size: 3

      The key-value pairs in the custom resource (CR) spec are passed to Ansible as extra variables.

    There are three ways you can use the Operator SDK CLI to build and run your Operator:

    • Run locally outside the cluster as a Go program.

    • Run as a deployment on the cluster.

    • Bundle your Operator and use Operator Lifecycle Manager (OLM) to deploy on the cluster.

    You can run your Operator project as a Go program outside of the cluster. This is useful for development purposes to speed up deployment and testing.

    Procedure

    • Run the following command to install the custom resource definitions (CRDs) in the cluster configured in your ~/.kube/config file and run the Operator locally:

      1. $ make install run

      Example output

      1. ...
      2. {"level":"info","ts":1612589622.7888272,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"cache.example.com","Options.Version":"v1","Options.Kind":"Memcached"}
      3. {"level":"info","ts":1612589622.7897573,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
      4. {"level":"info","ts":1612589622.789971,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
      5. {"level":"info","ts":1612589622.7899997,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting EventSource","source":"kind source: cache.example.com/v1, Kind=Memcached"}
      6. {"level":"info","ts":1612589622.8904517,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting Controller"}
      7. {"level":"info","ts":1612589622.8905244,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting workers","worker count":8}

    You can run your Operator project as a deployment on your cluster.

    Procedure

    1. Run the following make commands to build and push the Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

      1. Build the image:

        1. $ make docker-build IMG=<registry>/<user>/<image_name>:<tag>

      2. Push the image to a repository:

        1. $ make docker-push IMG=<registry>/<user>/<image_name>:<tag>

    2. Run the following command to deploy the Operator:

      By default, this command creates a namespace with the name of your Operator project in the form <project_name>-system and is used for the deployment. This command also installs the RBAC manifests from config/rbac.

    3. Verify that the Operator is running:

      1. $ oc get deployment -n <project_name>-system

      Example output

      1. NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
      2. <project_name>-controller-manager       1/1     1            1           8m

    Bundling an Operator

    The Operator bundle format is the default packaging method for Operator SDK and Operator Lifecycle Manager (OLM). You can get your Operator ready for use on OLM by using the Operator SDK to build and push your Operator project as a bundle image.

    Prerequisites

    • Operator SDK CLI installed on a development workstation

    • OpenShift CLI (oc) v4.8+ installed

    • Operator project initialized by using the Operator SDK

    Procedure

    1. Run the following make commands in your Operator project directory to build and push your Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

      1. Build the image:

        1. $ make docker-build IMG=<registry>/<user>/<operator_image_name>:<tag>

      2. Push the image to a repository:

        1. $ make docker-push IMG=<registry>/<user>/<operator_image_name>:<tag>

    2. Update your Makefile by setting the IMG URL to your Operator image name and tag that you pushed:

      1. $ # Image URL to use all building/pushing image targets
      2. IMG ?= <registry>/<user>/<operator_image_name>:<tag>

      This value is used for subsequent operations.

    3. Create your Operator bundle manifest by running the make bundle command, which invokes several commands, including the Operator SDK generate bundle and bundle validate subcommands:

      1. $ make bundle

      Bundle manifests for an Operator describe how to display, create, and manage an application. The make bundle command creates the following files and directories in your Operator project:

      • A bundle manifests directory named bundle/manifests that contains a ClusterServiceVersion object

      • A bundle metadata directory named bundle/metadata

      • A Dockerfile bundle.Dockerfile

      These files are then automatically validated by using to ensure the on-disk bundle representation is correct.

    4. Build and push your bundle image by running the following commands. OLM consumes Operator bundles using an index image, which reference one or more bundle images.

      1. Build the bundle image. Set BUNDLE_IMG with the details for the registry, user namespace, and image tag where you intend to push the image:

        1. $ make bundle-build BUNDLE_IMG=<registry>/<user>/<bundle_image_name>:<tag>

    Deploying an Operator with Operator Lifecycle Manager

    Operator Lifecycle Manager (OLM) helps you to install, update, and manage the lifecycle of Operators and their associated services on a Kubernetes cluster. OLM is installed by default on OKD and runs as a Kubernetes extension so that you can use the web console and the OpenShift CLI (oc) for all Operator lifecycle management functions without any additional tools.

    The Operator bundle format is the default packaging method for Operator SDK and OLM. You can use the Operator SDK to quickly run a bundle image on OLM to ensure that it runs properly.

    Prerequisites

    • Operator SDK CLI installed on a development workstation

    • Operator bundle image built and pushed to a registry

    • OLM installed on a Kubernetes-based cluster (v1.16.0 or later if you use apiextensions.k8s.io/v1 CRDs, for example OKD 4.8)

    • Logged in to the cluster with oc using an account with cluster-admin permissions

    Procedure

    1. Check the status of OLM on your cluster by using the following Operator SDK command:

      1. $ operator-sdk olm status \
      2.     --olm-namespace=openshift-operator-lifecycle-manager

    2. Run the Operator on your cluster by using the OLM integration in Operator SDK:

      1. $ operator-sdk run bundle \
      2.     [-n <namespace>] \(1)
      3.     <registry>/<user>/<bundle_image_name>:<tag>

      This command performs the following actions:

      • Create an index image referencing your bundle image. The index image is opaque and ephemeral, but accurately reflects how a bundle would be added to a catalog in production.

      • Create a catalog source that points to your new index image, which enables OperatorHub to discover your Operator.

      • Deploy your Operator to your cluster by creating an OperatorGroup, Subscription, InstallPlan, and all other required objects, including RBAC.

    Creating a custom resource

    After your Operator is installed, you can test it by creating a custom resource (CR) that is now provided on the cluster by the Operator.

    Prerequisites

    • Example Memcached Operator, which provides the Memcached CR, installed on a cluster

    Procedure

    1. Change to the namespace where your Operator is installed. For example, if you deployed the Operator using the make deploy command:

      1. $ oc project memcached-operator-system

    2. Edit the sample Memcached CR manifest at config/samples/cache_v1_memcached.yaml to contain the following specification:

    3. Create the CR:

      1. $ oc apply -f config/samples/cache_v1_memcached.yaml

    4. Ensure that the Memcached Operator creates the deployment for the sample CR with the correct size:

      1. $ oc get deployments

      Example output

      1. NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
      2. memcached-operator-controller-manager   1/1     1            1           8m
      3. memcached-sample                        3/3     3            3           1m

    5. Check the pods and CR status to confirm the status is updated with the Memcached pod names.

      1. Check the pods:

        1. $ oc get pods

        Example output

        1. NAME                                  READY     STATUS    RESTARTS   AGE
        2. memcached-sample-6fd7c98d8-7dqdr      1/1       Running   0          1m
        3. memcached-sample-6fd7c98d8-g5k7v      1/1       Running   0          1m
        4. memcached-sample-6fd7c98d8-m7vn7      1/1       Running   0          1m

      2. Check the CR status:

        1. $ oc get memcached/memcached-sample -o yaml

        Example output

        1. apiVersion: cache.example.com/v1
        2. kind: Memcached
        3. metadata:
        4. ...
        5.   name: memcached-sample
        6. ...
        7. spec:
        8.   size: 3
        9. status:
        10.   nodes:
        11.   - memcached-sample-6fd7c98d8-7dqdr
        12.   - memcached-sample-6fd7c98d8-g5k7v
        13.   - memcached-sample-6fd7c98d8-m7vn7

    6. Update the deployment size.

      1. Update config/samples/cache_v1_memcached.yaml file to change the spec.size field in the Memcached CR from 3 to 5:

        1. $ oc patch memcached memcached-sample \
        2.     -p '{"spec":{"size": 5}}' \
        3.     --type=merge

      2. Confirm that the Operator changes the deployment size:

        1. $ oc get deployments

        Example output

        1. NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
        2. memcached-operator-controller-manager   1/1     1            1           10m
        3. memcached-sample                        5/5     5            5           3m

    7. Clean up the resources that have been created as part of this tutorial.

        1. $ make undeploy

      • If you used the operator-sdk run bundle command to test the Operator, run the following command:

    • See to learn about the directory structures created by the Operator SDK.