Running a custom scheduler

    To schedule a given pod using a specific scheduler, specify the name of the scheduler in that specification.

    To include a custom scheduler in your cluster, include the image for a custom scheduler in a deployment.

    Prerequisites

    • You have access to the cluster as a user with the cluster-admin role.

    • You have a scheduler binary.

      Information on how to create a scheduler binary is outside the scope of this document. For an example, see in the Kubernetes documentation. Note that the actual functionality of your custom scheduler is not supported by Red Hat.

    • You have created an image containing the scheduler binary and pushed it to a registry.

    Procedure

    1. Example custom-scheduler.yaml file

    2. Create the deployment resources in the cluster:

      1. $ oc create -f custom-scheduler.yaml

    Verification

    • Verify that the scheduler pod is running:

      1. $ oc get pods -n kube-system

      The custom scheduler pod is listed as Running:

      1. NAME                                                       READY   STATUS    RESTARTS   AGE
      2. custom-scheduler-6cd7c4b8bc-854zb                          1/1     Running   0          2m

    After the custom scheduler is deployed in your cluster, you can configure pods to use that scheduler instead of the default scheduler.

    Each scheduler has a separate view of resources in a cluster. For that reason, each scheduler should operate over its own set of nodes.

    If two or more schedulers operate on the same node, they might intervene with each other and schedule more pods on the same node than there are available resources for. Pods might get rejected due to insufficient resources in this case.

    Prerequisites

    • You have access to the cluster as a user with the cluster-admin role.

    Procedure

    1. If your cluster uses role-based access control (RBAC), add the custom scheduler name to the system:kube-scheduler cluster role.

      1. Edit the system:kube-scheduler cluster role:

      2. Add the name of the custom scheduler to the resourceNames lists for the leases and endpoints resources:

        1. apiVersion: rbac.authorization.k8s.io/v1
        2. kind: ClusterRole
        3. metadata:
        4.   annotations:
        5.     rbac.authorization.kubernetes.io/autoupdate: "true"
        6.   creationTimestamp: "2021-07-07T10:19:14Z"
        7.   labels:
        8.     kubernetes.io/bootstrapping: rbac-defaults
        9.   name: system:kube-scheduler
        10.   resourceVersion: "125"
        11. rules:
        12. ...
        14.   - coordination.k8s.io
        15.   resources:
        16.   - leases
        17.   verbs:
        18.   - create
        19. - apiGroups:
        20.   - coordination.k8s.io
        21.   resourceNames:
        22.   - kube-scheduler
        23.   - custom-scheduler (1)
        24.   resources:
        25.   - leases
        26.   verbs:
        27.   - get
        28.   - update
        29. - apiGroups:
        30.   - ""
        31.   resources:
        32.   - endpoints
        33.   verbs:
        34.   - create
        35. - apiGroups:
        36.   - ""
        37.   resourceNames:
        38.   - kube-scheduler
        39.   resources:
        40.   - endpoints
        42.   - get
        43.   - update
        44. ...

    2. Create a Pod configuration and specify the name of the custom scheduler in the schedulerName parameter:

      Example custom-scheduler-example.yaml file

      1. apiVersion: v1
      2. kind: Pod
      3. metadata:
      4.   name: custom-scheduler-example
      5.   labels:
      6.     name: custom-scheduler-example
      7. spec:
      8.   schedulerName: custom-scheduler (1)
      9.   containers:
      10.   - name: pod-with-second-annotation-container
      11.     image: docker.io/ocpqe/hello-pod
      1The name of the custom scheduler to use, which is custom-scheduler in this example. When no scheduler name is supplied, the pod is automatically scheduled using the default scheduler.

    3. Create the pod:

      1. $ oc create -f custom-scheduler-example.yaml

    Verification

    1. Enter the following command to check that the pod was created:

      The custom-scheduler-example pod is listed in the output:

      1. NAME                       READY     STATUS    RESTARTS   AGE
      2. custom-scheduler-example   1/1       Running   0          4m
      1. $ oc describe pod custom-scheduler-example

      The scheduler, custom-scheduler, is listed as shown in the following truncated output:

      1. Events:
      2.   Type    Reason          Age        From                                               Message
      3.   ----    ------          ----       ----                                               -------