Feature and extension flags

    This document introduces two concepts:

    Feature

    A way to stage the introduction of features to the Knative API.

    Extension

    A way to extend Knative beyond the portable concepts of the Knative API.

    Features and extensions are controlled by flags.

    You can define flags in the ConfigMap in the knative-serving namespace.

    Flags can have the following values:

    Enabled

    The feature or extension is enabled and currently in use.

    Allowed

    The feature or extension is enabled and can be used, for example, by using an additional annotation or spec configuration for a resource.

    Disabled

    The feature cannot be used.

    When features and extensions are introduced to Knative, they follow a lifecycle of three stages:

    Alpha stage

    Might contain bugs.

    Support for the feature might be dropped at any time without notice.

    The API might change in a later software release in ways that make it incompatible with older releases without notice.

    Recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.

    Beta stage

    The feature is well tested and safe to enable.

    Support for the overall feature will not be dropped, though details might change.

    Recommended for only non-business-critical uses because of the potential for incompatible changes in subsequent releases. If you have multiple clusters that can be upgraded independently, you might be able to relax this restriction.

    General Availability (GA) stage

    Stable versions of the feature or extension are included in official, stable Knative releases.

    Features use flags to safely introduce new changes to the Knative API. The following definitions explain the default implementation for features at different stages:

    Alpha stage

    The feature is disabled by default, but you can manually enable it.

    Beta stage

    The feature is enabled by default, but you can manually disable it.

    GA stage

    The feature is always enabled; you cannot disable it.

    The corresponding feature flag is no longer needed and is removed from Knative.

    Extension lifecycle stages

    An extension surfaces details of a specific Knative implementation, or features of the underlying environment.

    Note

    Extensions are never included in the core Knative API due to their lack of portability.

    Each extension is always controlled by a flag and is never enabled by default.

    Alpha stage

    The feature is disabled by default, but you can manually enable it.

    Beta stage

    The feature is allowed by default.

    GA stage

    The feature is allowed by default.

    Multiple containers

    • Type: Feature
    • ConfigMap key: multi-container

    This flag allows specifying multiple user containers in a Knative Service spec.

    Only one container can handle requests, so exactly one container must have a port specified.

    Kubernetes EmptyDir Volume

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-volumes-emptydir
    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       containers:
    8.           ...
    9.           volumeMounts:
    10.             - name: cache
    11.               mountPath: /cache
    12.       volumes:
    13.         - name: cache
    14.           emptyDir: {}

    Kubernetes PersistentVolumeClaim (PVC)

    • Type: Extension
    • ConfigMap keys: kubernetes.podspec-persistent-volume-claim
      kubernetes.podspec-persistent-volume-write

    This extension controls whether PersistentVolumeClaim (PVC) can be specified and whether write access is allowed for the corresponding volume.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.  template:
    6.    spec:
    7.      containers:
    8.          ...
    9.          volumeMounts:
    10.            - mountPath: /data
    11.              name: mydata
    12.              readOnly: true
    13.      volumes:
    14.        - name: mydata
    15.          persistentVolumeClaim:
    16.            claimName: minio-pv-claim
    17.            readOnly: true

    Kubernetes node affinity

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-affinity

    This extension controls whether node affinity can be specified.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.       affinity:
    7.         nodeAffinity:
    8.           requiredDuringSchedulingIgnoredDuringExecution:
    9.             nodeSelectorTerms:
    10.             - matchExpressions:
    11.               - key: kubernetes.io/e2e-az-name
    12.                 operator: In
    13.                 values:
    14.                 - e2e-az1
    15.                 - e2e-az2
    • Type: Extension
    • ConfigMap key: kubernetes.podspec-hostaliases

    This flag controls whether host aliases can be specified.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    7.       hostAliases:
    8.       - ip: "127.0.0.1"
    9.         hostnames:
    10.         - "foo.local"
    11.         - "bar.local"

    Kubernetes node selector

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-nodeselector

    This flag controls whether node selector can be specified.

    Kubernetes toleration

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-tolerations

    This flag controls whether tolerations can be specified.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       tolerations:
    8.       - key: "example-key"
    9.         operator: "Exists"
    10.         effect: "NoSchedule"

    Kubernetes Downward API

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-fieldref

    This flag controls whether the Downward API (environment variable based) can be specified.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       containers:
    8.         - name: user-container
    9.           image: gcr.io/knative-samples/helloworld-go
    10.           env:
    11.             - name: MY_NODE_NAME
    12.               valueFrom:
    13.                 fieldRef:
    14.                   fieldPath: spec.nodeName

    Kubernetes priority class name

    • Type: extension
    • ConfigMap key: kubernetes.podspec-priorityclassname

    This flag controls whether the priorityClassName can be specified.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       priorityClassName: high-priority
    8. ...

    Kubernetes dry run

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-dryrun

    This flag controls whether Knative attempts to validate the Pod spec derived from a Knative Service spec, by using the Kubernetes API server before accepting the object.

    When this extension is enabled, the server always runs this validation.

    When this extension is allowed, the server does not run this validation by default.

    When this extension is allowed, you can run this validation for individual Services, by adding the features.knative.dev/podspec-dryrun: enabled annotation:

    1. kind: Service
    2. metadata:
    3.   annotations: features.knative.dev/podspec-dryrun: enabled
    4. ...
    • Type: Extension
    • ConfigMap key: kubernetes.podspec-runtimeclassname

    This flag controls whether the can be used.

    Kubernetes security context

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-securitycontext

    This flag controls whether a subset of the can be used.

    When set to enabled or allowed, the following PodSecurityContext properties are permitted:

    • FSGroup
    • RunAsGroup
    • RunAsNonRoot
    • SupplementalGroups
    • RunAsUser

    When set to enabled or allowed, the following container SecurityContext properties are permitted:

    • RunAsNonRoot (also allowed without this flag only when set to true)
    • RunAsUser (already allowed without this flag)

    Warning

    Use this flag with caution. PodSecurityContext properties can affect non-user sidecar containers that come from Knative or your service mesh.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       securityContext:
    8.         runAsUser: 1000
    9. ...

    Kubernetes security context capabilities

    • Type: Extension
    • ConfigMap key: kubernetes.containerspec-addcapabilities

    This flag controls whether users can add capabilities on the securityContext of the container.

    When set to enabled or allowed it allows to be added to the container.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. metadata:
    4.   name: helloworld-go
    5. spec:
    6.  template:
    7.   spec:
    8.    containers:
    9.      - image: gcr.io/knative-samples/helloworld-go
    10.        env:
    11.          - name: TARGET
    12.            value: "Go Sample v1"
    13.        securityContext:
    14.          capabilities:
    15.            add:
    16.              - NET_BIND_SERVICE

    Tag header based routing

    • Type: Extension
    • ConfigMap key: tag-header-based-routing

    This flags controls whether is enabled.

    Kubernetes init containers

    • Type: Extension
    • ConfigMap key: kubernetes.podspec-init-containers

    This flag controls whether can be used.

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. ...
    4. spec:
    5.   template:
    6.     spec:
    7.       ...
    8.       initContainers:
    9.         - name: init-myservice
    10.           image: busybox
    11.           command: ['sh', '-c', "service_setup.sh"]
    12. ...

    Queue Proxy Pod Info

    • Type: Extension
    • ConfigMap key: queueproxy.mount-podinfo

    You must set this feature to either “enabled or “allowed” when using QPOptions. The flag controls whether Knative mounts the pod-info volume to the queue-proxy container.

    Mounting the pod-info volume allows extensions that use QPOptions to access the Service annotations, by reading the /etc/podinfo/annnotations file. See for more details.

    When this feature is allowed, the pod-info volume is not mounted by default. Instead, the volume is mounted only for Services that add the features.knative.dev/queueproxy-podinfo: enabled annotation as shown below:

    1. apiVersion: serving.knative.dev/v1
    2. kind: Service
    3. metadata:
    4.   annotations: features.knative.dev/queueproxy-podinfo: enabled