Understanding TargetRef policies

    What do targetRef policies look like?

    There are two parts in a policy:

    1. The metadata
    2. The spec

    Metadata identifies the policies by its name, type and what mesh it is part of.

    This is how it looks:

    A policy metadata looks like:

    In Kubernetes all our policies are implemented as in the group kuma.io/v1alpha1.

    A policy metadata looks like:

    1. apiVersion: kuma.io/v1alpha1
    2. kind: ExamplePolicy
    3. metadata:
    4.   name: my-policy-name
    5.   namespace: kuma-system
    6. spec:
    7.   ... # spec data specific to the policy kind

    By default the policy is created in the default mesh. You can specify the mesh by using the kuma.io/mesh label.

    For example:

    1. apiVersion: kuma.io/v1alpha1
    2. kind: ExamplePolicy
    3. metadata:
    4.   name: my-policy-name
    5.   namespace: kuma-system
    6.   labels:
    7.     kuma.io/mesh: "my-mesh"
    8. spec:
    9.   ... # spec data specific to the policy kind

    Policies are namespaced scope and currently the namespace must be the one the control-plane is running in kuma-system by default.

    The spec contains the actual configuration of the policy.

    All specs have a top level targetRef which identifies which proxies this policy applies to. In particular, it defines which proxies have their Envoy configuration modified.

    Some policies also support further narrowing.

    The actual configuration is defined in a default map.

    For example:

    1. type: ExamplePolicy
    2. name: my-example
    3. mesh: default
    4. spec:
    5.   targetRef:
    6.     kind: Mesh
    7.   to:
    8.     - targetRef:
    9.         kind: Mesh
    10.       default: # Configuration that applies to outgoing traffic
    11.         key: value
    12.   from:
    13.     - targetRef:
    14.       default: # Configuration that applies to incoming traffic
    15.         key: value

    Some policies are not directional and will not have to and from. For example

    One of the benefits of targetRef policies is that the spec is always the same between Kubernetes and Universal.

    This means that converting policies between Universal and Kubernetes only means rewriting the metadata.

    Writing a targetRef

    targetRef is a concept borrowed from Kubernetes Gateway API its usage is fully defined in . Its goal is to select subsets of proxies with maximum flexibility.

    It looks like:

    1. targetRef:
    2.   kind: Mesh | MeshSubset | MeshService | MeshServiceSubset | MeshGatewayRoute
    3.   name: "my-name" # For kinds MeshService, MeshServiceSubset and MeshGatewayRoute a name can be defined
    4.   tags:
    5.     key: value # For kinds MeshServiceSubset and MeshSubset a list of matching tags can be used

    Here’s an explanation of each kinds and their scope:

    • Mesh: applies to all proxies running in the mesh
    • MeshSubset: same as Mesh but filters only proxies who have matching targetRef.tags
    • MeshService: all proxies with a tag kuma.io/service equal to targetRef.name
    • MeshServiceSubset: same as but further refine to proxies that have matching targetRef.tags
    • MeshGatewayRoute: gateway using MeshGatewayRoute that have a name equal to targetRef.name

    Consider the example below:

    1. apiVersion: kuma.io/v1alpha1
    2. kind: MeshAccessLog
    3. metadata:
    4.   name: example
    5.   namespace: kuma-system
    6.   labels:
    7.     kuma.io/mesh: default
    8. spec:
    9.   targetRef: # top level targetRef
    10.     kind: MeshService
    11.     name: web-frontend
    12.   to:
    13.     - targetRef: # to level targetRef
    14.         kind: MeshService
    15.         name: web-backend
    16.       default:
    17.         backends:
    18.           - file:
    19.               format:
    20.                 plain: '{"start_time": "%START_TIME%"}'
    21.               path: "/tmp/logs.txt"
    22.   from:
    23.     - targetRef: # from level targetRef
    24.         kind: Mesh
    25.       default:
    26.         backends:
    27.           - file:
    28.               format:
    29.                 plain: '{"start_time": "%START_TIME%"}'
    30.               path: "/tmp/logs.txt"

    Using spec.targetRef, this policy targets all proxies that implement the service web-frontend. It defines the scope of this policy as applying to traffic either from or to web-frontend services.

    The spec.to.targetRef section enables logging for any traffic going to web-backend. The spec.from.targetRef section enables logging for any traffic coming from any service in the Mesh.

    Not every policy supports to and from levels. Additionally, not every resource can appear at every supported level. The specified top level resource can also affect which resources can appear in to or from.

    This table looks like:

    Here it indicates that the top level can use any targetRef kinds. But in targetRef.to only kind Mesh can be used and in targetRef.from only kind MeshService.

    It is necessary to define a policy for merging configuration, because a proxy can be targeted by multiple targetRef’s.

    We define a total order of policies:

    • Mesh > MeshSubset > MeshService > MeshServiceSubset > MeshGatewayRoute (the more a targetRef is focused the higher priority it has)
    • If levels are equal the lexicographic order of policy names is used

    For to and from policies we concatenate the array for each matching policies. We then build configuration by merging each level using JSON patch merge.

    For example if I have 2 default ordered this way:

    2.   sub:
    3.     array: [1, 2, 3]
    4.     other: 50
    5.     other-array: [3, 4, 5]
    6. ---
    7. default:
    8.   sub:
    9.     array: []
    10.     other: null
    11.     other-array: [5, 6]
    12.     extra: 2

    The merge result is:

    Applying a global default

    1. type: ExamplePolicy
    2. name: example
    3. mesh: default
    4. spec:
    5.   targetRef:
    6.     kind: Mesh
    7.   to:
    8.     - targetRef:
    9.         kind: Mesh
    10.       default:
    11.         key: value

    All traffic from any proxy (top level targetRef) going to any proxy (to targetRef) will have this policy applied with value key=value.

    Recommending to users

    1. type: ExamplePolicy
    2. name: example
    3. mesh: default
    4. spec:
    5.   targetRef:
    6.     kind: Mesh
    7.   to:
    8.     - targetRef:
    9.         kind: MeshService
    10.         name: my-service
    11.       default:
    12.         key: value

    All traffic from any proxy (top level targetRef) going to the service “my-service” (to targetRef) will have this policy applied with value key=value.

    This is useful when a service owner wants to suggest its clients as set of configuration.

    Configuring all proxies of a team

    1. type: ExamplePolicy
    2. name: example
    3. mesh: default
    4. spec:
    5.   targetRef:
    6.     kind: MeshSubset
    7.     tags:
    8.       team: "my-team"
    9.   from:
    10.     - targetRef:
    11.         kind: Mesh
    12.         key: value

    All traffic from any proxies (from targetRef) going to any proxy that has the tag team=my-team (top level targetRef) will have this policy applied with value key=value.

    This is a useful way to define coarse grain rules for example.

    Configuring all proxies in a zone

    This can be very useful when observability stores are different for each zone for example.