Declarative Management of Kubernetes Objects Using Kustomize

    Since 1.14, Kubectl also supports the management of Kubernetes objects using a kustomization file. To view Resources found in a directory containing a kustomization file, run the following command:

    To apply those Resources, run with --kustomize or -k flag:

    1. kubectl apply -k <kustomization_directory>

    Install kubectl.

    You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using , or you can use one of these Kubernetes playgrounds:

    To check the version, enter kubectl version.

    Overview of Kustomize

    Kustomize is a tool for customizing Kubernetes configurations. It has the following features to manage application configuration files:

    • generating resources from other sources
    • setting cross-cutting fields for resources
    • composing and customizing collections of resources

    ConfigMap and Secret hold config or sensitive data that are used by other Kubernetes objects, such as Pods. The source of truth of ConfigMap or Secret are usually from somewhere else, such as a .properties file or a ssh key file. Kustomize has secretGenerator and configMapGenerator, which generate Secret and ConfigMap from files or literals.

    configMapGenerator

    To generate a ConfigMap from a file, add an entry to files list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a file content.

    1. # Create a application.properties file
    2. cat <<EOF >application.properties
    3. FOO=Bar
    4. EOF
    6. cat <<EOF >./kustomization.yaml
    7. configMapGenerator:
    8. - name: example-configmap-1
    9.   files:
    10.   - application.properties
    11. EOF

    The generated ConfigMap can be checked by the following command:

    1. kubectl kustomize ./

    The generated ConfigMap is:

    1. apiVersion: v1
    2. data:
    3.   application.properties: |
    4.     FOO=Bar
    5. kind: ConfigMap
    6. metadata:
    7.   name: example-configmap-1-8mbdf7882g

    ConfigMap can also be generated from literal key-value pairs. To generate a ConfigMap from a literal key-value pair, add an entry to literals list in configMapGenerator. Here is an example of generating a ConfigMap with a data item from a key-value pair.

    1. cat <<EOF >./kustomization.yaml
    2. configMapGenerator:
    3. - name: example-configmap-2
    4.   literals:
    5.   - FOO=Bar
    6. EOF

    The generated ConfigMap can be checked by the following command:

    1. kubectl kustomize ./

    The generated ConfigMap is

    1. apiVersion: v1
    2. data:
    3.   FOO: Bar
    4. kind: ConfigMap
    5. metadata:
    6.   name: example-configmap-2-g2hdhfc6tk

    secretGenerator

    1. # Create a password.txt file
    2. cat <<EOF >./password.txt
    3. username=admin
    4. password=secret
    5. EOF
    7. cat <<EOF >./kustomization.yaml
    8. secretGenerator:
    9. - name: example-secret-1
    10.   files:
    11.   - password.txt
    12. EOF

    The generated Secret is as follows:

    1. apiVersion: v1
    2. data:
    3.   password.txt: dXNlcm5hbWU9YWRtaW4KcGFzc3dvcmQ9c2VjcmV0Cg==
    4. kind: Secret
    5. metadata:
    6.   name: example-secret-1-t2kt65hgtb
    7. type: Opaque

    To generate a Secret from a literal key-value pair, add an entry to literals list in secretGenerator. Here is an example of generating a Secret with a data item from a key-value pair.

    1. cat <<EOF >./kustomization.yaml
    2. secretGenerator:
    3. - name: example-secret-2
    4.   literals:
    5.   - username=admin
    6.   - password=secret
    7. EOF

    The generated Secret is as follows:

    generatorOptions

    The generated ConfigMaps and Secrets have a suffix appended by hashing the contents. This ensures that a new ConfigMap or Secret is generated when the content is changed. To disable the behavior of appending a suffix, one can use generatorOptions. Besides that, it is also possible to specify cross-cutting options for generated ConfigMaps and Secrets.

    1. cat <<EOF >./kustomization.yaml
    2. configMapGenerator:
    3. - name: example-configmap-3
    4.   literals:
    5.   - FOO=Bar
    6. generatorOptions:
    7.   disableNameSuffixHash: true
    8.   labels:
    9.     type: generated
    10.   annotations:
    11.     note: generated
    12. EOF

    Runkubectl kustomize ./ to view the generated ConfigMap:

    1. apiVersion: v1
    2. data:
    3.   FOO: Bar
    4. kind: ConfigMap
    5. metadata:
    6.   annotations:
    7.     note: generated
    8.   labels:
    9.     type: generated
    10.   name: example-configmap-3

    It is quite common to set cross-cutting fields for all Kubernetes resources in a project. Some use cases for setting cross-cutting fields:

    • setting the same namespace for all Resource
    • adding the same name prefix or suffix
    • adding the same set of labels
    • adding the same set of annotations

    Here is an example:

    1. # Create a deployment.yaml
    2. cat <<EOF >./deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: nginx-deployment
    7.   labels:
    8.     app: nginx
    9. spec:
    10.   selector:
    11.     matchLabels:
    12.       app: nginx
    13.   template:
    14.     metadata:
    15.       labels:
    16.         app: nginx
    17.     spec:
    18.       containers:
    19.       - name: nginx
    20.         image: nginx
    21. EOF
    23. cat <<EOF >./kustomization.yaml
    24. namespace: my-namespace
    25. namePrefix: dev-
    26. nameSuffix: "-001"
    27. commonLabels:
    28.   app: bingo
    29. commonAnnotations:
    30.   oncallPager: 800-555-1212
    31. resources:
    32. - deployment.yaml
    33. EOF

    Run kubectl kustomize ./ to view those fields are all set in the Deployment Resource:

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   annotations:
    5.     oncallPager: 800-555-1212
    6.   labels:
    7.     app: bingo
    8.   name: dev-nginx-deployment-001
    9.   namespace: my-namespace
    10. spec:
    11.   selector:
    12.     matchLabels:
    13.       app: bingo
    14.   template:
    15.     metadata:
    16.       annotations:
    17.         oncallPager: 800-555-1212
    18.       labels:
    19.         app: bingo
    20.     spec:
    21.       containers:
    22.       - image: nginx
    23.         name: nginx

    It is common to compose a set of Resources in a project and manage them inside the same file or directory. Kustomize offers composing Resources from different files and applying patches or other customization to them.

    Composing

    Kustomize supports composition of different resources. The resources field, in the kustomization.yaml file, defines the list of resources to include in a configuration. Set the path to a resource’s configuration file in the resources list. Here is an example for an nginx application with a Deployment and a Service.

    1. # Create a deployment.yaml file
    2. cat <<EOF > deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: my-nginx
    7. spec:
    8.   selector:
    9.     matchLabels:
    10.       run: my-nginx
    11.   replicas: 2
    12.   template:
    13.     metadata:
    14.       labels:
    15.         run: my-nginx
    16.     spec:
    17.       containers:
    18.       - name: my-nginx
    19.         image: nginx
    20.         ports:
    21.         - containerPort: 80
    23. # Create a service.yaml file
    24. cat <<EOF > service.yaml
    25. apiVersion: v1
    26. kind: Service
    27. metadata:
    28.   name: my-nginx
    29.   labels:
    30.     run: my-nginx
    31. spec:
    32.   ports:
    34.     protocol: TCP
    35.   selector:
    36.     run: my-nginx
    37. EOF
    39. # Create a kustomization.yaml composing them
    40. cat <<EOF >./kustomization.yaml
    41. resources:
    42. - deployment.yaml
    43. - service.yaml
    44. EOF

    The Resources from kubectl kustomize ./ contains both the Deployment and the Service objects.

    Customizing

    On top of Resources, one can apply different customizations by applying patches. Kustomize supports different patching mechanisms through patchesStrategicMerge and patchesJson6902. patchesStrategicMerge is a list of file paths. Each file should be resolved to a strategic merge patch. The names inside the patches must match Resource names that are already loaded. Small patches that do one thing are recommended. For example, create one patch for increasing the deployment replica number and another patch for setting the memory limit.

    1. # Create a deployment.yaml file
    2. cat <<EOF > deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: my-nginx
    7. spec:
    8.   selector:
    9.     matchLabels:
    10.       run: my-nginx
    11.   replicas: 2
    12.   template:
    13.     metadata:
    14.       labels:
    15.         run: my-nginx
    16.     spec:
    17.       containers:
    18.       - name: my-nginx
    19.         image: nginx
    20.         ports:
    21.         - containerPort: 80
    22. EOF
    24. # Create a patch increase_replicas.yaml
    25. cat <<EOF > increase_replicas.yaml
    26. apiVersion: apps/v1
    27. kind: Deployment
    28. metadata:
    29.   name: my-nginx
    30. spec:
    31.   replicas: 3
    32. EOF
    34. # Create another patch set_memory.yaml
    35. cat <<EOF > set_memory.yaml
    36. apiVersion: apps/v1
    37. kind: Deployment
    38. metadata:
    39.   name: my-nginx
    40. spec:
    41.   template:
    42.     spec:
    43.       containers:
    44.       - name: my-nginx
    45.         resources:
    46.         limits:
    47.           memory: 512Mi
    48. EOF
    50. cat <<EOF >./kustomization.yaml
    51. resources:
    52. - deployment.yaml
    53. patchesStrategicMerge:
    54. - increase_replicas.yaml
    55. - set_memory.yaml
    56. EOF

    Run kubectl kustomize ./ to view the Deployment:

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   name: my-nginx
    5. spec:
    6.   replicas: 3
    7.   selector:
    8.     matchLabels:
    9.       run: my-nginx
    10.   template:
    11.     metadata:
    12.       labels:
    13.         run: my-nginx
    14.     spec:
    15.       containers:
    16.       - image: nginx
    17.         limits:
    18.           memory: 512Mi
    19.         name: my-nginx
    20.         ports:
    21.         - containerPort: 80
    1. # Create a deployment.yaml file
    2. cat <<EOF > deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: my-nginx
    7. spec:
    8.   selector:
    9.     matchLabels:
    10.       run: my-nginx
    11.   replicas: 2
    12.   template:
    13.     metadata:
    14.       labels:
    15.         run: my-nginx
    16.     spec:
    17.       containers:
    18.       - name: my-nginx
    19.         image: nginx
    20.         ports:
    21.         - containerPort: 80
    22. EOF
    24. # Create a json patch
    25. cat <<EOF > patch.yaml
    26. - op: replace
    27.   path: /spec/replicas
    28.   value: 3
    29. EOF
    31. # Create a kustomization.yaml
    32. cat <<EOF >./kustomization.yaml
    33. resources:
    34. - deployment.yaml
    36. patchesJson6902:
    37. - target:
    38.     group: apps
    39.     version: v1
    40.     kind: Deployment
    41.     name: my-nginx
    42.   path: patch.yaml
    43. EOF

    Run kubectl kustomize ./ to see the replicas field is updated:

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   name: my-nginx
    5. spec:
    6.   replicas: 3
    7.   selector:
    8.     matchLabels:
    9.       run: my-nginx
    10.   template:
    11.     metadata:
    12.       labels:
    13.         run: my-nginx
    14.     spec:
    15.       containers:
    16.       - image: nginx
    17.         name: my-nginx
    18.         ports:
    19.         - containerPort: 80

    In addition to patches, Kustomize also offers customizing container images or injecting field values from other objects into containers without creating patches. For example, you can change the image used inside containers by specifying the new image in images field in kustomization.yaml.

    1. cat <<EOF > deployment.yaml
    2. apiVersion: apps/v1
    3. kind: Deployment
    4. metadata:
    5.   name: my-nginx
    6. spec:
    7.   selector:
    8.     matchLabels:
    9.       run: my-nginx
    10.   replicas: 2
    11.   template:
    12.     metadata:
    13.       labels:
    14.         run: my-nginx
    15.     spec:
    16.       containers:
    17.       - name: my-nginx
    18.         image: nginx
    19.         ports:
    20.         - containerPort: 80
    22. cat <<EOF >./kustomization.yaml
    23. resources:
    24. - deployment.yaml
    25. images:
    26. - name: nginx
    27.   newName: my.image.registry/nginx
    29. EOF

    Run kubectl kustomize ./ to see that the image being used is updated:

    Sometimes, the application running in a Pod may need to use configuration values from other objects. For example, a Pod from a Deployment object need to read the corresponding Service name from Env or as a command argument. Since the Service name may change as namePrefix or nameSuffix is added in the kustomization.yaml file. It is not recommended to hard code the Service name in the command argument. For this usage, Kustomize can inject the Service name into containers through vars.

    1. # Create a deployment.yaml file
    2. cat <<EOF > deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: my-nginx
    7. spec:
    8.   selector:
    9.     matchLabels:
    10.       run: my-nginx
    11.   replicas: 2
    12.   template:
    13.     metadata:
    14.       labels:
    15.         run: my-nginx
    16.     spec:
    17.       containers:
    18.       - name: my-nginx
    19.         image: nginx
    20.         command: ["start", "--host", "\$(MY_SERVICE_NAME)"]
    21. EOF
    23. # Create a service.yaml file
    24. cat <<EOF > service.yaml
    25. apiVersion: v1
    26. kind: Service
    27. metadata:
    28.   name: my-nginx
    29.   labels:
    30.     run: my-nginx
    31. spec:
    32.   ports:
    33.   - port: 80
    34.     protocol: TCP
    35.   selector:
    36.     run: my-nginx
    37. EOF
    39. cat <<EOF >./kustomization.yaml
    40. namePrefix: dev-
    41. nameSuffix: "-001"
    43. resources:
    44. - deployment.yaml
    45. - service.yaml
    47. vars:
    48. - name: MY_SERVICE_NAME
    49.   objref:
    50.     kind: Service
    51.     name: my-nginx
    52.     apiVersion: v1
    53. EOF

    Run kubectl kustomize ./ to see that the Service name injected into containers is dev-my-nginx-001:

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   name: dev-my-nginx-001
    5. spec:
    6.   replicas: 2
    7.   selector:
    8.     matchLabels:
    9.       run: my-nginx
    10.   template:
    11.     metadata:
    12.       labels:
    13.         run: my-nginx
    14.     spec:
    15.       containers:
    16.       - command:
    17.         - start
    18.         - --host
    19.         - dev-my-nginx-001
    20.         image: nginx
    21.         name: my-nginx

    Kustomize has the concepts of bases and overlays. A base is a directory with a kustomization.yaml, which contains a set of resources and associated customization. A base could be either a local directory or a directory from a remote repo, as long as a kustomization.yaml is present inside. An overlay is a directory with a kustomization.yaml that refers to other kustomization directories as its bases. A base has no knowledge of an overlay and can be used in multiple overlays. An overlay may have multiple bases and it composes all resources from bases and may also have customization on top of them.

    Here is an example of a base:

    1. # Create a directory to hold the base
    2. mkdir base
    3. # Create a base/deployment.yaml
    4. cat <<EOF > base/deployment.yaml
    5. apiVersion: apps/v1
    6. kind: Deployment
    7. metadata:
    8.   name: my-nginx
    9. spec:
    10.   selector:
    11.     matchLabels:
    12.       run: my-nginx
    13.   replicas: 2
    14.   template:
    15.     metadata:
    16.       labels:
    17.         run: my-nginx
    18.     spec:
    19.       containers:
    20.       - name: my-nginx
    21.         image: nginx
    22. EOF
    24. # Create a base/service.yaml file
    25. cat <<EOF > base/service.yaml
    26. apiVersion: v1
    27. kind: Service
    28. metadata:
    29.   name: my-nginx
    30.   labels:
    31.     run: my-nginx
    32. spec:
    33.   ports:
    34.   - port: 80
    35.     protocol: TCP
    36.   selector:
    37.     run: my-nginx
    38. EOF
    39. # Create a base/kustomization.yaml
    40. cat <<EOF > base/kustomization.yaml
    41. resources:
    42. - deployment.yaml
    43. - service.yaml
    44. EOF

    This base can be used in multiple overlays. You can add different namePrefix or other cross-cutting fields in different overlays. Here are two overlays using the same base.

    1. mkdir dev
    2. cat <<EOF > dev/kustomization.yaml
    3. bases:
    4. - ../base
    5. namePrefix: dev-
    6. EOF
    8. mkdir prod
    9. cat <<EOF > prod/kustomization.yaml
    10. bases:
    11. - ../base
    12. namePrefix: prod-
    13. EOF

    How to apply/view/delete objects using Kustomize

    Use --kustomize or -k in kubectl commands to recognize Resources managed by kustomization.yaml. Note that -k should point to a kustomization directory, such as

    1. kubectl apply -k <kustomization directory>/

    Given the following kustomization.yaml,

    1. # Create a deployment.yaml file
    2. cat <<EOF > deployment.yaml
    3. apiVersion: apps/v1
    4. kind: Deployment
    5. metadata:
    6.   name: my-nginx
    7. spec:
    8.   selector:
    9.     matchLabels:
    10.       run: my-nginx
    11.   replicas: 2
    12.   template:
    13.     metadata:
    14.       labels:
    15.         run: my-nginx
    16.     spec:
    17.       containers:
    18.       - name: my-nginx
    19.         image: nginx
    20.         ports:
    21.         - containerPort: 80
    22. EOF
    24. # Create a kustomization.yaml
    25. cat <<EOF >./kustomization.yaml
    26. namePrefix: dev-
    27. commonLabels:
    28.   app: my-nginx
    29. resources:
    30. - deployment.yaml
    31. EOF

    Run the following command to apply the Deployment object dev-my-nginx:

    1. > kubectl apply -k ./
    2. deployment.apps/dev-my-nginx created

    Run one of the following commands to view the Deployment object dev-my-nginx:

    1. kubectl get -k ./
    1. kubectl describe -k ./

    Run the following command to delete the Deployment object dev-my-nginx:

    1. > kubectl delete -k ./

    What’s next

    Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on . Open an issue in the GitHub repo if you want to report a problem or .