Continuous Integration
Running kOps in a CI environment can also be useful for upgrading kOps. Simply download a newer version in the CI environment and run a new pipeline. This will allow any changes to be reviewed prior to being applied. This strategy can be extended to sequentially upgrade kOps on multiple clusters, allowing changes to be tested on non-production environments first.
If you have a solution for a different CI platform or deployment strategy, feel free to open a Pull Request!
- The GitLab runners that run the jobs need the appropriate permissions to invoke the kOps commands. For clusters in AWS this means providing AWS IAM credentials either with IAM User Keys set as secret variables in the project, or having the runner run on an EC2 instance with an instance profile attached.
- A cluster administrator makes a change to a cluster manifest, commits and pushes to a feature branch on GitLab and opens a Merge Request
- A reviewer reviews the change to confirm it is as intended, and approves or merges the MR
- A “master” pipeline is triggered from this merge commit which runs a
kops update cluster
. - Once the cluster is updated, is ran which can be used to confirm any nodes that need replacement. The administrator then starts the
roll
job which runskops rolling-update cluster --yes
and replaces any nodes as necessary.
- The
only:
field in each job will need to be updated to reflect the real project’s namespace and name. The two variables will also need to be set to real values. - The jobs that make actual changes to the clusters are manually invoked (
when: manual
) though this could easily be removed to make them automatic. - This pipeline setup will create and update existing clusters in place. It does not perform a “blue/green” deployment of multiple clusters.
In this case, it is possible to use to manage one YAML template and per-cluster values files with which to render the template. See the documentation for more information. kops toolbox template
would then be ran before kops replace
.
- This pipeline does not have a true “dryrun” job that can be ran on non-master branches, for example before a merge request is merged. This is because the required
kops replace
before thekops update cluster
will update the live assets in the state store which could impact newly launched nodes that download these assets. PR #6465 could add support for copying the state store to a local filesystem prior tokops replace
, allowing the dryrun pipeline to be completely isolated from the live state store.