Operator SDK Version upgrade guide

    In most cases the upgrading the SDK version should only entail updating the operator’s SDK dependency version in the Gopkg.toml or go.mod file. For some versions it might also be necessary to update the upstream Kubernetes and controller-runtime dependencies.

    The full list of dependencies and their versions required by a particular version of the SDK can be viewed by using the operator-sdk print-deps command. Use this command to update the project Gopkg.toml/go.mod file accordingly as you upgrade to a new SDK version.

    For some SDK versions after v0.1.0 there might be minor breaking changes in the controller-runtime APIs, operator-sdk CLI or the expected project layout and file names. These breaking changes will usually be outlined in the /release-notes for each release version.

    For releases that have a large number of breaking changes or involve a significant refactoring of the APIs and project layout there will be a migration guide similar to the .

    The following sections outline the upgrade steps for each SDK version along with steps necessary for any associated breaking changes.

    • Update the SDK constraint in Gopkg.toml to version v0.2.1 and run dep ensure to update the vendor directory.

    v0.3.x

    • Update the SDK constraint in Gopkg.toml to version v0.3.0, the kubernetes dependencies to kubernetes-1.12.3 revisions, and the controller-runtime version to v0.1.8. Then run dep ensure to update the vendor directory.

      1. [[override]]
      2.   name = "k8s.io/code-generator"
      3.   **revision for tag "kubernetes-1.12.3"**
      4.   revision = "3dcf91f64f638563e5106f21f50c31fa361c918d"
      6. [[override]]
      7.   name = "k8s.io/api"
      8.   **revision for tag "kubernetes-1.12.3"**
      9.   revision = "b503174bad5991eb66f18247f52e41c3258f6348"
      11. [[override]]
      12.   name = "k8s.io/apiextensions-apiserver"
      13.   **revision for tag "kubernetes-1.12.3"**
      14.   revision = "0cd23ebeb6882bd1cdc2cb15fc7b2d72e8a86a5b"
      16. [[override]]
      17.   name = "k8s.io/apimachinery"
      18.   **revision for tag "kubernetes-1.12.3"**
      19.   revision = "eddba98df674a16931d2d4ba75edc3a389bf633a"
      21. [[override]]
      22.   name = "k8s.io/client-go"
      23.   **revision for tag "kubernetes-1.12.3"**
      24.   revision = "d082d5923d3cc0bfbb066ee5fbdea3d0ca79acf8"
      26. [[override]]
      27.   name = "sigs.k8s.io/controller-runtime"
      28.   version = "=v0.1.8"
      30. [[constraint]]
      31.   name = "github.com/operator-framework/operator-sdk"
      32.   version = "=v0.3.0"

    v0.4.x

    • Update the SDK constraint in Gopkg.toml to version v0.4.1 and run dep ensure to update the vendor directory.

      1. [[constraint]]
      2.   name = "github.com/operator-framework/operator-sdk"
      3.   version = "=v0.4.1"

    v0.5.x

    • Update the SDK constraint in Gopkg.toml to version v0.5.0, the kubernetes dependencies to kubernetes-1.13.1 revisions, and the controller-runtime version to v0.1.10.

      1. [[override]]
      2.   name = "k8s.io/code-generator"
      3.   **revision for tag "kubernetes-1.13.1"**
      4.   revision = "c2090bec4d9b1fb25de3812f868accc2bc9ecbae"
      6. [[override]]
      7.   name = "k8s.io/api"
      8.   **revision for tag "kubernetes-1.13.1"**
      9.   revision = "05914d821849570fba9eacfb29466f2d8d3cd229"
      11. [[override]]
      12.   name = "k8s.io/apiextensions-apiserver"
      13.   **revision for tag "kubernetes-1.13.1"**
      14.   revision = "0fe22c71c47604641d9aa352c785b7912c200562"
      16. [[override]]
      17.   name = "k8s.io/apimachinery"
      18.   **revision for tag "kubernetes-1.13.1"**
      19.   revision = "2b1284ed4c93a43499e781493253e2ac5959c4fd"
      21. [[override]]
      22.   name = "k8s.io/client-go"
      23.   **revision for tag "kubernetes-1.13.1"**
      24.   revision = "8d9ed539ba3134352c586810e749e58df4e94e4f"
      26. [[override]]
      27.   name = "sigs.k8s.io/controller-runtime"
      28.   version = "=v0.1.10"
      30. [[constraint]]
      31.   name = "github.com/operator-framework/operator-sdk"
      32.   version = "=v0.5.0"

    • Append the following new constraints to your Gopkg.toml.

      1. [[override]]
      2.   name = "k8s.io/kube-openapi"
      3.   revision = "0cf8f7e6ed1d2e3d47d02e3b6e559369af24d803"
      5. [[override]]
      6.   name = "github.com/go-openapi/spec"
      7.   branch = "master"
      9. [[override]]
      10.   name = "sigs.k8s.io/controller-tools"
      11.   version = "=v0.1.8"

    • Update the required dependencies in Gopkg.toml to include sigs.k8s.io/controller-tools/pkg/crd/generator and change k8s.io/code-generator/cmd/openapi-gen to k8s.io/kube-openapi/cmd/openapi-gen. Then run dep ensure to update the vendor directory.

      1. required = [
      2.   "k8s.io/code-generator/cmd/defaulter-gen",
      3.   "k8s.io/code-generator/cmd/deepcopy-gen",
      4.   "k8s.io/code-generator/cmd/conversion-gen",
      5.   "k8s.io/code-generator/cmd/client-gen",
      6.   "k8s.io/code-generator/cmd/lister-gen",
      7.   "k8s.io/code-generator/cmd/informer-gen",
      8.   "k8s.io/kube-openapi/cmd/openapi-gen",
      9.   "k8s.io/gengo/args",
      10.   "sigs.k8s.io/controller-tools/pkg/crd/generator",
      11. ]

    v0.6.x

    • Update the SDK constraint in Gopkg.toml to version v0.6.0 and run dep ensure to update the vendor directory.

      1. [[constraint]]
      2.   name = "github.com/operator-framework/operator-sdk"
      3.   version = "=v0.6.0"

    • The operator-sdk olm-catalog command now expects and generates manifests in the operator-registry manifest format. Generate your CSVs in the new layout by using operator-sdk olm-catalog gen-csv command, or modify the layout of your existing CSV manifests directory similar to the example below.

      1. $ tree deploy/olm-catalog
      2. deploy/olm-catalog/
      3. └── memcached-operator
      4.     ├── 0.1.0
      5.     │   └── memcached-operator.v0.1.0.clusterserviceversion.yaml
      6.     ├── 0.2.0
      7.     │   └── memcached-operator.v0.2.0.clusterserviceversion.yaml
      8.     └── memcached-operator.package.yaml

    • Update the SDK constraint in Gopkg.toml to version v0.7.1 and run dep ensure to update the vendor directory.

      1. [[constraint]]
      2.   name = "github.com/operator-framework/operator-sdk"
      3.   version = "=v0.7.1"

    v0.8.x

    The SDK version v0.8.x supports scaffolding projects to use Go modules by default. It is recommended that you migrate your operator project to use modules for dependency management, however you can choose to keep using dep. The upgrade steps for both are outlined below:

    dep

    • Update the SDK constraint in Gopkg.toml to version v0.8.2.

      1. [[constraint]]
      2.   name = "github.com/operator-framework/operator-sdk"
      3.   version = "=v0.8.2"

    • Pin the controller-tools dependency to the following revision. See the release notes or #1278 for why this is needed.

      1. [[override]]
      2.   name = "sigs.k8s.io/controller-tools"
      3.   revision = "9d55346c2bde73fb3326ac22eac2e5210a730207"

    • Run dep ensure to update the vendor directory.

    modules

    To get familiar with Go modules read the . In particular the section on migrating to modules.

    • Ensure that you have Go 1.12+ and 3.9+ installed.
    • Activate Go modules support for your project in $GOPATH/src by setting the env GO111MODULE=on. See activating modules for more details.
    • Initialize a new go.mod file by running go mod init.
    • Append the following to the end of your go.mod file to pin the operator-sdk and other upstream dependencies to the required versions.
    1. // Pinned to kubernetes-1.13.1
    2. replace (
    3.     k8s.io/api => k8s.io/api v0.0.0-20181213150558-05914d821849
    4.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20181213153335-0fe22c71c476
    5.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20181127025237-2b1284ed4c93
    6.     k8s.io/client-go => k8s.io/client-go v0.0.0-20181213151034-8d9ed539ba31
    7. )
    9. replace (
    10.     github.com/coreos/prometheus-operator => github.com/coreos/prometheus-operator v0.29.0
    11.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20181117043124-c2090bec4d9b
    12.     k8s.io/kube-openapi => k8s.io/kube-openapi v0.0.0-20180711000925-0cf8f7e6ed1d
    13.     sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.1.10
    14.     sigs.k8s.io/controller-tools => sigs.k8s.io/controller-tools v0.1.11-0.20190411181648-9d55346c2bde
    15. )
    17. replace github.com/operator-framework/operator-sdk => github.com/operator-framework/operator-sdk v0.8.2
    • Run go mod tidy to clean up the go.mod file.
      • In case of any go module loading errors, consult the default scaffolded by the operator-sdk to resolve any differences. You can also view this file by scaffolding a new project with operator-sdk v0.8.2.
    • Ensure that you can build the project with operator-sdk build
    • Finally remove Gopkg.lock, Gopkg.toml and the vendor directory.

    Breaking changes

    Upon updating the project to v0.8.2 the following breaking changes apply:

    • On running the command operator-sdk generate openapi, the CRD manifests at deploy/crds/<group>_<version>_<kind>.crd for all API types will now be regenerated based on their source files pkg/apis/..._types.go. So if you have made any manual edits to the default generated CRD manifest, e.g manually written the validation block or specified the naming (spec.names), then that information be overwritten when the CRD is regenerated.

      The correct way to specify CRD fields like naming, validation, subresources etc is by using // +kubebuilder marker comments. Consult the legacy kubebuilder documentation to see what CRD fields can be generated via // +kubebuilder marker comments.

      Note: The version of controller-tools tied to this release does not support settting the spec.scope field of the CRD. Use the marker comment +genclient:nonNamespaced to set spec.scope=Cluster if necessary. See the example below:

      1. // MemcachedSpec defines the desired state of Memcached
      2. type MemcachedSpec struct {
      3.   // +kubebuilder:validation:Maximum=5
      4.   // +kubebuilder:validation:Minimum=1
      5.   Size int32 `json:"size"`
      6. }
      8. // MemcachedStatus defines the observed state of Memcached
      9. type MemcachedStatus struct {
      10.   // +kubebuilder:validation:MaxItems=5
      11.   // +kubebuilder:validation:MinItems=1
      12.   Nodes []string `json:"nodes"`
      13. }
      15. // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
      17. // Memcached is the Schema for the memcacheds API
      18. // +kubebuilder:subresource:status
      19. // +kubebuilder:resource:shortName="mc"
      20. // +genclient:nonNamespaced
      21. type Memcached struct {
      22.   metav1.TypeMeta   `json:",inline"`
      23.   metav1.ObjectMeta `json:"metadata,omitempty"`
      25.   Spec   MemcachedSpec   `json:"spec,omitempty"`
      26.   Status MemcachedStatus `json:"status,omitempty"`
      27. }

    v0.9.x

    • The function ExposeMetricsPort() has been replaced with CreateMetricsService() #1560.

      Replace the following line in cmd/manager/main.go

      1.   _, err = metrics.ExposeMetricsPort(ctx, metricsPort)

      with

      1.   servicePorts := []v1.ServicePort{
      2.   {Port: metricsPort, Name: metrics.OperatorPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: metricsPort}},
      3.   {Port: operatorMetricsPort, Name: metrics.CRPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: operatorMetricsPort}},
      4. }
      6. _, err = metrics.CreateMetricsService(ctx, servicePorts)

    dep

    • Update the SDK constraint in Gopkg.toml to version v0.9.0, the kubernetes dependencies to kubernetes-1.13.4 revisions, and the controller-runtime version to v0.1.12.

      1. [[override]]
      2.   name = "k8s.io/api"
      3.   **revision for tag "kubernetes-1.13.4"**
      4.   revision = "5cb15d34447165a97c76ed5a60e4e99c8a01ecfe"
      5. [[override]]
      6.   name = "k8s.io/apiextensions-apiserver"
      7.   **revision for tag "kubernetes-1.13.4"**
      8.   revision = "d002e88f6236312f0289d9d1deab106751718ff0"
      9. [[override]]
      10.   name = "k8s.io/apimachinery"
      11.   **revision for tag "kubernetes-1.13.4"**
      12.   revision = "86fb29eff6288413d76bd8506874fddd9fccdff0"
      13. [[override]]
      14.   name = "k8s.io/client-go"
      15.   **revision for tag "kubernetes-1.13.4"**
      16.   revision = "b40b2a5939e43f7ffe0028ad67586b7ce50bb675"
      17. [[override]]
      18.   name = "github.com/coreos/prometheus-operator"
      19.   version = "=v0.29.0"
      20. [[override]]
      21.   name = "sigs.k8s.io/controller-runtime"
      22.   version = "=v0.1.12"
      23. [[constraint]]
      24.   name = "github.com/operator-framework/operator-sdk"
      25.   version = "=v0.9.0"

    • Append the contraint for k8s.io/kube-state-metrics.

      1. [[override]]
      2.   name = "k8s.io/kube-state-metrics"
      3.   version = "v1.6.0"

    • Run dep ensure to update the vendor directory.

    modules

    • Update the replace directives in your go.mod file for the SDK, kubernetes, controller-runtime and kube-state metrics dependencies to the following versions.

      1. // Pinned to kubernetes-1.13.4
      2. replace (
      3.   k8s.io/api => k8s.io/api v0.0.0-20190222213804-5cb15d344471
      4.   k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20190228180357-d002e88f6236
      5.   k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20190221213512-86fb29eff628
      6.   k8s.io/client-go => k8s.io/client-go v0.0.0-20190228174230-b40b2a5939e4
      7. )
      8. replace (
      9.   github.com/coreos/prometheus-operator => github.com/coreos/prometheus-operator v0.29.0
      10.   sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.1.12
      11.   sigs.k8s.io/controller-tools => sigs.k8s.io/controller-tools v0.1.11-0.20190411181648-9d55346c2bde
      12.   k8s.io/kube-state-metrics => k8s.io/kube-state-metrics v1.6.0
      13. )
      14. replace github.com/operator-framework/operator-sdk => github.com/operator-framework/operator-sdk v0.9.0

    v0.10.x

    • The scorecard configuration format for the operator-sdk scorecard command has changed. See doc/test-framework/scorecard for more info.

    • The CSV config field role-path is now role-paths and takes a list of strings. Replace:

      1. role-path: path/to/role.yaml

      with:

      1. role-paths:
      2. - path/to/role.yaml

    modules

    • Ensure the the following replace directives are present in your go.mod file:

      1. replace (
      2.         github.com/coreos/prometheus-operator => github.com/coreos/prometheus-operator v0.29.0
      3.         // Pinned to v2.9.2 (kubernetes-1.13.1) so https://proxy.golang.org can
      4.         // resolve it correctly.
      5.         github.com/prometheus/prometheus => github.com/prometheus/prometheus v0.0.0-20190424153033-d3245f150225
      6.         k8s.io/kube-state-metrics => k8s.io/kube-state-metrics v1.6.0
      7.         sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.1.12
      8.         sigs.k8s.io/controller-tools => sigs.k8s.io/controller-tools v0.1.11-0.20190411181648-9d55346c2bde
      9. )
      11. replace github.com/operator-framework/operator-sdk => github.com/operator-framework/operator-sdk v0.10.0

    v0.11.x

    NOTE: this version uses Kubernetes v1.14.x and controller-runtime v0.2.x, both of which have breaking API changes. See the changelog for more details.

    dep

    • Remove the required = [ ... ] section and comment from the top of your Gopkg.toml file.

    • Update the following overrides in Gopkg.toml:

      1. [[override]]
      2.   name = "k8s.io/api"
      3.   **revision for tag "kubernetes-1.14.1"**
      4.   revision = "6e4e0e4f393bf5e8bbff570acd13217aa5a770cd"
      5. [[override]]
      6.   name = "k8s.io/apiextensions-apiserver"
      7.   **revision for tag "kubernetes-1.14.1"**
      8.   revision = "727a075fdec8319bf095330e344b3ccc668abc73"
      9. [[override]]
      10.   name = "k8s.io/apimachinery"
      11.   **revision for tag "kubernetes-1.14.1"**
      12.   revision = "6a84e37a896db9780c75367af8d2ed2bb944022e"
      13. [[override]]
      14.   name = "k8s.io/client-go"
      15.   **revision for tag "kubernetes-1.14.1"**
      16.   revision = "1a26190bd76a9017e289958b9fba936430aa3704"
      17. [[override]]
      18.   name = "github.com/coreos/prometheus-operator"
      20. [[override]]
      21.   name = "sigs.k8s.io/controller-runtime"
      22. [[constraint]]
      23.   name = "github.com/operator-framework/operator-sdk"
      24.   version = "=v0.11.0"

    • Append an override for gopkg.in/fsnotify.v1, which is required when resolving controller-runtime dependencies:

    • Remove the k8s.io/kube-state-metrics override.

    • Run dep ensure to update the vendor directory.

    modules

    • Ensure the the following replace directives are present in your go.mod file:

      1. // Pinned to kubernetes-1.14.1
      2. replace (
      3.     k8s.io/api => k8s.io/api kubernetes-1.14.1
      4.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver kubernetes-1.14.1
      5.     k8s.io/apimachinery => k8s.io/apimachinery kubernetes-1.14.1
      6.     k8s.io/client-go => k8s.io/client-go kubernetes-1.14.1
      7.     k8s.io/cloud-provider => k8s.io/cloud-provider kubernetes-1.14.1
      8. )
      10. replace (
      11.     // Indirect operator-sdk dependencies use git.apache.org, which is frequently
      12.     // down. The github mirror should be used instead.
      13.     // Locking to a specific version (from 'go mod graph'):
      14.     git.apache.org/thrift.git => github.com/apache/thrift v0.0.0-20180902110319-2566ecd5d999
      15.     github.com/coreos/prometheus-operator => github.com/coreos/prometheus-operator v0.31.1
      16.     // Pinned to v2.10.0 (kubernetes-1.14.1) so https://proxy.golang.org can
      17.     // resolve it correctly.
      18.     github.com/prometheus/prometheus => github.com/prometheus/prometheus d20e84d0fb64aff2f62a977adc8cfb656da4e286
      19. )
      21. replace github.com/operator-framework/operator-sdk => github.com/operator-framework/operator-sdk v0.11.0

    Import updates

    • Replace import sigs.k8s.io/controller-runtime/pkg/runtime/scheme with sigs.k8s.io/controller-runtime/pkg/scheme in:
      • ./pkg/apis/<group>/<version>/register.go
    • Replace import sigs.k8s.io/controller-runtime/pkg/runtime/log with sigs.k8s.io/controller-runtime/pkg/log in:
      • cmd/manager/main.go
      • ./pkg/controller/<kind>/<kind>_controller.go
    • Replace import sigs.k8s.io/controller-runtime/pkg/runtime/signals with sigs.k8s.io/controller-runtime/pkg/manager/signals in:
      • cmd/manager/main.go
    • Remove import sigs.k8s.io/controller-tools/pkg/crd/generator from:
      • tools.go

    controller-runtime API updates

    All method signatures for and sigs.k8s.io/controller-runtime/pkg/client.StatusWriter (except for Client.Get()) have been updated. Each now uses a variadic option interface parameter typed for each method.

    OpenAPI updates

    • Run the command operator-sdk generate openapi and ensure that no errors such as API rule violation are raised. For further information see the API rules documentation.

    NOTE: You may need to add or remove markers (code annotations) to fix issues found when running generate openapi. Usage of markers in API code is discussed in the kubebuilder CRD generation and in marker documentation. A full list of OpenAPIv3 validation markers can be found .

    TIPS:

    • If the +kubebuilder:validation:Pattern has commas, then surround the expressions in backticks.
    • If you are using +kubebuilder:validation:Enum then either surround the expression list in curly braces and quote each expression, or separate each expression using semicolons.

    Operator SDK updates

    • pkg/test.FrameworkClient List() and Delete() method invocations should be updated to match those of Client.List() and Client.Delete(), described above.
    • The annotation to assign a scope to your CRD has changed. For the following changes, note that <resource> is the plural lower-case CRD Kind found at spec.names.plural.
      • For Namespaced-scoped operators, add a +kubebuilder:resource:path=<resource>,scope=Namespaced comment above your kind type in pkg/apis/<group>/<version>/<kind>_types.go.
      • For Cluster-scoped operators, replace the +genclient:nonNamespaced comment above your kind type in pkg/apis/<group>/<version>/<kind>_types.go with +kubebuilder:resource:path=<resource>,scope=Cluster.
    • CRD file names now have the form <full group>_<resource>_crd.yaml, and CR file names now have the form <full group>_<version>_<kind>_cr.yaml. <full group> is the full group name of your CRD found at spec.group, and <resource> is the plural lower-case CRD Kind found at spec.names.plural. To migrate:
      • Run operator-sdk generate openapi. CRD manifest files with new names containing versioned validation and subresource blocks will be generated.
      • Delete the old CRD manifest files.
      • Rename CR manifest file names from <group>_<version>_<kind>_cr.yaml to <full group>_<version>_<kind>_cr.yaml.

    Go version

    • Ensure that you are using a go version 1.13+

    dep

    Using dep is no longer supported. Follow to learn how to migrate your project.

    modules

    • Ensure the the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/go-openapi/spec v0.19.0
    3.     github.com/operator-framework/operator-sdk v0.12.1-0.20191112211508-82fc57de5e5b
    4.     github.com/spf13/pflag v1.0.3
    5.     k8s.io/api v0.0.0
    6.     k8s.io/apimachinery v0.0.0
    7.     k8s.io/client-go v11.0.0+incompatible
    8.     k8s.io/kube-openapi v0.0.0-20190918143330-0270cf2f1c1d
    9.     sigs.k8s.io/controller-runtime v0.3.0
    10. )
    12. // Pinned to kubernetes-1.15.4
    13. replace (
    14.     k8s.io/api => k8s.io/api v0.0.0-20190918195907-bd6ac527cfd2
    15.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20190918201827-3de75813f604
    16.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20190817020851-f2f3a405f61d
    17.     k8s.io/apiserver => k8s.io/apiserver v0.0.0-20190918200908-1e17798da8c1
    18.     k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20190918202139-0b14c719ca62
    19.     k8s.io/client-go => k8s.io/client-go v0.0.0-20190918200256-06eb1244587a
    20.     k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20190918203125-ae665f80358a
    21.     k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20190918202959-c340507a5d48
    22.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20190612205613-18da4a14b22b
    23.     k8s.io/component-base => k8s.io/component-base v0.0.0-20190918200425-ed2f0867c778
    24.     k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190817025403-3ae76f584e79
    25.     k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20190918203248-97c07dcbb623
    26.     k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20190918201136-c3a845f1fbb2
    27.     k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20190918202837-c54ce30c680e
    28.     k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20190918202429-08c8357f8e2d
    29.     k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20190918202713-c34a54b3ec8e
    30.     k8s.io/kubelet => k8s.io/kubelet v0.0.0-20190918202550-958285cf3eef
    31.     k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20190918203421-225f0541b3ea
    32.     k8s.io/metrics => k8s.io/metrics v0.0.0-20190918202012-3c1ca76f5bda
    33.     k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20190918201353-5cc279503896
    34. )

    NOTE: Check here an example of this upgrade to see the changes from the version 0.11.0 to 0.12.0.

    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate openapi and ensure that no errors such as API rule violation are raised. For further information see the documentation.

    (Optional) Update your operator to print its version

    In v0.12.0, the SDK team updated the scaffold for cmd/manager/main.go to include the operator’s version in the output produced by the printVersion() function. See #1953

    To add this feature to your operator, add the following lines in <project>/cmd/manager/main.go:

    1. import (
    2.     ...
    3.     "<your_module_path>/version"
    4.     ...
    5. )
    7. func printVersion() {
    8.     log.Info(fmt.Sprintf("Operator Version: %s", version.Version))
    9.     ...
    10. }

    v0.13.x

    modules

    • Ensure the the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/operator-framework/operator-sdk v0.13.0
    3.     sigs.k8s.io/controller-runtime v0.4.0
    4. )
    6. // Pinned to kubernetes-1.16.2
    7. replace (
    8.     k8s.io/api => k8s.io/api v0.0.0-20191016110408-35e52d86657a
    9.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20191016113550-5357c4baaf65
    10.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
    11.     k8s.io/apiserver => k8s.io/apiserver v0.0.0-20191016112112-5190913f932d
    12.     k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20191016114015-74ad18325ed5
    13.     k8s.io/client-go => k8s.io/client-go v0.0.0-20191016111102-bec269661e48
    14.     k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20191016115326-20453efc2458
    15.     k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20191016115129-c07a134afb42
    16.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20191004115455-8e001e5d1894
    17.     k8s.io/component-base => k8s.io/component-base v0.0.0-20191016111319-039242c015a9
    18.     k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190828162817-608eb1dad4ac
    19.     k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20191016115521-756ffa5af0bd
    20.     k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20191016112429-9587704a8ad4
    21.     k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20191016114939-2b2b218dc1df
    22.     k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20191016114407-2e83b6f20229
    23.     k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20191016114748-65049c67a58b
    24.     k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
    25.     k8s.io/kubelet => k8s.io/kubelet v0.0.0-20191016114556-7841ed97f1b2
    26.     k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20191016115753-cf0698c3a16b
    27.     k8s.io/metrics => k8s.io/metrics v0.0.0-20191016113814-3b1a734dba6e
    28.     k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20191016112829-06bb3c9d77c9
    29. )
    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate crds to regenerate CRDs

    (Optional) Update the roles.yaml file

    Replace * per verbs in order to solve the issue 671 and make clear the permissions used.

    Example

    1.   verbs:
    2.   - create
    3.   - delete
    4.   - get
    5.   - list
    6.   - patch
    7.   - update
    8.   - watch

    Notable changes

    • Deprecated: Deprecated the operator-sdk generate openapi command. CRD generation is still supported with operator-sdk generate crds. It is now recommended to use directly for OpenAPI code generation. The generate openapi subcommand will be removed in a future release.
    • Breaking change: An existing CSV’s spec.customresourcedefinitions.owned is now always overwritten except for each name, version, and kind on invoking olm-catalog gen-csv when Go API code annotations are present.
    • Potentially Breaking change: Be aware that there are potentially other breaking changes due to the controller-runtime and Kubernetes version be upgraded from v0.4.0 to `v1.16.2, respectively. There may be breaking changes to Go client code due to both of those changes.

    For further detailed information see CHANGELOG

    v0.14.x

    modules

    • Ensure the the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/operator-framework/operator-sdk v0.14.1
    3.     sigs.k8s.io/controller-runtime v0.4.0
    4. )
    5. // Pinned to kubernetes-1.16.2
    6. replace (
    7.     k8s.io/api => k8s.io/api v0.0.0-20191016110408-35e52d86657a
    8.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20191016113550-5357c4baaf65
    9.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
    10.     k8s.io/apiserver => k8s.io/apiserver v0.0.0-20191016112112-5190913f932d
    11.     k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20191016114015-74ad18325ed5
    12.     k8s.io/client-go => k8s.io/client-go v0.0.0-20191016111102-bec269661e48
    13.     k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20191016115326-20453efc2458
    14.     k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20191016115129-c07a134afb42
    15.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20191004115455-8e001e5d1894
    16.     k8s.io/component-base => k8s.io/component-base v0.0.0-20191016111319-039242c015a9
    17.     k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190828162817-608eb1dad4ac
    18.     k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20191016115521-756ffa5af0bd
    19.     k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20191016112429-9587704a8ad4
    20.     k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20191016114939-2b2b218dc1df
    21.     k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20191016114407-2e83b6f20229
    22.     k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20191016114748-65049c67a58b
    23.     k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
    24.     k8s.io/kubelet => k8s.io/kubelet v0.0.0-20191016114556-7841ed97f1b2
    25.     k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20191016115753-cf0698c3a16b
    26.     k8s.io/metrics => k8s.io/metrics v0.0.0-20191016113814-3b1a734dba6e
    27.     k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20191016112829-06bb3c9d77c9
    28. )
    29. replace github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309 // Required by Helm
    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate crds to regenerate CRDs

    (Optional) Skip metrics logs when the operator is running locally

    There are changes to the default implementation of the metrics export. These changes require cmd/manager/main.go to be updated as follows.

    Update imports:

    1. import (
    2.     ...
    3.     "errors"
    4.     ...
    5. )

    Replace:

    1. func main() {
    2.     ...
    3.     if err = serveCRMetrics(cfg); err != nil {
    4.         log.Info("Could not generate and serve custom resource metrics", "error", err.Error())
    5.     }
    7.     // Add to the below struct any other metrics ports you want to expose.
    8.     servicePorts := []v1.ServicePort{
    9.         {Port: metricsPort, Name: metrics.OperatorPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: metricsPort}},
    10.         {Port: operatorMetricsPort, Name: metrics.CRPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: operatorMetricsPort}},
    11.     }
    12.     // Create Service object to expose the metrics port(s).
    13.     service, err := metrics.CreateMetricsService(ctx, cfg, servicePorts)
    14.     if err != nil {
    15.         log.Info("Could not create metrics Service", "error", err.Error())
    16.     }
    18.     // CreateServiceMonitors will automatically create the prometheus-operator ServiceMonitor resources
    19.     // necessary to configure Prometheus to scrape metrics from this operator.
    20.     services := []*v1.Service{service}
    21.     _, err = metrics.CreateServiceMonitors(cfg, namespace, services)
    22.     if err != nil {
    23.         log.Info("Could not create ServiceMonitor object", "error", err.Error())
    24.         // If this operator is deployed to a cluster without the prometheus-operator running, it will return
    25.         // ErrServiceMonitorNotPresent, which can be used to safely skip ServiceMonitor creation.
    26.         if err == metrics.ErrServiceMonitorNotPresent {
    27.             log.Info("Install prometheus-operator in your cluster to create ServiceMonitor objects", "error", err.Error())
    28.         }
    29.     }
    30.   ...
    31. }

    With:

    1. func main() {
    2.     ...
    3.     // Add the Metrics Service
    4.     addMetrics(ctx, cfg, namespace)
    5.   ...
    6. }

    And then, add implementation for addMetrics:

    1. // addMetrics will create the Services and Service Monitors to allow the operator export the metrics by using
    2. // the Prometheus operator
    3. func addMetrics(ctx context.Context, cfg *rest.Config, namespace string) {
    4.     if err := serveCRMetrics(cfg); err != nil {
    5.         if errors.Is(err, k8sutil.ErrRunLocal) {
    6.             log.Info("Skipping CR metrics server creation; not running in a cluster.")
    7.             return
    8.         }
    9.         log.Info("Could not generate and serve custom resource metrics", "error", err.Error())
    10.     }
    12.     // Add to the below struct any other metrics ports you want to expose.
    13.     servicePorts := []v1.ServicePort{
    14.         {Port: metricsPort, Name: metrics.OperatorPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: metricsPort}},
    15.         {Port: operatorMetricsPort, Name: metrics.CRPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: operatorMetricsPort}},
    16.     }
    18.     // Create Service object to expose the metrics port(s).
    19.     service, err := metrics.CreateMetricsService(ctx, cfg, servicePorts)
    20.     if err != nil {
    21.         log.Info("Could not create metrics Service", "error", err.Error())
    22.     }
    24.     // CreateServiceMonitors will automatically create the prometheus-operator ServiceMonitor resources
    25.     // necessary to configure Prometheus to scrape metrics from this operator.
    26.     services := []*v1.Service{service}
    27.     _, err = metrics.CreateServiceMonitors(cfg, namespace, services)
    28.     if err != nil {
    29.         log.Info("Could not create ServiceMonitor object", "error", err.Error())
    30.         // If this operator is deployed to a cluster without the prometheus-operator running, it will return
    31.         // ErrServiceMonitorNotPresent, which can be used to safely skip ServiceMonitor creation.
    32.         if err == metrics.ErrServiceMonitorNotPresent {
    33.             log.Info("Install prometheus-operator in your cluster to create ServiceMonitor objects", "error", err.Error())
    34.         }
    35.     }
    36. }

    NOTE: For more information check the PR which is responsible for the above changes #2190.

    Deprecations

    The github.com/operator-framework/operator-sdk/pkg/restmapper package was deprecated in favor of the DynamicRESTMapper implementation in . Users should migrate to controller-runtime’s implementation, which is a drop-in replacement.

    Replace:

    1. github.com/operator-framework/operator-sdk/pkg/restmapper.DynamicRESTMapper

    With:

    1. sigs.k8s.io/controller-runtime/pkg/client/apiutil.DynamicRESTMapper

    Breaking Changes

    Add operator_sdk.util Ansible collection

    The Ansible module k8s_status was extracted and is now provided by the operator_sdk.util Ansible collection. See developer_guide for new usage.

    To use the collection in a role, declare it at the root level in meta/main.yaml:

    1. collections:
    2. - operator_sdk.util

    To use it in a playbook, declare it in the play:

    1. - hosts: all
    2.   collections:
    3.    - operator_sdk.util
    4.   tasks:
    5.    - k8s_status:
    6.        api_version: app.example.com/v1
    7.        kind: Foo
    8.        name: "{{ meta.name }}"
    9.        namespace: "{{ meta.namespace }}"
    10.        status:
    11.          foo: bar

    You can also use the fully-qualified name without declaring the collection:

    1.    - operator_sdk.util.k8s_status:
    2.        api_version: app.example.com/v1
    3.        kind: Foo
    4.        name: "{{ meta.name }}"
    5.        namespace: "{{ meta.namespace }}"
    6.          foo: bar

    Notable Changes

    These notable changes contain just the most important user-facing changes. See the for details of the release.

    Ansible version update

    The Ansible version in the init projects was upgraded from to 2.9 for collections support. Update the meta/main.yaml file.

    Replace:

    1. ...
    2.  min_ansible_version: 2.6
    3. ...

    With:

    1. ...
    2.  min_ansible_version: 2.9
    3. ...

    Helm Upgrade to V3

    The Helm operator packages and base image were upgraded from Helm v2 to Helm v3. Note that cluster state for pre-existing CRs using Helm v2-based operators will be automatically migrated to Helm v3’s new release storage format, and existing releases may be upgraded due to changes in Helm v3’s label injection.

    If you are using any external helm v2 tooling with the your helm operator-managed releases, you will need to upgrade to the equivalent helm v3 tooling.

    v0.15.x

    modules

    • Ensure the the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/operator-framework/operator-sdk v0.15.2
    3.     sigs.k8s.io/controller-runtime v0.4.0
    4. )
    5. // Pinned to kubernetes-1.16.2
    6. replace (
    7.     k8s.io/api => k8s.io/api v0.0.0-20191016110408-35e52d86657a
    8.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20191016113550-5357c4baaf65
    9.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
    10.     k8s.io/apiserver => k8s.io/apiserver v0.0.0-20191016112112-5190913f932d
    11.     k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20191016114015-74ad18325ed5
    12.     k8s.io/client-go => k8s.io/client-go v0.0.0-20191016111102-bec269661e48
    13.     k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20191016115326-20453efc2458
    14.     k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20191016115129-c07a134afb42
    15.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20191004115455-8e001e5d1894
    16.     k8s.io/component-base => k8s.io/component-base v0.0.0-20191016111319-039242c015a9
    17.     k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190828162817-608eb1dad4ac
    18.     k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20191016115521-756ffa5af0bd
    19.     k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20191016112429-9587704a8ad4
    20.     k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20191016114939-2b2b218dc1df
    21.     k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20191016114407-2e83b6f20229
    22.     k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20191016114748-65049c67a58b
    23.     k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
    24.     k8s.io/kubelet => k8s.io/kubelet v0.0.0-20191016114556-7841ed97f1b2
    25.     k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20191016115753-cf0698c3a16b
    26.     k8s.io/metrics => k8s.io/metrics v0.0.0-20191016113814-3b1a734dba6e
    27.     k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20191016112829-06bb3c9d77c9
    28. )
    29. replace github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309 // Required by Helm
    30. replace github.com/openshift/api => github.com/openshift/api v0.0.0-20190924102528-32369d4db2ad // Required until https://github.com/operator-framework/operator-lifecycle-manager/pull/1241 is resolved
    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate crds to regenerate CRDs

    Breaking Changes on Commands

    This release contains breaking changes in some commands.

    • The operator-sdk olm-catalog gen-csv was replaced by operator-sdk generate csv
    • The operator-sdk up local is now operator-sdk run --local. However, all functionality of this command is retained.
    • And then, the operator-sdk alpha olm [sub-commands] [flags] was moved from alpha to its own sub-command. However, all functionality of this command is retained. To check run; operator-sdk olm --help.

    Breaking Changes for Helm and Ansible

    The operator-sdk run ansible/helm are now hidden commands in exec-entrypoint ansible/helm. However, all functionality of each sub-command is still the same. If you are using this feature then you will need to replace the run for exec-entrypoint as the following examples.

    Replace:

    1. oprator-sdk run ansible --watches-file=/opt/ansible/watches.yaml

    Replace:

    1. oprator-sdk run helm --watches-file=$HOME/watches.yaml

    With:

    1. oprator-sdk run exec-entrypoint helm --watches-file=$HOME/watches.yaml

    See the for details of the release.

    v0.16.x

    modules

    • Ensure that the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/operator-framework/operator-sdk v0.16.0
    3.     sigs.k8s.io/controller-runtime v0.4.0
    4. )
    6. // Pinned to kubernetes-1.16.2
    7. replace (
    8.     k8s.io/api => k8s.io/api v0.0.0-20191016110408-35e52d86657a
    9.     k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.0.0-20191016113550-5357c4baaf65
    10.     k8s.io/apimachinery => k8s.io/apimachinery v0.0.0-20191004115801-a2eda9f80ab8
    11.     k8s.io/apiserver => k8s.io/apiserver v0.0.0-20191016112112-5190913f932d
    12.     k8s.io/cli-runtime => k8s.io/cli-runtime v0.0.0-20191016114015-74ad18325ed5
    13.     k8s.io/client-go => k8s.io/client-go v0.0.0-20191016111102-bec269661e48
    14.     k8s.io/cloud-provider => k8s.io/cloud-provider v0.0.0-20191016115326-20453efc2458
    15.     k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.0.0-20191016115129-c07a134afb42
    16.     k8s.io/code-generator => k8s.io/code-generator v0.0.0-20191004115455-8e001e5d1894
    17.     k8s.io/component-base => k8s.io/component-base v0.0.0-20191016111319-039242c015a9
    18.     k8s.io/cri-api => k8s.io/cri-api v0.0.0-20190828162817-608eb1dad4ac
    19.     k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20191016115521-756ffa5af0bd
    20.     k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.0.0-20191016112429-9587704a8ad4
    21.     k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.0.0-20191016114939-2b2b218dc1df
    22.     k8s.io/kube-proxy => k8s.io/kube-proxy v0.0.0-20191016114407-2e83b6f20229
    23.     k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.0.0-20191016114748-65049c67a58b
    24.     k8s.io/kubectl => k8s.io/kubectl v0.0.0-20191016120415-2ed914427d51
    25.     k8s.io/kubelet => k8s.io/kubelet v0.0.0-20191016114556-7841ed97f1b2
    26.     k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.0.0-20191016115753-cf0698c3a16b
    27.     k8s.io/metrics => k8s.io/metrics v0.0.0-20191016113814-3b1a734dba6e
    28.     k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.0.0-20191016112829-06bb3c9d77c9
    29. )
    31. replace github.com/docker/docker => github.com/moby/moby v0.7.3-0.20190826074503-38ab9da00309 // Required by Helm
    32. replace github.com/openshift/api => github.com/openshift/api v0.0.0-20190924102528-32369d4db2ad // Required until https://github.com/operator-framework/operator-lifecycle-manager/pull/1241 is resolved
    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate crds to regenerate CRDs

    Bug Fixes and Improvements for Metrics

    There are changes to the default implementation of the metrics export. These changes require cmd/manager/main.go to be updated as follows.

    Replace:

    1. func main() {
    2.   ...
    3.   // Add the Metrics Service
    4.     addMetrics(ctx, cfg, namespace)
    5.   ...
    6. }

    With:

    1. func main() {
    2.   ...
    3.     // Add the Metrics Service
    4.     addMetrics(ctx, cfg)
    5.   ...
    6. }

    And then, update the default implementation of addMetrics and serveCRMetrics with:

    1. // addMetrics will create the Services and Service Monitors to allow the operator export the metrics by using
    2. // the Prometheus operator
    3. func addMetrics(ctx context.Context, cfg *rest.Config) {
    4.     // Get the namespace the operator is currently deployed in.
    5.     operatorNs, err := k8sutil.GetOperatorNamespace()
    6.     if err != nil {
    7.         if errors.Is(err, k8sutil.ErrRunLocal) {
    8.             log.Info("Skipping CR metrics server creation; not running in a cluster.")
    9.             return
    10.         }
    11.     }
    13.     if err := serveCRMetrics(cfg, operatorNs); err != nil {
    14.         log.Info("Could not generate and serve custom resource metrics", "error", err.Error())
    15.     }
    17.     // Add to the below struct any other metrics ports you want to expose.
    18.     servicePorts := []v1.ServicePort{
    19.         {Port: metricsPort, Name: metrics.OperatorPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: metricsPort}},
    20.         {Port: operatorMetricsPort, Name: metrics.CRPortName, Protocol: v1.ProtocolTCP, TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: operatorMetricsPort}},
    21.     }
    23.     // Create Service object to expose the metrics port(s).
    24.     service, err := metrics.CreateMetricsService(ctx, cfg, servicePorts)
    25.     if err != nil {
    26.         log.Info("Could not create metrics Service", "error", err.Error())
    27.     }
    29.     // CreateServiceMonitors will automatically create the prometheus-operator ServiceMonitor resources
    30.     // necessary to configure Prometheus to scrape metrics from this operator.
    31.     services := []*v1.Service{service}
    33.     // The ServiceMonitor is created in the same namespace where the operator is deployed
    34.     _, err = metrics.CreateServiceMonitors(cfg, operatorNs, services)
    35.     if err != nil {
    36.         log.Info("Could not create ServiceMonitor object", "error", err.Error())
    37.         // If this operator is deployed to a cluster without the prometheus-operator running, it will return
    38.         // ErrServiceMonitorNotPresent, which can be used to safely skip ServiceMonitor creation.
    39.         if err == metrics.ErrServiceMonitorNotPresent {
    40.             log.Info("Install prometheus-operator in your cluster to create ServiceMonitor objects", "error", err.Error())
    41.         }
    42.     }
    43. }
    45. // serveCRMetrics gets the Operator/CustomResource GVKs and generates metrics based on those types.
    46. // It serves those metrics on "http://metricsHost:operatorMetricsPort".
    47. func serveCRMetrics(cfg *rest.Config, operatorNs string) error {
    48.     // The function below returns a list of filtered operator/CR specific GVKs. For more control, override the GVK list below
    49.     // with your own custom logic. Note that if you are adding third party API schemas, probably you will need to
    50.     // customize this implementation to avoid permissions issues.
    51.     filteredGVK, err := k8sutil.GetGVKsFromAddToScheme(apis.AddToScheme)
    52.     if err != nil {
    53.         return err
    54.     }
    56.     // The metrics will be generated from the namespaces which are returned here.
    57.     // NOTE that passing nil or an empty list of namespaces in GenerateAndServeCRMetrics will result in an error.
    58.     ns, err := kubemetrics.GetNamespacesForMetrics(operatorNs)
    59.     if err != nil {
    60.         return err
    61.     }
    63.     // Generate and serve custom resource specific metrics.
    64.     err = kubemetrics.GenerateAndServeCRMetrics(cfg, ns, filteredGVK, metricsHost, operatorMetricsPort)
    65.     if err != nil {
    66.         return err
    67.     }
    68.     return nil
    69. }

    NOTE: For more information check the PRs which are responsible for the above changes ,#2603 and .

    (Optional) Support for watching multiple namespaces

    There are changes to add support for watching multiple namespaces. These changes require cmd/manager/main.go to be updated as follows.

    Update imports:

    1. import (
    2.     ...
    3.     "strings"
    5.     ...
    6.     "sigs.k8s.io/controller-runtime/pkg/cache"
    7.     ...
    8. )

    Replace:

    1. func main() {
    2.     ...
    3.     // Create a new Cmd to provide shared dependencies and start components
    4.     mgr, err := manager.New(cfg, manager.Options{
    5.         Namespace:          namespace,
    6.         MetricsBindAddress: fmt.Sprintf("%s:%d", metricsHost, metricsPort),
    7.     })
    8.     if err != nil {
    9.         log.Error(err, "")
    10.         os.Exit(1)
    11.     }
    12.   ...
    13. }

    With:

    1. func main() {
    2.     ...
    3.     // Set default manager options
    4.     options := manager.Options{
    5.         Namespace:          namespace,
    6.         MetricsBindAddress: fmt.Sprintf("%s:%d", metricsHost, metricsPort),
    7.     }
    9.     // Add support for MultiNamespace set in WATCH_NAMESPACE (e.g ns1,ns2)
    10.     // Note that this is not intended to be used for excluding namespaces, this is better done via a Predicate
    11.     // Also note that you may face performance issues when using this with a high number of namespaces.
    12.     // More Info: https://godoc.org/github.com/kubernetes-sigs/controller-runtime/pkg/cache#MultiNamespacedCacheBuilder
    13.     if strings.Contains(namespace, ",") {
    14.         options.Namespace = ""
    15.         options.NewCache = cache.MultiNamespacedCacheBuilder(strings.Split(namespace, ","))
    16.     }
    18.     // Create a new manager to provide shared dependencies and start components
    19.     mgr, err := manager.New(cfg, options)
    20.     if err != nil {
    21.         log.Error(err, "")
    22.         os.Exit(1)
    23.     }
    24.   ...
    25. }

    NOTE: For more information check the PR which is responsible for the above changes #2522.

    Breaking changes

    TestCtx in pkg/test has been deprecated

    The type name TestCtx in pkg/test has been deprecated and renamed to Context. Users of the e2e framework should do the following:

    • Replace TestCtx with Context
    • Replace NewTestCtx with NewContext

    Scorecard only supports YAML config files

    The scorecard feature now only supports YAML config files. Config files with other extensions are no longer supported and should be changed to the YAML format. For further information see

    Breaking Changes for Ansible

    Remove Ansible container sidecar

    The Ansible logs are now output in the operator container, so there is no longer a need for the Ansible container sidecar. To reflect this change, update the deploy/operator.yaml file as follows.

    Remove:

    1. - name: ansible
    2.   command:
    3.     - /usr/local/bin/ao-logs
    4.     - /tmp/ansible-operator/runner
    5.     - stdout
    6.   # Replace this with the built image name**
    7.   image: "REPLACE_IMAGE"
    8.   imagePullPolicy: "Always"
    9.   volumeMounts:
    10.     - mountPath: /tmp/ansible-operator/runner
    11.     name: runner
    12.     readOnly: true

    Replace:

    1. - name: operator

    With:

    1. - name: {{your operator name which is the value of metadata.name in this file}}

    By default the full Ansible logs will not be output, however, you can setup it via the ANSIBLE_DEBUG_LOGS environment variable in the deploy/operator.yaml file. See:

    1. ...
    2. - name: ANSIBLE_DEBUG_LOGS
    3.   value: "True"
    4. ...

    Migration to Ansible collections

    The core Ansible Kubernetes modules have been moved to the community.kubernetes Ansible collection. Future development of the modules will occur there, with only critical bugfixes going into the modules in core. Additionally, the operator_sdk.util collection is no longer installed by default in the base image. Instead, users should add a requirements.yml to their project root, with the following content:

    1. collections:
    2.   - community.kubernetes
    3.   - operator_sdk.util

    Users should then add the following stages to their build/Dockerfile:

    1. COPY requirements.yml ${HOME}/requirements.yml
    2. RUN ansible-galaxy collection install -r ${HOME}/requirements.yml \
    3.  && chmod -R ug+rwx ${HOME}/.ansible

    modules

    • Ensure that the following require modules and replace directives with the specific versions are present in your go.mod file:
    1. require (
    2.     github.com/operator-framework/operator-sdk v0.17.2
    3.     sigs.k8s.io/controller-runtime v0.5.2
    4. )
    6. replace (
    7.   k8s.io/client-go => k8s.io/client-go v0.17.4 // Required by prometheus-operator
    8.   github.com/Azure/go-autorest => github.com/Azure/go-autorest v13.3.2+incompatible // Required by OLM
    9. )
    • Run go mod tidy to update the project modules
    • Run the command operator-sdk generate k8s to ensure that your resources will be updated
    • Run the command operator-sdk generate crds to regenerate CRDs

    Breaking Changes

    OpenAPI generation

    • The deprecated operator-sdk generate openapi command has been removed. This command generated CRDs and zz_generated.openapi files for operator APIs.

    To generate CRDs, use operator-sdk generate crds.

    To generate Go OpenAPI code, use openapi-gen directly. For example:

    1. # Build the latest openapi-gen from source
    2. which ./bin/openapi-gen > /dev/null || go build -o ./bin/openapi-gen k8s.io/kube-openapi/cmd/openapi-gen
    4. # Run openapi-gen for each of your API group/version packages
    5. ./bin/openapi-gen --logtostderr=true \
    6.                   -i ./pkg/apis/<group>/<version> \
    7.                   -o "" \
    8.                   -O zz_generated.openapi \
    9.                   -p ./pkg/apis/<group>/<version> \
    10.                   -h ./hack/boilerplate.go.txt \
    11.                   -r "-"

    Molecule Upgrade for Ansible based-operators

    The Molecule version for Ansible based-operators was upgraded from 2.22 to 3.0.2. The following changes are required in the default scaffold files.

    • Remove the scenario.name from molecule.yaml and then, ensure that any condition with will look for the folder name which determines the scenario name from now on
    • Replace the lint with newer syntax from . See:

    Replace:

    1. lint:
    2.   name: yamllint
    3.   options:
    4.     config-data:
    5.       line-length:
    6.         max: 120

    With:

    1. lint: |
    2.   set -e
    3.   yamllint -d "{extends: relaxed, rules: {line-length: {max: 120}}}" .

    Replace:

    1. lint:
    2.   name: ansible-lint

    With:

    1. lint: |
    2.   set -e
    3.   ansible-lint
    • Rename molecule/$SCENARIO/playbook.yml to molecule/$SCENARIO/converge.yml to avoid a deprecation message.
    • Update the .travis.yml file to install the supported lints as follows.

    Replace:

    With:

    1. install:
    2.   - pip3 install docker molecule ansible-lint yamllint flake8 openshift jmespath

    NOTE To know more about how to upgrade your project to use the V3 Molecule version see here.

    Deprecations

    Test Framework

    • The methods ctx.GetOperatorNamespace() and ctx.GetWatchNamespace() were added to pkg/test in order to replace ctx.GetNamespace() which is deprecated. In this way, replace the use of ctx.GetNamespace() in your project with ctx.GetOperatorNamespace().

    • The --namespace flag from operator-sdk run --local, operator-sdk test --local, and operator-sdk cleanup was deprecated and is replaced by --watch-namespace and --operator-namespace.

      The --operator-namespace flag can be used to set the namespace where the operator will be deployed. It will set the environment variable OPERATOR_NAMESPACE. If this value is not set, then it will be the namespace defined as in your current kubeconfig context.

      The --watch-namespace flag can be used to set the namespace(s) which the operator will watch for changes. It will set the environment variable WATCH_NAMESPACE. Use an explicit empty string to watch all namespaces or a comma-separated list of namespaces (e.g. “ns1,ns2”) to watch multiple namespace when the operator is cluster-scoped. If using a list, then it should contain the namespace where the operator is deployed since the default metrics implementation will manage resources in the Operator’s namespace. By default, --watch-namespace will be set to the operator namespace.