Manually creating IAM for GCP

    The Cloud Credential Operator (CCO) manages cloud provider credentials as Kubernetes custom resource definitions (CRDs). You can configure the CCO to suit the security requirements of your organization by setting different values for the credentialsMode parameter in the install-config.yaml file.

    If you prefer not to store an administrator-level credential secret in the cluster kube-system project, you can choose one of the following options when installing OKD:

    • Manage cloud credentials manually:

      You can set the credentialsMode parameter for the CCO to Manual to manage cloud credentials manually. Using manual mode allows each cluster component to have only the permissions it requires, without storing an administrator-level credential in the cluster. You can also use this mode if your environment does not have connectivity to the cloud provider public IAM endpoint. However, you must manually reconcile permissions with new release images for every upgrade. You must also manually supply credentials for every component that requests them.

    • Remove the administrator-level credential secret after installing OKD with mint mode:

      If you are using the CCO with the credentialsMode parameter set to Mint, you can remove or rotate the administrator-level credential after installing OKD. Mint mode is the default configuration for the CCO. This option requires the presence of the administrator-level credential during an installation. The administrator-level credential is used during the installation to mint other credentials with some permissions granted. The original credential secret is not stored in the cluster permanently.

    Additional resources

    To learn how to rotate or remove the administrator-level credential secret after installing OKD, see Rotating or removing cloud provider credentials.

    For a detailed description of all available CCO credential modes and their supported platforms, see .

    Manually create IAM

    The Cloud Credential Operator (CCO) can be put into manual mode prior to installation in environments where the cloud identity and access management (IAM) APIs are not reachable, or the administrator prefers not to store an administrator-level credential secret in the cluster kube-system namespace.

    Procedure

    1. Change to the directory that contains the installation program and create the install-config.yaml file:

    2. Edit the install-config.yaml configuration file so that it contains the credentialsMode parameter set to Manual.

      Example install-config.yaml configuration file

      1. apiVersion: v1
      2. baseDomain: cluster1.example.com
      3. credentialsMode: Manual (1)
      4. compute:
      5. - architecture: amd64
      6.   hyperthreading: Enabled
      7. ...

    3. From the directory that contains the installation program, obtain details of the OKD release image that your openshift-install binary is built to use:

      Example output

      1. release image quay.io/openshift-release-dev/ocp-release:4.y.z-x86_64

    4. Locate all CredentialsRequest objects in this release image that target the cloud you are deploying on:

      1. $ oc adm release extract quay.io/openshift-release-dev/ocp-release:4.y.z-x86_64 --credentials-requests --cloud=gcp

      This command creates a YAML file for each CredentialsRequest object.

    5. Create YAML files for secrets in the openshift-install manifests directory that you generated previously. The secrets must be stored using the namespace and secret name defined in the spec.secretRef for each CredentialsRequest object. The format for the secret data varies for each cloud provider.

    6. From the directory that contains the installation program, proceed with your cluster creation:

      1. $ openshift-install create cluster --dir=<installation_directory>

    Each cloud provider uses a credentials root secret in the kube-system namespace by convention, which is then used to satisfy all credentials requests and create their respective secrets. This is done either by minting new credentials, with mint mode, or by copying the credentials root secret, with passthrough mode.

    The format for the secret varies by cloud, and is also used for each CredentialsRequest secret.

    Google Cloud Platform (GCP) secret format

    1. apiVersion: v1
    2. kind: Secret
    3. metadata:
    4.   namespace: kube-system
    5.   name: gcp-credentials
    6. stringData:
    7.   service_account.json: <ServiceAccount>

    Upgrading clusters with manually maintained credentials

    The Cloud Credential Operator (CCO) Upgradable status for a cluster with manually maintained credentials is False by default.

    • For minor releases, for example, from 4.7 to 4.8, this status prevents you from upgrading until you have addressed any updated permissions and annotated the resource to indicate that the permissions are updated as needed for the next version. This annotation changes the Upgradable status to True.

    • For z-stream releases, for example, from 4.8.9 to 4.8.10, no permissions are added or changed, so the upgrade is not blocked.

    Before upgrading a cluster with manually maintained credentials, you must create any new credentials for the release image that you are upgrading to. Additionally, you must review the required permissions for existing credentials and accommodate any new permissions requirements in the new release for those components.

    Procedure

    1. Extract and examine the CredentialsRequest custom resource for the new release.

      The Manually creating IAM section of the installation content for your cloud provider explains how to obtain and use the credentials required for your cloud.

    2. When all of the secrets are correct for the new release, indicate that the cluster is ready to upgrade:

      1. Log in to the OKD CLI as a user with the cluster-admin role.

      2. Edit the CloudCredential resource to add an upgradeable-to annotation within the metadata field:

        Text to add

        1. ...
        2.   metadata:
        3.     annotations:
        4.       cloudcredential.openshift.io/upgradeable-to: <version_number>
        5. ...

        It may take several minutes after adding the annotation for the upgradeable status to change.

    3. Verify that the CCO is upgradeable:

      1. In the Administrator perspective of the web console, navigate to Administration → Cluster Settings.

      2. To view the CCO status details, click cloud-credential in the Cluster Operators list.

      3. If the Upgradeable status in the Conditions section is False, verify that the upgradeable-to annotation is free of typographical errors.

    When the Upgradeable status in the Conditions section is True, you can begin the OKD upgrade.

    Mint mode is the default and recommended Cloud Credential Operator (CCO) credentials mode for OKD. In this mode, the CCO uses the provided administrator-level cloud credential to run the cluster. Mint mode is supported for AWS, GCP, and Azure.

    In mint mode, the admin credential is stored in the kube-system namespace and then used by the CCO to process the CredentialsRequest objects in the cluster and create users for each with specific permissions.

    The benefits of mint mode include:

    • Each cluster component has only the permissions it requires

    • Automatic, on-going reconciliation for cloud credentials, including additional credentials or permissions that might be required for upgrades

    One drawback is that mint mode requires admin credential storage in a cluster secret.

    Mint mode with removal or rotation of the administrator-level credential

    Currently, this mode is only supported on AWS and GCP.

    In this mode, a user installs OKD with an administrator-level credential just like the normal mint mode. However, this process removes the administrator-level credential secret from the cluster post-installation.

    The administrator can have the Cloud Credential Operator make its own request for a read-only credential that allows it to verify if all CredentialsRequest objects have their required permissions, thus the administrator-level credential is not required unless something needs to be changed. After the associated credential is removed, it can be deleted or deactivated on the underlying cloud, if desired.

    The administrator-level credential is not stored in the cluster permanently.

    Following these steps still requires the administrator-level credential in the cluster for brief periods of time. It also requires manually re-instating the secret with administrator-level credentials for each upgrade.