Using manual mode with STS

In manual mode with STS, the individual OKD cluster components use AWS Secure Token Service (STS) to assign components IAM roles that provide short-term, limited-privilege security credentials. These credentials are associated with IAM roles that are specific to each component that makes AWS API calls.

Requests for new and refreshed credentials are automated by using an appropriately configured AWS IAM OpenID Connect (OIDC) identity provider, combined with AWS IAM roles. OKD signs service account tokens that are trusted by AWS IAM, and can be projected into a pod and used for authentication. Tokens are refreshed after one hour.

Figure 1. STS authentication flow

Using manual mode with STS changes the content of the AWS credentials that are provided to individual OKD components.

AWS secret format using long-lived credentials

AWS secret format with STS


kind: Secret
metadata:
  namespace: <target-namespace> (1)
  name: <target-secret-name> (2)
data:
  role_name: <operator-role-name> (3)
  web_identity_token_file: <path-to-token> (4)

1	The namespace for the component.
2	The name of the component secret.
3	The IAM role for the component.
4	The path to the service account token inside the pod. By convention, this is `/var/run/secrets/openshift/serviceaccount/token` for OKD components.

To install a cluster that is configured to use the Cloud Credential Operator (CCO) in manual mode with STS:

Configure the Cloud Credential Operator utility.
Create the required AWS resources , or with a single command.
.
Verify that the cluster is using short-lived credentials.

To create and manage cloud credentials from outside of the cluster when the Cloud Credential Operator (CCO) is operating in manual mode with STS, extract and prepare the CCO utility (ccoctl) binary.

The ccoctl is a Linux binary that must run in a Linux environment.

Prerequisites

Obtain the OKD release image.

Procedure

Get the CCO container image from the OKD release image:

$ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE)

Extract the ccoctl binary from the CCO container image within the OKD release image:
```
$ oc image extract $CCO_IMAGE --file="/usr/bin/ccoctl" -a ~/.pull-secret
```
Change the permissions to make ccoctl executable:
```
$ chmod 775 ccoctl
```

Verification

To verify that ccoctl is ready to use, display the help file:
```
$ ccoctl aws --help
```
Output of ccoctl aws --help:

``` Creating/updating/deleting cloud credentials objects for AWS cloud

Usage:
  ccoctl aws [command]
  create-all               Create all the required credentials objects
  create-iam-roles         Create IAM roles
  create-identity-provider Create IAM identity provider
  create-key-pair          Create a key pair
  delete                   Delete credentials objects
Flags:
  -h, --help   help for aws
Use "ccoctl aws [command] --help" for more information about a command.
```

You can use the CCO utility (ccoctl) to create the required AWS resources , or with a single command.

Creating AWS resources individually

Otherwise, you can use the command to create the AWS resources automatically.

Prerequisites

Extract and prepare the ccoctl binary.

Procedure

Generate the public and private RSA key files that are used to set up the OpenID Connect provider for the cluster:
```
$ ccoctl aws create-key-pair
```
Example output:
```
2021/04/13 11:01:02 Generating RSA keypair
2021/04/13 11:01:03 Writing private key to /<path_to_ccoctl_output_dir>/serviceaccount-signer.private
2021/04/13 11:01:03 Writing public key to /<path_to_ccoctl_output_dir>/serviceaccount-signer.public
2021/04/13 11:01:03 Copying signing key for use by installer
```
where serviceaccount-signer.private and serviceaccount-signer.public are the generated key files.

This command also creates a private key that the cluster requires during installation in /*<path_to_ccoctl_output_dir>*/tls/bound-service-account-signing-key.key.
Create an OpenID Connect identity provider and S3 bucket on AWS:

where:
- *<name>* is the name used to tag any cloud resources that are created for tracking.
- *<aws-region>* is the AWS region in which cloud resources will be created.
- *<path_to_ccoctl_output_dir>* is the path to the public key file that the ccoctl aws create-key-pair command generated.
  
  Example output:
```
2021/04/13 11:16:09 Bucket <name>-oidc created
2021/04/13 11:16:10 OpenID Connect discovery document in the S3 bucket <name>-oidc at .well-known/openid-configuration updated
2021/04/13 11:16:10 Reading public key
2021/04/13 11:16:10 JSON web key set (JWKS) in the S3 bucket <name>-oidc at keys.json updated
2021/04/13 11:16:18 Identity Provider created with ARN: arn:aws:iam::<aws_account_id>:oidc-provider/<name>-oidc.s3.<aws_region>.amazonaws.com
```
  where 02-openid-configuration is a discovery document and 03-keys.json is a JSON web key set file.
  
  This command also creates a YAML configuration file in /*<path_to_ccoctl_output_dir>*/manifests/cluster-authentication-02-config.yaml. This file sets the issuer URL field for the service account tokens that the cluster generates, so that the AWS IAM identity provider trusts the tokens.
Create IAM roles for each component in the cluster.
1. Extract the list of CredentialsRequest objects from the OKD release image:
```
$ oc adm release extract --credentials-requests --cloud=aws --to=<path_to_directory_with_list_of_credentials_requests>/credrequests quay.io/<path_to>/ocp-release:<version>
```
2. Use the ccoctl tool to process all CredentialsRequest objects in the credrequests directory:
```
$ ccoctl aws create-iam-roles --name=<name> --region=<aws_region> --credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests --identity-provider-arn=arn:aws:iam::<aws_account_id>:oidc-provider/<name>-oidc.s3.<aws_region>.amazonaws.com
```
  For AWS environments that use alternative IAM API endpoints, such as GovCloud, you must also specify your region with the —region parameter.
  
  For each CredentialsRequest object, ccoctl creates an IAM role with a trust policy that is tied to the specified OIDC identity provider, and a permissions policy as defined in each CredentialsRequest object from the OKD release image.

Verification

Creating AWS resources with a single command

If you do not need to review the JSON files that the ccoctl tool creates before modifying AWS resources, and if the process the ccoctl tool uses to create AWS resources automatically meets the requirements of your organization, you can use the ccoctl aws create-all command to automate the creation of AWS resources.

Otherwise, you can create the AWS resources individually.

Prerequisites

Extract and prepare the ccoctl binary.

Procedure

Extract the list of CredentialsRequest objects from the OKD release image:

$ oc adm release extract --credentials-requests --cloud=aws --to=<path_to_directory_with_list_of_credentials_requests>/credrequests quay.io/<path_to>/ocp-release:<version>

Use the ccoctl tool to process all CredentialsRequest objects in the credrequests directory:

$ ccoctl aws create-all --name=<name> --region=<aws_region> --credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests

Verification

To verify that the OKD secrets are created, list the files in the *<path_to_ccoctl_output_dir>*/manifests directory:
```
$ ll <path_to_ccoctl_output_dir>/manifests
```
Example output:

You can verify that the IAM roles are created by querying AWS. For more information, refer to AWS documentation on listing IAM roles.

Prerequisites

Create the install-config.yaml file for your installation.

Procedure

From the installation directory that contains your install-config.yaml file, configure the Cloud Credential Operator (CCO) to run in manual mode:
```
$ echo “credentialsMode: Manual” >> install-config.yaml
```
Create the required OKD installation manifests:
```
$ openshift-install create manifests
```
Copy the manifests that ccoctl generated to the manifests directory that the installation program created:
```
$ cp /<path_to_ccoctl_output_dir>/manifests/* ./manifests/
```
Copy the private key that the ccoctl generated in the tls directory to the installation directory:
```
$ cp -a /<path_to_ccoctl_output_dir>/tls .
```
Run the OKD installer:
```
$ ./openshift-install create cluster
```

Connect to the OKD cluster.

Verify that the cluster does not have root credentials:

$ oc get secrets -n kube-system aws-creds

The output should look similar to:

Error from server (NotFound): secrets "aws-creds" not found

Verify that the components are assuming the IAM roles that are specified in the secret manifests, instead of using credentials that are created by the CCO:

Example command with the Image Registry Operator
```
$ oc get secrets -n openshift-image-registry installer-cloud-credentials -o json | jq -r .data.credentials | base64 --decode
```
The output should show the role and web identity token that are used by the component and look similar to:

Example output with the Image Registry Operator