How to use kOps in AWS China Region

Most of the following procedures to provision a cluster are the same with the guide to use kOps in AWS. The differences will be highlighted and the similar parts will be omitted.

NOTE: THE FOLLOWING PROCEDURES ARE ONLY TESTED WITH KOPS 1.10.0, 1.10.1 AND KUBERNETES 1.9.11, 1.10.12

Setup your environment

AWS

When aws configure, remember to set the default region name to the correct one, e.g. cn-north-1.

And export it correctly.

export AWS_REGION=$(aws configure get region)

As the note kindly pointing out, a gossip-based cluster can be easily created by having the cluster name end with .k8s.local. We will adopt this trick below. Rest of this section can be skipped safely.

Thanks to gossip, this section can be skipped safely as well.

Since we are provisioning a cluster in AWS China Region, we need to create a dedicated S3 bucket in AWS China Region.

aws s3api create-bucket --bucket prefix-example-com-state-store --create-bucket-configuration LocationConstraint=$AWS_REGION

Ensure you have a VPC which can access the internet NORMALLY

We have to build our own AMI because there is . There’re two ways to accomplish so.

ImageBuilder RECOMMENDED

First, launch an instance in a private subnet which accesses the internet fast and stably.

Because the instance launched in a private subnet, we need to ensure it can be connected by using the private ip via a VPN or a bastion.

SUBNET_ID=<subnet id> # a private subnet
SECURITY_GROUP_ID=<security group id>
KEY_NAME=<key pair name on aws>
AMI_ID=$(aws ec2 describe-images --filters Name=name,Values=debian-jessie-amd64-hvm-2016-02-20-ebs --query 'Images[*].ImageId' --output text)
INSTANCE_ID=$(aws ec2 run-instances --image-id $AMI_ID --instance-type m3.medium --key-name $KEY_NAME --security-group-ids $SECURITY_GROUP_ID --subnet-id $SUBNET_ID --no-associate-public-ip-address --query 'Instances[*].InstanceId' --output text)
aws ec2 create-tags --resources ${INSTANCE_ID} --tags Key=k8s.io/role/imagebuilder,Value=1

Now follow the documentation of in kube-deploy to build the image.

Copy AMI from another region

Following to copy the kOps image from another region, e.g. .

Get the AMI id

No matter how to build the AMI, we get an AMI finally, e.g. k8s-1.9-debian-jessie-amd64-hvm-ebs-2018-07-18.

Prepare local environment

Set up a few environment variables.

export NAME=example.k8s.local
export KOPS_STATE_STORE=s3://prefix-example-com-state-store

Create cluster configuration

aws ec2 describe-availability-zones

Below is a create cluster command which will create a complete internal cluster . The below command will generate a cluster configuration, but not start building it. Make sure that you have generated SSH key pair before creating the cluster.

VPC_ID=<vpc id>
VPC_NETWORK_CIDR=<vpc network cidr> # e.g. 172.30.0.0/16
AMI=<owner id/ami name> # e.g. 123456890/k8s-1.9-debian-jessie-amd64-hvm-ebs-2018-07-18
kops create cluster \
    --zones ${AWS_REGION}a \
    --network-cidr ${VPC_NETWORK_CIDR} \
    --image ${AMI} \
    --associate-public-ip=false \
    --api-loadbalancer-type internal \
    --topology private \
    --networking calico \
    ${NAME}

Now we have a cluster configuration, we adjust the subnet config to reuse shared subnets by editing the description.

Then change the corresponding subnets to specify the and remove the cidr, e.g.

spec:
  subnets:
  - id: subnet-12345678
    name: cn-north-1a
    type: Private
    zone: cn-north-1a
  - id: subnet-87654321
    name: utility-cn-north-1a
    type: Utility
    zone: cn-north-1a

Another tweak we can adopt here is to add a docker section to change the mirror to . This will increase stability and download speed of pulling images from docker hub.

spec:
  docker:
    registryMirrors:
    - https://registry.docker-cn.com

Please note that this mirror MIGHT BE not suitable for some cases. It’s can be replaced by any other registry mirror as long as it’s compatible with the docker api.

Add more master nodes

In one AZ

To achieve this, we can add more parameters to kops create cluster.

  --zones ${AWS_REGION}a --node-count 2 \

In two AZs

Please note that this will still have 50% chance to break the cluster when one of the AZs are down.

AWS China