Significant changes

  • Components are no longer allowed to interact with etcd directly. Calico will be switched to use CRDs instead of directly with etcd. This is a disruptive upgrade, please read the calico notes in the etcd migration documentation

Required Actions

  • Please back-up important data before upgrading, as the etcd2 to etcd3 migration is higher risk than most upgrades. The upgrade is disruptive to the masters, see notes above.
  • Note that the upgrade for Calico users is disruptive, because it requires switching from direct-etcd-storage to CRD backed storage.

Full change list since 1.11.0 release

1.11.0 to 1.12.0-alpha.1

  • machine-type generator: Warn if instance type not in ENI map #6118
  • Include name of unhealthy component in validation error #6122
  • Bump alpha channel kubernetes versions #6123
  • Add missing locking to awsmock LaunchConfigurations #6124
  • Add a1 and c5n instance types #6117
  • Simplify makefile for update-machine-types #6121
  • Update docs, removing brew —devel #6125
  • machine-types: remove duplicate dedup #6127
  • Update amazon cni to 1.3.0 #6128
  • Enable HPA tolerance configuration #6130
  • Update addons dashboard version #6136
  • Spotinst: Bump controller image #6129
  • Add cni to usage network option for kops create cluster #6139
  • Workspace updates for bazel / fix tests #6144
  • Promote alpha channels to stable #6146
  • Add GCE europe-north1-{a,b,c} #6152
  • Add self to security contacts #6147
  • Fix missed stable channel upgrade path #6158
  • Fix Calico upgrade job to use the correct version #6156
  • Fix for when node and master use the same SG. #6175
  • Add experimental and metrics flags for docker #6171
  • Add y flag for upgrade command for consistency #6177
  • Add-ons spec example is missing “manifest”. #6170
  • ExperimentalAllowedUnsafeSysctls has moved to AllowedUnsafeSysctls in k8s 1.11 #6179
  • Let a user specify the validation timeout when rotating a cluster. #6185
  • fix(docs): fix the compatibility matrics on hpa.md #6193
  • bump prometheus-operator version and deploy file #6196
  • update heapster version and mark it retired #6195
  • Add Docker 18.06.1 for CentOS and RHEL 7 #6202
  • Print —name with #6208
  • Add —post-drain-delay to rolling-update cluster command #6211
  • Adding kubernetes-dashboard v1.10.1 deployment to kops addons #6224
  • Consider pending pods to be a validation failure #6231
  • Adding support for the new Stockholm region #6212
  • Document how to update an existing vendored dependency #6238
  • Update to k8s 1.12 libraries #5932
  • Bump channels and bump alpha to latest #6239
  • Automagically use curl instead of wget if that’s what’s available #6090
  • cloudmock: replace unimplemented methods with interface embedding #6243
  • bazel: cleanup gobindata generation #6235
  • Update apimachinery for k8s 1.12 #6245
  • Bulk spelling fixes #6242
  • Don’t panic when an etcd cluster is added #6180
  • Update aws-sdk-go to 1.16.9 #6237
  • Add p3dn.24xlarge #6253
  • Rationalize deserialiation code #6259
  • Always log when a retry loop fails #6260
  • Update compatibility for v1.11.0 #6258
  • AWS SDK v1.16.11 #6276
  • nodeup: include underlying error in error message #6279
  • release process: add the relnotes command #6269
  • Fix missed error check in hasPlaceHolderIP #6272
  • Create dev-upload tasks, for a faster upload during dev builds #6233
  • Update recommended kubernetes version #6271
  • Release notes for 1.11 #6270
  • fixed the sentence mistake #6281
  • update calico version to version 3.4.0 #6263
  • Remove duplicate Deployment for prometheus-operator #6265
  • Update aws-china.md #6262
  • Recognize 2019 as a year #6288
  • Change jessie to stretch #6293
  • Included type in SSL certificate documentation #6289
  • Update distroless #6287
  • Promote alpha kubernetes versions to stable #6298
  • Create prow-postsubmit target for release candidates #6299
  • Include windows build in distribution #6300
  • Fix kubelet api admin #6312
  • GCE terraform: map source tags in firewallrule #6295
  • GCE terraform: support labels #6296
  • Add extra privilege to prometheus-k8s ClusterRole
  • Kubelet API RBAC Manifest @gambol99
  • Upgrading coredns version to 1.3.0 @harshal-shah
  • Release 1.12.0-alpha.1 @justinsb
  • Retry Logging @gambol99
  • Fix prow-postsubmit by copying prebuilt archive in bazel @justinsb
  • Remove Initializers from default admission plugins for 1.12+ @liggitt
  • include docker 18.06.1 missed dependency @nareshku
  • Fix alternative AWS partitions in custom instance profiles @rifelpet
  • Add doc regarding upgrading to CoreDNS @joshbranham
  • AWS: Enable ICMP Type 3 Code 4 for API server ELBs @davidarcher
  • Additional Storage & Volume Mounting @gambol99
  • kOps for Openstack @jrperritt,,@wozniakjan, #6351
  • Update go version to 1.10.8 #6401
  • Suffix openstack subnet name with cluster name #6380
  • minor grammar improvements to kops terraform docs #6301
  • Docs: Drop last DrainAndValidateRollingUpdate note #6374
  • Allow users to set kubelet cpu-cfs-quota and cpu-cfs-quota-period flags #6375
  • implement etcd status for openstack #6381
  • remove using deviceowner when filtering existing routerinterfaces #6382
  • ignore openstack managed volume tags #6383
  • kops version: Add —short flag, use it to get version in scripts #6232
  • find sshkey resource when updating cluster #6384
  • implement GetCloudGroups for openstack #6386
  • minor fixes to openstack #6387
  • fix openstack lb pool member logic #6388
  • Support “egress: External” to avoid configuring networking ,@cassandracomar, #6218
  • Bump alpha channels #6405
  • Update bazel rules #6406
  • implement delete cluster for openstack #6385
  • Openstack Floating IP Deletion #6425
  • update openstack documentation #6423
  • Updated OWNERS file to include link to docs #6450
  • [jjo] add docker-ce 18.06.2 for CVE-2019-5736 #6460
  • Add permission for CreateTag on ENI to amazon-vpc-cni-k8s #6389
  • Document etcd3 migration process #6408
  • Normalize etcd cluster provider names #6410
  • Support etcd-manager v3, suitable for backporting #6411
  • Openstack loadbalancers erronous modification requests #6413
  • fix typos for addon doc #6416
  • upgrade calico to 2.6.12 to fix TTA-2018-001 #6422
  • Use the forward plugin instead of proxy plugin in CoreDNS #6424
  • Update bazel workspace #6426
  • Fix machine types and cleanup makefile #6427
  • Add jessie patch ,@mikesplain
  • Allow NodeAuthorizer to speak via HTTP Proxy if configured @KashifSaadat
  • Updated Canal manifest to v3.5.0 for k8s v1.12+ @KashifSaadat
  • Update document for GPU support @yujunz
  • Fixing kops-4049 @mmerrill3
  • kube-apiserver: Add oidc-required-claim flag @jeyglk
  • add OWNERS file to openstack spesific folders @zetaab
  • Update Loadbalancer Pools @drekle
  • fix hostnames in kops openstack @zetaab
  • implement ig deletegroup for openstack @zetaab
  • Removing openstack credential file support @drekle
  • fix error when updating/creating lb in openstack @zetaab
  • recheck floatingip after server is active @zetaab
  • Ability to scale down instancegroup in openstack @zetaab
  • expose DryRunTarget changes and deletions @zetaab
  • support both octavia and old lbaasv2 api in openstack @zetaab
  • Guess SSH usernames for RHEL & Centos in toolbox dump @justinsb
  • Choose docker version 18.06.2 for k8s >= 1.12 @justinsb
  • Install kubelet config for default centos user @justinsb
  • Update the CoreDNS manifest @rajansandeep
  • docs: improve the queries for finding RHEL/CentOS images @justinsb
  • Workaround for overlay2 vs rhel-family docker bug @justinsb
  • retry l3floatingip list in fresh cluster @zetaab
  • Update 1.12 addon manifests to use apps/v1, rbac v1 @liggitt
  • Fix package name & version for container-selinux @justinsb
  • AWS Mixed Instances Policy / Fleet @gambol99
  • Adding Comment @gambol99
  • Kube Proxy Metrics Option @gambol99
  • Sprig (Toolbox Templating) @gambol99
  • Etcd memory and cpu requests @integrii
  • Map docker 18.06.3 @justinsb
  • Make docker 18.06.3 the default for k8s >= 1.12 @justinsb
  • Document strategy for cve_2019_5736 @justinsb
  • Try using chattr to mark docker-runc as immutable @justinsb
  • Simple mirror support @justinsb
  • Bump etcd-manager version to 3.0.20190224 @justinsb
  • update gophercloud vendor dependencies @zetaab
  • specify dns servers to openstack subnet @zetaab
  • possibility to specify floatingip subnet for resources in openstack @zetaab
  • Add Experimental Cluster Signing Duration flag @pgdagenais
  • set net.ipv4.ip_local_reserved_ports to the KubeAPIServer ServiceNodePortRange parameter on nodeup @sp-joseluis-ledesma
  • spread instances equally to all AZs @zetaab
  • update-machine-types: more metal instance types @justinsb
  • Add changelist for 1.11.1 @justinsb
  • Fix panic when using etcd-manager and resource requests are nil @KashifSaadat
  • Promote Kubernetes 1.11.7 to stable @olemarkus
  • Upgrade alpha to latest @mikesplain
  • implement delete instance, this is needed in rolling-update @zetaab
  • Stop setting deprecated —allow-privileged Kubelet flag in 1.14 @mtaufen
  • Openstack Security Group hardening @drekle
  • Update embargo doc link in SECURITY_CONTACTS and change PST to PSC @joelsmith
  • Instance LaunchConfig/Template Bug Fix @gambol99
  • add docker.insecureRegistries flag @kimxogus
  • Add line breaks in example release cycle @MMeent
  • [jjo] Update Weave Net to version 2.5.1 @jjo
  • Adding installation guidelines for Windows @EchoDelta
  • Remove confusing comma in README @mattjmcnaughton
  • Add ServiceAccountKeyFile to KubeAPIServerConfig @Smirl
  • moving chrisz100 to approver level @chrisz100
  • Fix dashboard yaml that returned 404 @mausch
  • Replace Y / N Markings of Compatibility Matrix in readme with ✔ / ❌ @compilenix
  • Rename addon.yml to addon.yaml @jsharpe
  • addons/cluster-autoscaler: Add jq installation for OSX environment @iBluemind
  • Update docs on authentication @flands
  • Omit IP-in-IP protocols in Openstack CNI Rules @marsavela
  • External out-of-tree CloudControllerManager support for openstack @zetaab
  • Use EnsureTask for create static pod directory @Smirl
  • Fix documentation about targetGroupArn key @phyrog
  • Update rolling_update.md @rj03hou, #6247
  • fix typo #6017
  • Correcly handle CRLF in the manifest #6570
  • Fix confusing k8s upgrade docs for Terraform users ,@justinsb
  • Added Audit Webhook config @mbelangerupgrade, #6361
  • Spotinst: Avoid spurious changes #6028
  • Fix amazon-vpc-routed-eni yaml template #6502
  • Replace gcr.io URL with k8s.gcr.io vanity URL #6623
  • support gossip for AliCloud #6319
  • add natGateways tasks for ALICloud #6402
  • Fix some of the docker package names & versions #6620
  • Apply scope fix in #6502 to all manifest versions ,@justinsb
  • Add —kubeconfig flag to kops export kubecfg @adamyy
  • add support to set cluster spec.kubelet @phedoreanu
  • Upgrade bazel gazelle @mikesplain
  • Fix typo @justinsb
  • Support g3s for gpu driver installation @reverson
  • Fix docker-healthcheck to work around Docker bug. @tsuna
  • docs: create checklist for new kubernetes version @justinsb
  • Fix metrics server addon @itskingori
  • Always create /var/lib/kubelet, even in bootstrap mode @justinsb
  • Launch Template Feature Flag @gambol99
  • Remove docker-prestart hook @stevenjm
  • kops 1.12 configuration for calico: use CRDs @justinsb
  • Quick Clean @gambol99
  • Sync data-types for webhook config with upstream @justinsb
  • Add manage security groups for loadbalancers @zetaab
  • Enable etcd-manager / etcd3 / etcd-tls in kops 1.12 @justinsb
  • Use EnsureTask for internal api route53 record @Smirl
  • Added reminder to publish conformance results in release process @chrisz100
  • Update aws-china.md @qqshfox
  • Openstack server name collisions @drekle
  • tiny backslash arrangement @sevenfourk
  • Openstack environment escaping @drekle
  • Update upgrade.md @gamename
  • add ALI flags @LilyFaFa
  • Override volume zone name @zetaab
  • Updated Flannel manifest to 0.11.0 @gordonbondon
  • Update flannel version in bootstrapchannelbuilder @gordonbondon, #6663
  • Add flags for TLS Cipher suites customization for API Server, Kubelet and Controller-Manager #6470
  • If using etcd-backup and TLS is enabled, pass relevant options #6562
  • Bump etcd-manager / etcd-backup to 3.0.20190325 #6664
  • 2048 - Add cloudLabels as tags to API ELB resource #6646
  • Bump K8s 1.11 to 1.11.9 in the alpha channel #6665
  • Upgrade rules go #6667
  • Fix a missing dep lock #6668

1.12.0-alpha.1 to 1.12.0-alpha.2

  • Support download protokube from mirror @justinsb
  • Promote alpha to stable and update alpha @mikesplain
  • Upload protokube to github as part of release @justinsb
  • Use CNI 0.7.5 @justinsb
  • Put 1.12 into stable channel, for users of kops 1.12-alphas @justinsb
  • Support mirrors with restricted characters @justinsb

1.12.0-alpha.2 to 1.12.0-alpha.3

  • Fix Key error change Overrides to Override #6691
  • Add selector back to calico 1.12 deployment #6682
  • Update etcd-manager to 1.0.20190328 #6695

1.12.0-alpha.3 to 1.12.0-beta.1

1.12.0-beta.1 to 1.12.0-beta.2

  • kube-dns: Update to 1.14.13 #6741
  • Launch Template use version number as well as name. #6755
  • use dynamic s3 prefix in addAmazonVPCCNIPermissions func #6765

1.12.0-beta.2 to 1.12.0

  • IAM Permission to Support Scaling from 0 with Lauch Templates @granular-ryanbonham
  • Avoid concurrent write corruption to /etc/hosts @justinsb, #6893
  • Add i3en instance types #6898
  • Add t3a family #6905
  • Use existing SSHKeyName if no public key is created. #6886
  • bazel: fix distroless imports for latest bazel #6910
  • pkg/model: Fix dropped error #6911
  • Add ability to specify cpuRequest for API Server #6706
  • KubeAPIServer HTTP2 Stream Parameter #6913
  • Add support for AWS ap-east-1 region #6835
  • Add min-resync-period for Controller Manager #6737
  • Allow the AWS IAM Authenticator image name to be overridden #6730
  • Add cpu management policy config #5961
  • Carry Provisioned IOPS to Terraform and CloudFormation templates #6776
  • update tolerations to openstack external cloud provider #6821
  • Fix typo in aws-iam-authenticator image field name #6840
  • add the registry-qps kubelet flag #6357
  • Deep-copy proto state to prevent concurrent modification #6707
  • Publish utils.tar.gz to github releases also #6680
  • Allow uneven etcd zones #6641
  • Add terraform support for additional CIDR blocks. #6693
  • Canal manifest updates for k8s v1.12+ #6823
  • Update to etcd-manager 1.0.20190509 #6917
  • S3 VFS: Default to current region from metadata service ,@granular-ryanbonham
  • etcd-manager: Update to 3.0.20190513 @justinsb
  • Fix Docker not being installed on Ubuntu 16.04 @meeee
  • Issue #6945 @pkutishch, #6951

1.12.0 to 1.12.1

  • Don’t panic when deleting instancegroups @justinsb
  • etcd-manager: update to 3.0.20190516 @justinsb
  • Terraform: fix options field, should be spot_options @kimxogus

1.12.1 to 1.12.2

  • Mark ENI 0 as delete_on_termination for LaunchTemplates #7094

1.12.2 to 1.12.3

  • Cherry pick of #7211: Use NodeAuthorizer config options instead of soely @jacksontj
  • Cherry pick of #7219: Make an actual deep-copy of the state @jacksontj
  • Upgrade Calico to 3.7.2 @asincu
  • Update canal to 3.6.4, for TTA-2019-002 @justinsb
  • Bumping calico to 3.7.4. @michalschott
  • Cherry pick of #7185: Replace behavior for aws hostnameOverride @jacksontj
  • Calico -> 3.7.4 for older versions @justinsb
  • Warn/prevent if the version of etcd is unsupported with etcd-manager @justinsb