Breaking changes

Significant changes

  • kops now supports running with objects as CRDs, stored in a kubernetes apiserver.
  • The apiGroup for kops objects has changed from kops to , to support CRDs. You can continue to provide either apiGroup as input (but you should ideally move to kops.k8s.io), but the output will always be of the kops.k8s.io form.
  • Rolling updates are much faster by default. A lot of the time-padding that was in previous versions has been replaced with reliance on validation. The --cloudonly case is much faster than previously, which we believe to be correct because we expect this is normally for disaster-recovery scenarios, but you may want to specify longer timings via flags if you are relying on time-based delays.

Required Actions

Full change list since 1.14.0 release

kops 1.14.0-beta.2 to 1.15.0-alpha.1

  • Release 1.14.0-alpha.1 #6772
  • Put 1.13 and 1.14 into channels #6781
  • 1.12 release notes: populate list of PRs #6780
  • Carry Provisioned IOPS to Terraform and CloudFormation templates #6776
  • pin nvidia-docker2 version to avoid installation failure #6768
  • pkg/model: Fix dropped error #6769
  • Using const() defines constants together (part:1) ,@xichengliudui
  • Using const() defines constants together (part:3) @xichengliudui
  • Update rules go @mikesplain
  • Update etcd3-migration.md doc @bksteiny, #6774
  • KubeAPIServer HTTP2 Stream Parameter #6787
  • Refactor names of URLs in assets to clarify their purpose #6420
  • Update docker README.md file, delete #6802
  • Switch to golang 1.11.5 #6798
  • Switch to golang 1.12.1 #6799
  • Using const() defines constants together (part:3) #6809
  • Using const() defines constants together (part:4) #6810
  • [docs] Use env var for state store examples #6278
  • Update README.md #6820
  • set kubernetes version to 1.12.8 to match current release #6833
  • Canal manifest updates for k8s v1.12+ #6823
  • Update readme chart given alphas and betas #6836
  • Fix typo in aws-iam-authenticator image field name #6840
  • Remove verify bazel and expose error #6841
  • Add t3a family #6837
  • Add support for AWS ap-east-1 region #6835
  • update tolerations to openstack external cloud provider #6821
  • [Unit Tests] Add unit tests for create_kubecfg file #6826
  • Protect against nil derefence #6859
  • Support Scale from 0 with Lauch Templates #6861
  • Remove spurious cadvisor dependency #6860
  • makefile: add gazelle alias for bazel-gazelle #6876
  • bazel: fix distroless imports for latest bazel #6877
  • Update kubernetes dependencies to k8s 1.13.5 #6857
  • Switch from glog to klog #6878
  • travis: Remove go-vet and boilerplate checking #6882
  • Use existing SSHKeyName if no public key is created. #6886
  • Start CRDification: Change apigroup to kops.k8s.io #6887
  • Simply bazel test using exclude pattern #6896
  • Include aws-cloud-provider roles in 1.15 #6899
  • Fix machine types with klog #6890
  • Avoid concurrent write corruption to /etc/hosts #6893
  • Add i3en instance types #6898
  • Fix typo in docker healthcheck #6901
  • Update to etcd-manager 1.0.20190509 #6917
  • Call klog.InitFlags in dns-controller #6925
  • Use klog logging from 1.15 #6924
  • S3 VFS: Default to current region from metadata service #6943
  • Canal v3.7.2 for k8s v1.12+ #6950
  • Subnet Update Consistency #6941
  • Configure AMIs for 1.12 #6963
  • Fix Docker not being installed on Ubuntu 16.04 #6965
  • bumped k8s 1.11 versions to 1.11.10 in alpha channel #6969
  • Issue #6945 #6951
  • Generate CRDs for kops API types #6891
  • etcd-manager: Update to 3.0.20190513 #6959
  • add node-exporter to allowed ports #6944
  • Make gofmt fails find usage #6954
  • Update commitlog relnotes for 1.12.0 #6981
  • 1.12 highlight changelog #6982
  • Mention version of kOps that introduced new features #6983
  • Terraform: fix options field, should be spot_options #6988
  • Add shortNames and columns to InstanceGroup CRD #6995
  • Add script to verify CRD generation #6996
  • Update README.md to reflect 1.12 release #7002
  • add kops instancegroup tag to metadata #6991
  • Don’t panic when deleting instancegroups #7000
  • Support using kops CLI with CRDs #7006
  • etcd-manager: update to 3.0.20190516 #7007
  • VPC cleanup: recognize the error code for concurrent VPC deletion #7008
  • Recommend kops 1.12.1 #7023
  • Add relnotes list for 1.12.1 #7022
  • Makefile: keep go vet simple #7030
  • Update go_version to 1.12.5 #7035
  • Start relnotes for 1.13 and 1.14 #7037
  • Speed up rolling-update - longer timeout on validation, less scheduled holds #6747
  • Update stretch dependencies and kubeup #7041
  • Bump alpha-channel of k8s #7038
  • Fix machine empheral disks #7062
  • Add docs for cpuCFSQuota / cpuCFSQuotaPeriod #7074
  • implement append admission controllers #7070
  • Add documentation for etcd-manager backup/restore procedures #7072
  • Fix typo on node-authorizer prometheus metric #7080
  • Openstack delete dynamic floating ip in delete cluster #7045
  • Updated docs for openstack cloud provider. #7092
  • Change versions to fix memory.limit_in_bytes: device or resource busy #7067
  • K8s 1.12.8 to stable 1.12.9 to alpha #7090
  • Fix link to Calico route reflectors documentation + typo #7088
  • Mark ENI 0 as delete_on_termination for LaunchTemplates #7094
  • Add Debian 10 (buster) support #7071
  • Openstack support for rolling-update status #7050
  • Upgrade AWS VPC CNI provider to 1.5.0 #7122
  • Documentation Cleanup #7123
  • Adding affinity and PDB to dns. #7077
  • bumped k8s 1.11 versions to 1.11.10 in stable channel #6984
  • Add support for SpotPrice and Mixed Instance ASGs #7066
  • support apiserver admission-control-config-file flag #7109
  • typo fix: fix kops-server-push -> make kops-server-push #7150
  • Add rdrgmnzs to the approvers list in OWNERS #7154
  • Flatcar support #7084
  • Don’t precreate etcd DNS records if we’re using etcd-manager #7141
  • Update Docs for Calico Backend for kops 1.12 #7164
  • Update Canal to v3.7.3 #7169
  • Improve docs on labels #7139
  • Allow user to set the —kube-api-qps and —kube-api-burst flags on KubeControllerManager #7153
  • Egress proxy for etcd manager #7103
  • [Unit Tests] Added unit test for kube proxy builder #7124
  • add c5.12xlarge, c5.24xlarge, c5.metal, i3en.metal #7166
  • Simplify go test command #7003
  • Spotinst: New instance group type: Ocean #7040
  • Fix the link to the Prow commands. #7162
  • add masterPublicName support in kops set cluster #7160
  • Update aws-iam-authenticator image to 0.4.0 #6803
  • Added some changes to openstack.md file #6985
  • Relnotes for 1.12.2 #7174
  • Add updated 1.12 image to the alpha channel #7176
  • Instance protection #7177
  • Remove kube-proxy resource-container flag #7224
  • Drop missing sources when building utils image #7217
  • goimports update #7218
  • Add more debug info for when cluster path doesnt match #7202
  • Canal v3.7.4 #7206
  • Upgrade Calico to 3.7.2 #7051
  • Spotinst: Ocean’s Strategy object is optional #7183
  • update instances list with make update-machine-types #7195
  • Possibility to use OpenStack without lbaas (loadbalancer) #7178
  • Clear append admission plugins before inserting flags to kube-apiserver #7182
  • Use NodeAuthorizer config options instead of soely hard-codes #7211
  • doc: support to debug kops-apiserver #7151
  • GCE tutorial markdown formatting #7188
  • Make an actual deep-copy of the state #7219
  • Set priority for static pods #6897
  • Allow setting Limit & Request for aws-iam-authenticator #7260
  • Delete the function keyword to prevent shellcheck from failing #6811
  • Bumping calico to 3.7.4. #7249
  • Update metrics server image #6871
  • Use readinessProbe for weave-net instead of livenessProbe #7102
  • Add some permissions to cluster-autoscaler clusterrole #7248
  • Spotinst: Rolling update always reports NeedsUpdate #7251
  • Add documentation example for running kOps in a CI environment #7256
  • Calico -> 3.7.4 for older versions #7282
  • [Issue-7148] Legacyetcd support for Digital Ocean #7221
  • Stop .gitignoring all files named go-bindata #7288
  • Create hack/update-expected.sh to update test output #7291
  • replace behavior for @aws hostnameOverride #7185
  • Rhel8 support #7287
  • Update DigitalOcean CCM to v0.1.16 #7293
  • Replace use of cmdutil IsFilenameSliceEmpty #7289
  • GCE: support ipalias networking mode, named “gce” #6229
  • Move NTP and misc packages initialization to code #6236
  • Machine types fix #7300
  • Improve channel updates ,@justinsb
  • Rationalize golden-output comparison @justinsb
  • hack/update-expected: regenerate gobindata @justinsb
  • Add me as reviewer @granular-ryanbonham
  • Update Calico to v3.8.0 @tmjd
  • Manifest hashing: move trimming out of hash function @justinsb
  • Adding documentation to mitigate workload outages on AWS @endzyme
  • Mount FlexVolume directory in kube-controller-manager pod @kellanburket
  • remove code: remove kops-server chart @Sn0rt
  • Bump alpha channel with latest kubernetes versions @justinsb
  • Default etcd-version to 3.3.10 for >= 1.14 @justinsb
  • Warn/prevent if the version of etcd is unsupported with etcd-manager @justinsb
  • Update Image version and RBAC for Citrix Ingress Controller @christus02
  • Promote k8s 1.12.9 from alpha -> stable @justinsb
  • Update repo-infra and distroless for bazel fixes @rifelpet
  • Cross-Zone Load Balancing for API ELB @austinmoore-
  • stop kubelet to prevent orphan containers @qqshfox
  • Update default flexvolumepath for COS @justinsb
  • Promote 1.12 image from alpha to stable @idealhack
  • Update kube-router to 0.3.1 @combor
  • Enable scraping of weave metrics @zacblazic
  • Bump etcd-manager to 3.0.20190801 @justinsb
  • Add mappings for Webhook authorization mode. @anderseknert
  • Set and mount the correct volume plugin dir based on OS @KashifSaadat
  • Don’t default adding MIMEBOUNDARY headers when a mixed instances policy is set @KashifSaadat
  • Add release notes for 1.13.0 beta.2 -> 1.13.0 @justinsb
  • Release notes for 1.12.2 -> 1.12.3 @justinsb
  • correct typo in output message @beautytiger
  • Remove extraneous note in 1.13 release notes @rifelpet
  • protokube/gce_volume.go: error info correction @beautytiger
  • cleanup: client.go error message words correction @beautytiger
  • awstasks: fix misspelled words in logging. @beautytiger
  • AWS SDK v1.23.0 @gjtempleton
  • Update Compatibility Table in Readme @austinorth
  • Add maxPersistentVolumes to support the KUBE_MAX_PD_VOLS scheduler setting @ripta
  • add zetaab as reviewer @zetaab
  • Support mirroring for nodeup also @justinsb
  • Use Cluster Proportional Autoscaler for CoreDNS 1.12+ @gjtempleton
  • Upgrading k8s-srcdst to v0.2.2 @michalschott
  • add OpenStack etcd-manager support @zetaab
  • Bump k8s versions in alpha channel @justinsb
  • Update AMIs in alpha channels @justinsb
  • skip verification when the file already installed @qqshfox
  • cleanup: fix error message typos @beautytiger
  • fix Typo ‘the the’ -> ‘the’ @xichengliudui
  • ali.go: cleanup error words in messages @beautytiger
  • promote k8s versions @zetaab
  • Dont set ExperimentalCriticalPodAnnotation feature gate in k8s 1.16 @rifelpet
  • fix typo “specifiction” -> “specification” @928234269
  • Add relnotes for 1.14.0-beta.1 @justinsb
  • bash script: don’t assume nodeup filename @justinsb
  • Add doc for using custom CA @joshbranham
  • nodeup download should try all mirrors @justinsb
  • Add nodeup to shipbot targets for release upload @justinsb
  • Update channel recommended versions for kops versions @justinsb
  • fix typo “in ingards to” -> “in regard to” @928234269
  • Update weave to 2.5.2 @while1eq1
  • cloudformation tests: use standard file comparison @justinsb
  • Look for sha256 and sha1 files for artifacts @justinsb
  • [Digital Ocean] DO-7442 upgrade godo client to latest version @srikiz
  • Fix Flatcar distro @mazzy89
  • fix typo “new” -> “newer” @928234269
  • print all failure messages @zetaab
  • [DO-7148] Digital Ocean support for etcd-manager @srikiz
  • Publish sha256 artifacts for kops itself @justinsb
  • Ignore empty hashfiles @justinsb
  • Update to kubernetes 1.15 @justinsb
  • util/pkg/vfs: Fix swallowed errors @alrs
  • Set GOPROXY in travis builds @justinsb
  • Allow configure ip to ip mode in calico @zetaab
  • Add exec-opts options to dockerconfig @tvi
  • move OpenStack from alpha to beta @zetaab
  • Add support for netExtraArgs @ReillyProcentive
  • Cleanup versions, deprecate kops 1.10, remove k8s 1.8 @mikesplain
  • Create verify-gomod script @justinsb
  • Update to golang 1.12.9 @justinsb
  • Corrected spelling of ‘we’ in the documentation @ashishbharthi
  • fix-up docs/releases/1.11-NOTE.md spelling mistake @tanjunchen
  • Copy well-known users from apiserver @justinsb
  • Replace resource.FilenameOptions with []string @justinsb
  • Configure calico MTU @zetaab
  • Create env-var helper function @justinsb
  • Label AWS ASGs with kops.k8s.io/instancegroup @justinsb
  • Support for using hostPort when using flannel @shamil
  • Remove unused ClientGetter from Drain code @justinsb
  • DeleteLocalData on drain @justinsb
  • Updating the vendored gazelle to match workspace. @mikesplain
  • Add verbosity @mikesplain
  • fix(addons/coredns.addons.k8s.io) Workaound to stop coredns crashing on 1.3.1 version @phspagiari
  • Update rules_docker with python2 workaround @rifelpet
  • Update and add back some sizes @mikesplain
  • Don’t try to delete ElasticIPs of NatGateway is shared @dzoeteman
  • fix(addons/networking.projectcalico.org) calico kube-controllers is needed in CRD mode @phspagiari
  • remove default insecure from openstack @zetaab
  • docs: fix link to Metrics Server user guide @ruxandrafed
  • fix static check error in vfssync.go @beautytiger
  • fix(upup/models/cloudup/resources/addons/coredns.addons.k8s.io) missing resourceVersion @phspagiari
  • modify-doc-small-mistake @tanjunchen
  • Calico update and typha ,@mikesplain
  • [Feature] CoreDNS: External CoreFile option @gjtempleton
  • Fix gomod errors @mikesplain
  • Log more sensibly when we can’t get sha256 @justinsb
  • Add horizontalPodAutoscalerDownscaleStabilization @mikesplain
  • Fix kops for us-gov-east-1 #7564 @ibrf
  • Fix Dropped Errors in upup @alrs
  • add cilium in error message @PascalBourdier
  • [DO-7442] Digital Ocean add consistent volume and droplet tags for multi master feature @srikiz
  • Expose API Server flags needed for AWS pod identities @rifelpet
  • Add logrotate for etcd/etcd-events.log @mikesplain
  • Updated container-selinux url to point to the right path @igarcia-sugarcrm, #7609
  • Check the HTTP response code when downloading URLs #7611
  • Clean security groups if api/ssh ips are removed from config #7561
  • Skip Docker install #6957
  • Add —wait argument to kops validate #7371
  • Fixed “NeedsUpdate” status of nodes in mixedinstancegroups after rolling update #7445
  • Associate subnets to port within OpenStack #7578
  • fix instance name #7641
  • Use without external router (OpenStack) #7644
  • Updating master IAM policies. #7580
  • Cherrypick #7581 into release 1.15 #7671
  • Pull centos.org packages from the vault #7674
  • Align AWS and kops validation for spot allocation strategy #7660
  • Limit calico cpu request to 100m #7688
  • Cherrypick #7690 onto release 1.15 #7693
  • Update etcd-manager with OpenStack fixes #7710
  • Change Cilium templates to standalone version ,@olemarkus
  • Update DigitalOcean CCM to v0.1.20 @timoreimann
  • Cilium standalone continuation @olemarkus
  • Add calico 3.9.1 @mikesplain
  • Fix some bugs reported by staticcheck @rifelpet
  • Add arg min-port=1024 to dnsmasq container in kube-dns @nr17
  • Add artifacts.k8s.io to mirror list @justinsb
  • Upgrade Amazon VPC CNI plugin to 1.5.4 @rifelpet
  • Add event ttl flag @tioxy
  • Kubelet configuration: Maximum pods flag is miscalculated when using Amazon VPC CNI @liranp
  • fix(apiserver): allow multiple service-account-key-file @hatappi
  • Openstack: value if spec does not associate public ips @mitch000001

1.15.0-beta.1 to 1.15.0

  • Cherry-pick #7807 to release-1.15 #7809
  • allow protocol rules in master #7835
  • Revert “Upgrade Amazon VPC CNI plugin to 1.5.4” #7847
  • Add back calico metrics options #7885
  • Remove extraneous document separator causing failures applying addons #7857
  • add missing priorityClassName to flannel DaemonSet #7842
  • Create PodDisruptionBudget for kube-dns in kube-system namespace ,@justinsb
  • Machine types updates @mikesplain
  • Add support for newer Docker versions @hakman
  • Add indent template function and use it to fix KubeDNS.ExternalCoreFile rendering #7979
  • fix(openstack): fix additional security groups on instance groups #8004
  • Fix Handling of LaunchTemplate Versions for MixedInstancePolicy #8038
  • Fix mounting Calico “flexvol-driver-host” in CoreOS #8062
  • Complete support for Flatcar #7545
  • Openstack: Fix cluster floating ips #8115
  • Bump cilium version to 1.6.4 #8022
  • mark weavenet-pod as system-critical #7874
  • cilium: don’t try to mount sys/fs/bpf if already mounted #7832
  • Update copyrights for 2020 #8241
  • Fix rendering of the Node Authorizer template #7916
  • Cherry pick #7874 onto 1.15 #8090
  • Backport the k8s 1.9 required action release note #8378
  • Don’t output empty sections in the manifests ,@rifelpet
  • Fix issues with older versions of k8s for basic clusters @hakman, #8248
  • CoreDNS default image bump to 1.6.6 to resolve CVE #8333
  • Don’t load nonexistent calico-client cert when CNI is Cilium #8338
  • kOps releases - prefix git tags with v #8373

1.15.1 to 1.15.2

  • Fix Github download url for nodeup @adri, #8468
  • GCS: Don’t try to set ACLs if bucket-policy only is set #8493
  • Cilium - Add missing Identity Allocation Mode to Operator Template #8445
  • Make it possible to enable Prometheus metrics for Cilium #8433
  • Stabilize sequence of “export xx=xxx” statements @mitch000001
  • Properly detect that bpffs has been mounted @olemarkus
  • Update to etcd-manager 3.0.20200428 @justinsb