Karpenter

    Karpenter is a fairly new project, and it is still not determined how Karpenter should work with kOps. Because of this, Karpenter is behind the feature flag.

    Enable the Karpenter feature flag:

    Karpenter requires that external permissions for ServiceAccounts be enabled for the cluster. See AWS IAM roles for ServiceAccounts documentation for how to enable this.

    You also need to enable the Karpenter addon in the cluster spec:

    New clusters

    On new clusters, you can simply add the --instance-manager=karpenter flag:

    A Karpenter-managed InstanceGroup controls a corresponding Karpenter Provisioner resource. kOps will ensure that the Provisioner is configured with the correct AWS security groups, subnets, and launch templates. Just like with ASG-managed InstanceGroups, you can add labels and taints to Nodes and kOps will ensure those are added accordingly.

    If you do not specify a mixed instances policy, only the instance type specified by will be used. With Karpenter, one typically wants a wider range of instances to choose from. kOps supports both providing a list of instance types through spec.mixedInstancesPolicy.instances and providing instance type requirements through . See (/instance_groups)[InstanceGroup documentation] for more details.

    kOps will ignore the InstanceGroup spec.subnets configuration and unconditionally add all subnets eligible to run Nodes to all Provisioners. It is not currently possible to create Provisioners limited only to certain subnets.

    Karpenter-managed Launch Templates

    On EKS, Karpener creates its own launch templates for Provisioners. These launch templates will not work with a kOps cluster for a number of reasons. Most importantly, they do not use supported AMIs and they do not install and configure nodeup, the instance-side kOps component. The Karpenter features that require Karpenter to directly manage launch templates will not be available on kOps.

    Other minor limitations

    • Control plane nodes must be provisioned with an ASG, not Karpenter.
    • Provisioners will unconditionally use spot instances
    • Provisioners will unconditionally include burstable instance groups such as the T3 instance family.
    • kOps will not allow mixing arm64 and amd64 instances in the same Provider.