Getting Started with kOps on Spot Ocean

    Ocean seamlessly integrates with your existing instance groups, as a drop-in replacement for AWS Auto Scaling groups, and allows you to streamline and optimize the entire workflow, from initially creating your cluster to managing and optimizing it on an ongoing basis.

    • Simplify Cluster Management — Ocean’s Virtual Node Groups make it easy to run different infrastructure in a single cluster, which can span multiple AWS VPC availability zones and subnets for high-availability.

    • Container-Driven Autoscaling and Vertical Rightsizing — Auto-detect your container infrastructure requirements so the appropriate instance size or type will always be available. Measure real-time CPU/Memory consumption of your Pods for ongoing resource optimization.

    • Cloud-Native Showback — Gain a granular view of your cluster’s cost breakdown (compute and storage) for each and every one of the cluster’s resources such as Namespaces, Deployments, Daemon Sets, Jobs, and Pods.

    • Optimized Pricing and Utilization — Ocean not only intelligently leverages Spot Instances and reserved capacity to reduce costs, but also eliminates underutilized instances with container-driven autoscaling and advanced bin-packing.

    Prerequisites

    Make sure you have and installed kubectl.

    Setup your environment

    To use environment variables, run:

    To use credentials file, run the spotctl configure command:

    1. ? Enter your access token [? for help] **********************************
    2. ? Select your default account [Use arrows to move, ? for more help]
    3. act-0abcdefg (dev)

    Or, manually create an INI formatted file like this:

    1. [default]
    2. token = <spotinst_token>
    3. account = <spotinst_account>

    and place it in:

    • Unix/Linux/macOS:

    • Windows:

      1. %UserProfile%\.spotinst\credentials

    AWS

    Creating a Cluster

    You can add an Ocean instance group to new or existing clusters. To create a new cluster with a Ocean instance groups, run:

    1. # configure the feature flags
    2. export KOPS_FEATURE_FLAGS="Spotinst,SpotinstOcean"
    3. # create the cluster
    4. kops create cluster --zones=us-west-2a example

    Note

    It’s possible to have a cluster with both Ocean-managed and unmanaged instance groups.

    1. # instancegroups.yaml
    2. # A cluster with both Ocean-managed and unmanaged instance groups.
    3. ---
    4. # Use Ocean in hybrid mode.
    5. apiVersion: kops.k8s.io/v1alpha2
    6. labels:
    7. kops.k8s.io/cluster: "example"
    8. spotinst.io/hybrid: "true"
    9. ...
    10. ---
    11. # Use AWS Auto Scaling group.
    12. apiVersion: kops.k8s.io/v1alpha2
    13. kind: InstanceGroup
    14. metadata:
    15. labels:
    16. kops.k8s.io/cluster: "example"
    17. ...

    Creating an Instance Group

    To create a new instance group, run:

    1. # configure the feature flags
    2. export KOPS_FEATURE_FLAGS="Spotinst,SpotinstOcean"
    3. # create the instance group
    4. kops create instancegroup --role=node --name=example

    To create a new instance group and have more control over the configuration, a config file can be used.

    LabelDescriptionDefault
    spotinst.io/spot-percentageSpecify the percentage of Spot instances that should spin up from the target capacity.100
    spotinst.io/utilize-reserved-instancesSpecify whether reserved instances should be utilized.true
    spotinst.io/utilize-commitmentsSpecify whether reserved instance commitments should be utilized.none
    spotinst.io/fallback-to-ondemandSpecify whether fallback to on-demand instances should be enabled.
    spotinst.io/draining-timeoutSpecify a period of time, in seconds, after a node is marked for termination during which on running pods remains active.none
    spotinst.io/grace-periodSpecify a period of time, in seconds, that Ocean should wait before applying instance health checks.none
    spotinst.io/ocean-default-launchspecSpecify whether to use the InstanceGroup’s spec as the default Launch Spec for the Ocean cluster.none
    spotinst.io/ocean-instance-types-whitelistSpecify whether to whitelist specific instance types.none
    spotinst.io/ocean-instance-types-blacklistSpecify whether to blacklist specific instance types.none
    spotinst.io/ocean-instance-typesSpecify a list of instance types that should be used by the Ocean Launch Spec.none
    spotinst.io/autoscaler-disabledSpecify whether the auto scaler should be disabled.false
    spotinst.io/autoscaler-default-node-labelsSpecify whether default node labels should be set for the auto scaler.false
    spotinst.io/autoscaler-auto-configSpecify whether headroom resources should be automatically configured and optimized.true
    spotinst.io/autoscaler-auto-headroom-percentageSpecify the auto headroom percentage (a number in the range [0, 200]) which controls the percentage of headroom.none
    spotinst.io/autoscaler-headroom-cpu-per-unitSpecify the number of CPUs to allocate for headroom. CPUs are denoted in millicores, where 1000 millicores = 1 vCPU.none
    spotinst.io/autoscaler-headroom-gpu-per-unitSpecify the number of GPUs to allocate for headroom.none
    spotinst.io/autoscaler-headroom-mem-per-unitSpecify the amount of memory (MB) to allocate for headroom.none
    spotinst.io/autoscaler-headroom-num-of-unitsSpecify the number of units to retain as headroom, where each unit has the defined CPU and memory.none
    spotinst.io/autoscaler-cooldownSpecify a period of time, in seconds, that Ocean should wait between scaling actions.300
    spotinst.io/autoscaler-scale-down-max-percentageSpecify the maximum scale down percentage.none
    spotinst.io/autoscaler-scale-down-evaluation-periodsSpecify the number of evaluation periods that should accumulate before a scale down action takes place.5
    spotinst.io/autoscaler-resource-limits-max-vcpuSpecify the maximum number of virtual CPUs that can be allocated to the cluster.none
    spotinst.io/autoscaler-resource-limits-max-memorySpecify the maximum amount of total physical memory (in GiB units) that can be allocated to the cluster.none
    Specify whether the scale-down activities should be restricted.none

    Documentation

    If you’re new to and want to get started, please checkout our Getting Started guide, available on the website.

    Getting Help

    • Join our community on Slack.
    • Ask a question on and tag it with spot-ocean.