At the moment, this extension enables only Druid to autoscale instances in GCE.

    The extension manages the instances to be scaled up and down through the use of the Managed Instance Groups of GCE (MIG from now on). This choice has been made to ease the configuration of the machines and simplify their management.

    For this reason, in order to use this extension, the user must have created

    1. An instance template with the right machine type and image to bu used to run the MiddleManager
    2. A MIG that has been configured to use the instance template created in the point above
    • https://www.googleapis.com/auth/cloud-platform
    • https://www.googleapis.com/auth/compute

    The Overlord can dynamically change worker behavior.

    The JSON object can be submitted to the Overlord via a POST request at:

    Optional Header Parameters for auditing the config change can also be specified.

    1. {
    2.   "autoScaler": {
    3.       "numInstances" : 1,
    4.       "projectId" : "super-project",
    6.       "managedInstanceGroupName" : "druid-middlemanagers"
    7.     },
    8.     "maxNumWorkers" : 4,
    9.     "minNumWorkers" : 2,
    10.   }
    11. }

    The configuration of the autoscaler is quite simple and it is made of two levels only.

    The external level specifies the type—always gce in this case— and two numeric values, the and minNumWorkers used to define the boundaries in between which the number of instances must be at any time.

    The internal level is the envConfig and it is used to specify

    • The numInstances used to specify how many workers will be spawned at each request to provision more workers. This is safe to be left to 1
    • The projectId used to specify the name of the project in which the MIG resides
    • The zoneName used to identify in which zone of the worlds the MIG is
    • The managedInstanceGroupName used to specify the MIG containing the instances created or removed

    Known limitations

    • The module internally uses the ListManagedInstances call from the API and, while the documentation of the API states that the call can be paged through using the pageToken argument, the responses to such call do not provide any to set such parameter. This means that the extension can operate safely with a maximum of 500 MiddleManagers instances at any time (the maximum number of instances to be returned for each call).