Docker

    Before beginning the quickstart, it is helpful to read the general Druid overview and the , as the tutorials will refer to concepts discussed on those pages. Additionally, familiarity with Docker is recommended.

    • Docker

    The Druid source code contains an example which can pull an image from Docker Hub and is suited to be used as an example environment and to experiment with Docker based Druid configuration and deployments.

    The example docker-compose.yml will create a container for each Druid service, as well as ZooKeeper and a PostgreSQL container as the metadata store.

    It will also create a named volumes druid_shared, which is mounted as opt/shared in container, as deep storage to keep and share segments and task logs among Druid services.

    The Druid containers are configured via an .

    Configuration

    Special environment variables:

    • JAVA_OPTS — set java options
    • DRUID_LOG4J — set the entire log4j.xml verbatim
    • DRUID_LOG_LEVEL — override the default log level in default log4j
    • DRUID_XMX — set Java
    • DRUID_MAXNEWSIZE — set Java max new size
    • DRUID_NEWSIZE — set Java new size
    • DRUID_MAXDIRECTMEMORYSIZE — set Java max direct memory size
    • DRUID_CONFIG_COMMON — full path to a file for druid ‘common’ properties
    • DRUID_CONFIG_${service} — full path to a file for druid ‘service’ properties

    In addition to the special environment variables, the script which launches Druid in the container will also attempt to use any environment variable starting with the druid_ prefix as a command-line configuration. For example, an environment variable

    would be translated into

    -Ddruid.metadata.storage.type=postgresql

    The Druid docker-compose.yml example utilizes a single environment file to specify the complete Druid configuration; however, in production use cases we suggest using either DRUID_COMMON_CONFIG and DRUID_CONFIG_${service} or specially tailored, service-specific environment files.

    Run docker-compose up to launch the cluster with a shell attached, or docker-compose up -d to run the cluster in the background. If using the example files directly, this command should be run from distribution/docker/ in your Druid installation directory.

    Once the cluster has started, you can navigate to . The Druid router process, which serves the , resides at this address.

    It takes a few seconds for all the Druid processes to fully start up. If you open the console immediately after starting the services, you may see some errors that you can safely ignore.

    If you experience any processes crashing with a 137 error code you likely don’t have enough memory allocated to Docker. 6 GB may be a good place to start.