Metadata storage

    Derby is the default metadata store for Druid, however, it is not suitable for production. MySQL and are more production suitable metadata stores.

    Add the following to your Druid configuration.

    MySQL

    See .

    See postgresql-metadata-storage.

    Adding custom dbcp properties

    Example supported properties:

    See BasicDataSource Configuration for full list.

    This is dictated by the druid.metadata.storage.tables.segments property.

    This table stores metadata about the segments that should be available in the system. (This set of segments is called “used segments” elsewhere in the documentation and throughout the project.) The table is polled by the to determine the set of segments that should be available for querying in the system. The table has two main functional columns, the other columns are for indexing purposes.

    Value 1 in the used column means that the segment should be “used” by the cluster (i.e., it should be loaded and available for requests). Value 0 means that the segment should not be loaded into the cluster. We do this as a means of unloading segments from the cluster without actually removing their metadata (which allows for simpler rolling back if that is ever an issue).

    Note that the format of this blob can and will change from time-to-time.

    The rule table is used to store the various rules about where segments should land. These rules are used by the Coordinator when making segment (re-)allocation decisions about the cluster.

    The config table is used to store runtime configuration objects. We do not have many of these yet and we are not sure if we will keep this mechanism going forward, but it is the beginnings of a method of changing some configuration parameters across the cluster at runtime.

    There are also a number of tables created and used by the and MiddleManager when managing tasks.

    The Audit table is used to store the audit history for configuration changes e.g rule changes done by and other config changes.

    Accessed by

    1. Indexing Service Processes (if any)
    2. Realtime Processes (if any)
    3. Coordinator Processes

    Thus you need to give permissions (e.g., in AWS Security Groups) only for these machines to access the Metadata storage.