Resource Management

    A resource contains basic information such as name and type. The name is globally unique. Different types of resources contain different attributes. Please refer to the introduction of each resource for details.

    The creation and deletion of resources can only be performed by users own permission. One resource belongs to the entire Doris cluster. Users with admin permission can assign permission of resource to other users. Please refer to HELP GRANT or doris document.

    There are three main commands for resource management: create resource, drop resource and show resources. They are to create, delete and check resources. The specific syntax of these three commands can be viewed by executing help CMD after MySQL client connects to Doris.

    1. CREATE RESOURCE

      In the command to create a resource, the user must provide the following information:

      • PROPERTIES related parameters, as follows:
        • type: resource type, required. Currently, only spark and odbc_catalog are supported.
        • For other parameters, see the resource introduction

    2. DROP RESOURCE

      This command can delete an existing resource. For details, please refer to:

    3. SHOW RESOURCES

    Currently, Doris can support

    • Spark resource: do ETL work

    The following shows how the two resources are used.

    Parameter

    Spark Parameters:

    spark.master: required, currently supported yarn, spark://host:port.

    spark.submit.deployMode: The deployment mode of spark. required. It supports cluster and client.

    spark.hadoop.yarn.resourcemanager.address: required when master is yarn.

    spark.hadoop.fs.defaultFS: required when master is yarn.

    Other parameters are optional, refer to: http://spark.apache.org/docs/latest/configuration.html.

    If spark is used for ETL, also need to specify the following parameters:

    working_dir: Directory used by ETL. Spark is required when used as an ETL resource. For example: hdfs://host:port/tmp/doris.

    • broker.property_key: When the broker reads the intermediate file generated by ETL, it needs the specified authentication information.

    Example

    Create a spark resource named in the yarn cluster mode.

    ODBC

    Parameter

    ODBC Parameters:

    type: Required, must be odbc_catalog. As the type identifier of resource.

    user: The user name of the external table, required.

    password: The user password of the external table, required.

    host: The ip address of the external table, required.

    port: The port of the external table, required.

    odbc_type: Indicates the type of external table. Currently, Doris supports MySQL and Oracle. In the future, it may support more databases. The ODBC external table referring to the resource is required. The old MySQL external table referring to the resource is optional.

    : Indicates the driver dynamic library used by the ODBC external table. The ODBC external table referring to the resource is required. The old MySQL external table referring to the resource is optional.

    Example

    Create the ODBC resource of Oracle, named oracle_odbc.