The Proc system is a unique feature of Doris. Students who have used Linux may understand this concept better. In Linux systems, proc is a virtual file system, usually mounted in the /proc directory. Users can view the internal data structure of the system through this file system. For example, you can view the details of the specified pid process through /proc/pid.

    Similar to the proc system in Linux, the proc system in Doris is also organized into a directory-like structure to view different system information according to the “directory path (proc path)” specified by the user.

    The proc system is designed mainly for system administrators, so that it is convenient for them to view some running states inside the system. Such as the tablet status of the table, the cluster balance status, the status of various jobs, and so on. is a very useful function

    There are two ways to view the proc system in Doris.

    1. View through the WEB UI interface provided by Doris, visit the address:
    2. Another way is by command

    After connecting to Doris through the MySQL client, you can execute the SHOW PROC statement to view the information of the specified proc directory. The proc directory is an absolute path starting with “/“.

    The results of the show proc statement are presented in a two-dimensional table. And usually the first column of the result table is the next subdirectory of proc.

    illustrate:

    1. Statistics: It is mainly used to summarize and view the number of databases, tables, partitions, shards, and replicas in the Doris cluster. and the number of unhealthy copies. This information helps us to control the size of the cluster meta-information in general. It helps us view the cluster sharding situation from an overall perspective, and can quickly check the health of the cluster sharding. This further locates problematic data shards.
    2. brokers : View cluster broker node information, equivalent to SHOW BROKER
    3. frontends: Display all FE node information in the cluster, including IP address, role, status, whether it is a mater, etc., equivalent to
    4. routine_loads: Display all routine load job information, including job name, status, etc.
    5. auth: User name and corresponding permission information
    6. jobs:
    7. bdbje: To view the bdbje database list, you need to modify the fe.conf file to add enable_bdbje_debug_mode=true, and then start FE through sh start_fe.sh --daemon to enter the debug mode. After entering debug mode, only http server and MySQLServer will be started and the BDBJE instance will be opened, but no metadata loading and subsequent startup processes will be entered.
    8. dbs: Mainly used to view the metadata information of each database and the tables in the Doris cluster. This information includes table structure, partitions, materialized views, data shards and replicas, and more. Through this directory and its subdirectories, you can clearly display the table metadata in the cluster, and locate some problems such as data skew, replica failure, etc.
    9. resources : View system resources, ordinary accounts can only see resources that they have USAGE_PRIV permission to use. Only the root and admin accounts can see all resources. Equivalent to SHOW RESOURCES
    10. monitor : shows the resource usage of FE JVM
    11. transactions : used to view the transaction details of the specified transaction id, equivalent to
    12. colocation_group : This command can view the existing Group information in the cluster. For details, please refer to the Colocation Join chapter
    13. backends: Displays the node list of BE in the cluster, equivalent to
    14. trash: This statement is used to view the space occupied by garbage data in the backend. Equivalent to SHOW TRASH
    15. cluster_balance : To check the balance of the cluster, please refer to
    16. current_queries : View the list of queries being executed, the SQL statement currently running.
    17. load_error_hub: Doris supports centralized storage of error information generated by load jobs in an error hub. Then view the error message directly through the SHOW LOAD WARNINGS; statement. Shown here is the configuration information of the error hub.
    18. current_backend_instances : Displays a list of be nodes that are currently executing jobs
    19. tasks : Displays the total number of tasks of various jobs and the number of failures.
    20. Cluster_health: Run SHOW PROC '/cluster_health/tablet_health'; statement to view the replica status of the entire cluster.
    21. Current_query_stmts: Returns the currently executing query.
    22. Stream_loads: Returns the stream load task being executed.

    1. For example, “/dbs” displays all databases, and “/dbs/10002” displays all tables under the database with id 10002

      1. mysql> show proc "/dbs/10002";
      2. +---------+----------------------+----------+---------------------+--------------+--------+------+--------------------------+--------------+
      3. | TableId | TableName            | IndexNum | PartitionColumnName | PartitionNum | State  | Type | LastConsistencyCheckTime | ReplicaCount |
      4. +---------+----------------------+----------+---------------------+--------------+--------+------+--------------------------+--------------+
      5. | 10065   | dwd_product_live     | 1        | dt                  | 9            | NORMAL | OLAP | NULL                     | 18           |
      6. | 10109   | ODS_MR_BILL_COSTS_DO | 1        | NULL                | 1            | NORMAL | OLAP | NULL                     | 1            |
      7. | 10124   | test_parquet_import  | 1        | NULL                | 1            | NORMAL | OLAP | NULL                     | 1            |
      8. +---------+----------------------+----------+---------------------+--------------+--------+------+--------------------------+--------------+
      9. 4 rows in set (0.00 sec)
      1. mysql> show proc '/statistic';
      2. +-------+----------------------+----------+--------------+----------+-----------+------------+
      3. | DbId  | DbName               | TableNum | PartitionNum | IndexNum | TabletNum | ReplicaNum |
      5. | 10002 | default_cluster:test | 4        | 12           | 12       | 21        | 21         |
      6. | Total | 1                    | 4        | 12           | 12       | 21        | 21         |
      7. +-------+----------------------+----------+--------------+----------+-----------+------------+
      8. 2 rows in set (0.00 sec)

    3. The following command can view the existing Group information in the cluster.

      • GroupId: The cluster-wide unique identifier of a group, the first half is the db id, and the second half is the group id.
      • GroupName: The full name of the Group.
      • TabletIds: The id list of Tables contained in this Group.
      • BucketsNum: The number of buckets.
      • ReplicationNum: The number of replicas.
      • DistCols: Distribution columns, that is, the bucket column type.
      • IsStable: Whether the Group is stable (for the definition of stability, see the Colocation replica balance and repair section).

    4. Use the following commands to further view the data distribution of a Group:

      1. SHOW PROC '/colocation_group/10005.10008';
      3. +-------------+---------------------+
      4. | BucketIndex | BackendIds          |
      5. +-------------+---------------------+
      6. | 0           | 10004, 10002, 10001 |
      7. | 1           | 10003, 10002, 10004 |
      8. | 2           | 10002, 10004, 10001 |
      9. | 3           | 10003, 10002, 10004 |
      10. | 4           | 10002, 10004, 10003 |
      11. | 5           | 10003, 10002, 10001 |
      12. | 6           | 10003, 10004, 10001 |
      13. | 7           | 10003, 10004, 10002 |
      14. +-------------+---------------------+
      • BucketIndex: The index of the bucket sequence.

    5. Display the total number of tasks of various jobs and the number of failures.

      1. mysql> show proc '/tasks';
      2. +-------------------------+-----------+----------+
      3. | TaskType                | FailedNum | TotalNum |
      4. +-------------------------+-----------+----------+
      5. | CREATE                  | 0         | 0        |
      6. | DROP                    | 0         | 0        |
      8. | CLONE                   | 0         | 0        |
      9. | STORAGE_MEDIUM_MIGRATE  | 0         | 0        |
      10. | ROLLUP                  | 0         | 0        |
      11. | SCHEMA_CHANGE           | 0         | 0        |
      12. | CANCEL_DELETE           | 0         | 0        |
      13. | MAKE_SNAPSHOT           | 0         | 0        |
      14. | RELEASE_SNAPSHOT        | 0         | 0        |
      15. | CHECK_CONSISTENCY       | 0         | 0        |
      16. | UPLOAD                  | 0         | 0        |
      17. | DOWNLOAD                | 0         | 0        |
      18. | CLEAR_REMOTE_FILE       | 0         | 0        |
      19. | MOVE                    | 0         | 0        |
      20. | REALTIME_PUSH           | 0         | 0        |
      21. | PUBLISH_VERSION         | 0         | 0        |
      22. | CLEAR_ALTER_TASK        | 0         | 0        |
      23. | CLEAR_TRANSACTION_TASK  | 0         | 0        |
      24. | RECOVER_TABLET          | 0         | 0        |
      25. | STREAM_LOAD             | 0         | 0        |
      26. | UPDATE_TABLET_META_INFO | 0         | 0        |
      27. | ALTER                   | 0         | 0        |
      28. | INSTALL_PLUGIN          | 0         | 0        |
      29. | UNINSTALL_PLUGIN        | 0         | 0        |
      30. | Total                   | 0         | 0        |
      31. +-------------------------+-----------+----------+
      32. 26 rows in set (0.01 sec)

    6. Display the replica status of the entire cluster.

    1. SHOW, PROC