Data Recovery

    This feature requires Doris version 0.8.2+

    To use this function, you need to deploy the broker corresponding to the remote storage. Such as BOS, HDFS, etc. You can view the currently deployed broker through .

    The restore operation needs to specify an existing backup in the remote warehouse, and then restore the content of the backup to the local cluster. When the user submits the Restore request, the system will perform the following operations:

    1. Create the corresponding metadata locally

      This step will first create and restore the corresponding table partition and other structures in the local cluster. After creation, the table is visible, but not accessible.

    2. Local snapshot

      This step is to take a snapshot of the table created in the previous step. This is actually an empty snapshot (because the table just created has no data), and its purpose is to generate the corresponding snapshot directory on the Backend for later receiving the snapshot file downloaded from the remote warehouse.

    3. Download snapshot

      The snapshot files in the remote warehouse will be downloaded to the corresponding snapshot directory generated in the previous step. This step is done concurrently by each Backend.

    4. After the snapshot download is complete, we need to map each snapshot to the metadata of the current local table. These snapshots are then reloaded to take effect, completing the final recovery job.

    1. Restore the table backup_tbl in backup snapshot_1 from example_repo to database example_db1, the time version is “2018-05-04-16-45-08”. Revert to 1 copy:

    2. Restore partitions p1 and p2 of table backup_tbl in backup snapshot_2 from example_repo, and table backup_tbl2 to database example_db1, and rename it to new_tbl with time version “2018-05-04-17-11-01”. The default reverts to 3 replicas:

    3. View the execution of the restore job:

    For detailed usage of RESTORE, please refer to here.

    The commands related to the backup and restore function are as follows. For the following commands, you can use help cmd; to view detailed help after connecting to Doris through mysql-client.

    1. CREATE REPOSITORY

      Create a remote repository path for backup or restore. This command needs to use the Broker process to access the remote storage. Different brokers need to provide different parameters. For details, please refer to Broker documentation, or you can directly back up to support through the S3 protocol For the remote storage of AWS S3 protocol, directly back up to HDFS, please refer to

    2. RESTORE

    3. CANCEL RESTORE

      Cancel the currently executing restore job.

    4. DROP REPOSITORY

      Delete the created remote repository. Deleting a warehouse only deletes the mapping of the warehouse in Doris, and does not delete the actual warehouse data.

    1. Restore Report An Error：[20181: invalid md5 of downloaded file: /data/doris.HDD/snapshot/20220607095111.862.86400/19962/668322732/19962.hdr, expected: f05b63cca5533ea0466f62a9897289b5, get: d41d8cd98f00b204e9800998ecf8427e]

      If the number of copies of the table backed up and restored is inconsistent, you need to specify the number of copies when executing the restore command. For specific commands, please refer to command manual

    2. Restore Report An Error：[COMMON_ERROR, msg: Could not set meta version to 97 since it is lower than minimum required version 100]

    For more detailed syntax and best practices used by RESTORE, please refer to the command manual, You can also type HELP RESTORE on the MySql client command line for more help.