Admin Command Line Interface

    Alluxio’s admin command line interface provides admins with operations to manage the Alluxio filesystem. You can invoke the following command line utility to get all the subcommands:

    The command backs up all Alluxio metadata to the backup directory configured on the leader master.

    Back up to the default backup folder /alluxio_backups of the root under storage system. This default backup directory can be configured by setting alluxio.master.backup.directory.

    1. ./bin/alluxio fsadmin backup
    2. Backup Host        : masters-1                          
    3. Backup URI         : hdfs://masters-1:9000/alluxio_backups/alluxio-backup-2020-10-13-1602619110769.gz
    4. Backup Entry Count : 4

    Note that the user running the backup command need to have write permission to the backup folder of root under storage system.

    Back up to a specific directory in the root under storage system.

    1. ./bin/alluxio fsadmin backup /alluxio/special_backups
    2. Backup Host        : masters-1                          
    3. Backup URI         : hdfs://masters-1:9000/alluxio/special_backups/alluxio-backup-2020-10-13-1602619216699.gz
    4. Backup Entry Count : 4

    Back up to a specific directory on the leading master’s local filesystem.

    1. ./bin/alluxio fsadmin backup /opt/alluxio/backups/ --local
    2. Backup Host        : AlluxioSandboxEJSC-masters-1                          
    3. Backup URI         : file:///opt/alluxio/backups/alluxio-backup-2020-10-13-1602619298086.gz
    4. Backup Entry Count : 4```
    6. ### journal
    7. The `journal` command provides several sub-commands for journal management.
    9. **quorum:** is used to query and manage embedded journal powered leader election.
    11. ```console
    12. # Get information on existing state of the `MASTER` or `JOB_MASTER` leader election quorum.
    13. $ ./bin/alluxio fsadmin journal quorum info -domain <MASTER | JOB_MASTER>
    1. # Remove a member from leader election quorum.

    checkpoint: is used to create a checkpoint in the primary master journal system.

    This command is mainly used for debugging and to avoid master journal logs from growing unbounded.

    Checkpointing requires a pause in master metadata changes, so use this command sparingly to avoid interfering with other users of the system.

    1. $ ./bin/alluxio fsadmin journal checkpoint

    doctor

    The doctor command gives recommendations and warnings. It can diagnose inconsistent configurations across different Alluxio nodes as well as alert the operator when worker storage volumes are missing.

    1. # shows server-side configuration errors and warnings
    2. $ ./bin/alluxio fsadmin doctor configuration
    3. # shows worker storage health errors and warnings
    4. $ ./bin/alluxio fsadmin doctor storage

    The getBlockInfo command provides the block information and file path of a block id. It is primarily intended to assist power users in debugging their system.

    metrics

    The metrics command provides operations for Alluxio metrics system.

    If --master option is used, all the metrics stored in Alluxio leading master will be cleared. If --workers <WORKER_HOSTNAME_1>,<WORKER_HOSTNAME_2> is used, metrics in specific workers will be cleared.

    If you are clearing metrics on a large Alluxio cluster with many workers, you can use the --parallelism <#> option to choose the # of workers to clear in parallel. For example, if your cluster has 200 workers, running with a parallelism factor of 10 will clear execute the command on 10 workers at a time until all metrics are cleared.

    1. # Clear metrics of the whole alluxio cluster including leading master and workers
    2. $ ./bin/alluxio fsadmin metrics clear
    3. # Clear metrics of alluxio leading master
    4. $ ./bin/alluxio fsadmin metrics clear --master
    5. # Clear metrics of specific workers
    6. $ ./bin/alluxio fsadmin metrics clear --workers <WORKER_HOSTNAME_1>,<WORKER_HOSTNAME_2>
    7. # Clear metrics of an alluxio cluster with many workers in parallel
    8. $ ./bin/alluxio fsadmin metrics clear --parallelism 10

    The command manages path defaults.

    list

    pathConf list lists paths configured with path defaults.

    1. $ ./bin/alluxio fsadmin pathConf list
    3. /a
    4. /b

    The above command shows that there are path defaults set for paths with prefix /a and /b.

    show

    pathConf show shows path defaults for a specific path.

    It has two modes:

    1. without option --all, only show path defaults set for the specific path;
    2. with option --all, show path defaults set for all paths that are prefixes of the specified path.

    For example, suppose path defaults property1=value1 is set for /a, and property2=value2 is set for /a/b.

    Then without --all, only properties for /a/b are shown:

    1. $ ./bin/alluxio fsadmin pathConf show /a/b
    3. property2=value2

    With --all, since /a is a prefix of /a/b, properties for both /a and /a/b are shown:

    1. $ ./bin/alluxio fsadmin pathConf show --all /a/b
    3. property1=value1
    4. property2=value2

    add

    pathConf add adds or updates path defaults, only properties with scope equal to or broader than the client scope can be set as path defaults.

    1. $ ./bin/alluxio fsadmin pathConf add --property property1=value1 --property property2=value2 /tmp

    The above command adds two properties as path defaults for paths with prefix /tmp.

    1. $ ./bin/alluxio fsadmin pathConf add --property property1=value2 /tmp

    remove

    pathConf remove removes properties from path defaults for a path.

    The above command removes properties with key property1 and property2 from path defaults for paths with prefix /tmp.

    report

    The report command provides Alluxio running cluster information.

    If no argument is passed in, report will report the leading master, worker number, and capacity information.

    1. $ ./bin/alluxio fsadmin report
    2. Alluxio cluster summary:
    3.     Master Address: localhost:19998
    4.     Zookeeper Enabled: false
    5.     Live Workers: 1
    7.     Total Capacity: 10.45GB
    8.     (only a subset of the results is shown)

    report capacity will report Alluxio cluster capacity information for different subsets of workers:

    • -live Live workers
    • -lost Lost workers
    • -workers <worker_names> Specified workers, host names or ip addresses separated by ,.
    1. # Capacity information of all workers
    2. $ ./bin/alluxio fsadmin report capacity
    3. # Capacity information of live workers
    4. $ ./bin/alluxio fsadmin report capacity -live
    5. # Capacity information of specified workers
    6. $ ./bin/alluxio fsadmin report capacity -workers AlluxioWorker1,127.0.0.1

    report metrics will report the metrics stored in the leading master which includes leading master process metrics and aggregated cluster metrics.

    1. $ ./bin/alluxio fsadmin report metrics

    report ufs will report all the mounted under storage system information of Alluxio cluster.

    1. $ ./bin/alluxio fsadmin report ufs
    2. Alluxio under storage system information:
    3. hdfs://localhost:9000/ on / (hdfs, capacity=-1B, used=-1B, not read-only, not shared, properties={})

    report jobservice will report a summary of the job service.

    1. $ bin/alluxio fsadmin report jobservice
    2. Worker: MigrationTest-workers-2  Task Pool Size: 10     Unfinished Tasks: 1303   Active Tasks: 10     Load Avg: 1.08, 0.64, 0.27
    3. Worker: MigrationTest-workers-3  Task Pool Size: 10     Unfinished Tasks: 1766   Active Tasks: 10     Load Avg: 1.02, 0.48, 0.21
    4. Worker: MigrationTest-workers-1  Task Pool Size: 10     Unfinished Tasks: 1808   Active Tasks: 10     Load Avg: 0.73, 0.5, 0.23
    6. Status: CREATED   Count: 4877
    7. Status: CANCELED  Count: 0
    8. Status: FAILED    Count: 1
    9. Status: RUNNING   Count: 0
    10. Status: COMPLETED Count: 8124
    12. 10 Most Recently Modified Jobs:
    13. Timestamp: 10-28-2020 22:02:34:001       Id: 1603922371976       Name: Persist             Status: COMPLETED
    14. Timestamp: 10-28-2020 22:02:34:001       Id: 1603922371982       Name: Persist             Status: COMPLETED
    15. (only a subset of the results is shown)
    17. 10 Most Recently Failed Jobs:
    18. Timestamp: 10-24-2019 17:15:22:946       Id: 1603922372008       Name: Persist             Status: FAILED
    20. 10 Longest Running Jobs:

    The statelock command provides information about the waiters and holders of the alluxio statelock. This command can help diagnose any long running requests issued by users or the Alluxio system.

    ufs

    The ufs command provides options to update attributes of a mounted under storage. The option mode can be used to put an under storage in maintenance mode. Certain operations can be restricted at this moment.

    For example, an under storage can enter readOnly mode to disallow write operations. Alluxio will not attempt any write operations on the under storage.

    1. $ ./bin/alluxio fsadmin ufs --mode readOnly hdfs://ns

    The fsadmin ufs subcommand takes a UFS URI as an argument. The argument should be a root UFS URI like , and not hdfs://<name-service>/<folder>.