Recon API

    Endpoints that are marked as admin only can only be accessed by Kerberos users specified in the ozone.administrators or ozone.recon.administrators configurations of a secure cluster. See Securing Ozone for more information. To restrict access to these endpoints, set the following configurations:

    Parameters

    • prevKey (optional)

      Only returns the containers with ID greater than the given prevKey. Example: prevKey=1

    • limit (optional)

      Only returns the limited number of results. The default limit is 1000.

    Returns

    Returns all the ContainerMetadata objects.

    GET /api/v1/containers/:id/keys

    Parameters

    • prevKey (optional)

      Only returns the keys that are present after the given prevKey key prefix. Example: prevKey=/vol1/bucket1/key1

    • limit (optional)

      Only returns the limited number of results. The default limit is 1000.

    Returns

    Returns all the KeyMetadata objects for the given ContainerID.

    2.       "totalCount":7,
    3.       "keys": [
    4.         {
    5.           "Volume":"vol-1-73141",
    6.           "Bucket":"bucket-3-35816",
    7.           "Key":"key-0-43637",
    8.           "DataSize":1000,
    9.           "Versions":[0],
    10.           "Blocks": {
    11.             "0": [
    12.               {
    13.                 "containerID":1,
    14.                 "localID":105232659753992201
    15.               }
    16.             ]
    17.           },
    18.           "CreationTime":"2020-11-18T18:09:17.722Z",
    19.           "ModificationTime":"2020-11-18T18:09:30.405Z"
    20.         },
    21.         ...
    22.       ]
    23.     }

    GET /api/v1/containers/missing

    Parameters

    No parameters.

    Returns

    Returns the MissingContainerMetadata objects for all the missing containers.

    1.     {
    2.         "totalCount": 26,
    3.         "containers": [{
    4.             "containerID": 1,
    5.             "missingSince": 1605731029145,
    6.             "keys": 7,
    7.             "pipelineID": "88646d32-a1aa-4e1a",
    8.             "replicas": [{
    9.                 "containerId": 1,
    10.                 "datanodeHost": "localhost-1",
    11.                 "firstReportTimestamp": 1605724047057,
    12.                 "lastReportTimestamp": 1605731201301
    13.             }, 
    14.             ...
    15.             ]
    16.         },
    17.         ...
    18.         ]
    19.     }

    GET /api/v1/containers/:id/replicaHistory

    Parameters

    No parameters.

    Returns

    Returns all the ContainerHistory objects for the given ContainerID.

    1.     [
    2.       {
    3.         "containerId": 1,
    4.         "datanodeHost": "localhost-1",
    5.         "firstReportTimestamp": 1605724047057,
    6.         "lastReportTimestamp": 1605730421294
    7.       },
    8.       ...
    9.     ]

    GET /api/v1/containers/unhealthy

    Parameters

    • batchNum (optional)

      The batch number (like “page number”) of results to return. Passing 1, will return records 1 to limit. 2 will return limit + 1 to 2 * limit, etc.

    • limit (optional)

      Only returns the limited number of results. The default limit is 1000.

    Returns

    Returns the UnhealthyContainerMetadata objects for all the unhealthycontainers.

    1.      {
    2.          "missingCount": 2,
    3.          "underReplicatedCount": 0,
    4.          "overReplicatedCount": 0,
    5.          "misReplicatedCount": 0,
    6.          "containers": [{
    7.              "containerID": 1,
    8.              "containerState": "MISSING",
    9.              "unhealthySince": 1605731029145,
    10.              "expectedReplicaCount": 3,
    11.              "actualReplicaCount": 0,
    12.              "reason": null,
    13.              "keys": 7,
    14.              "pipelineID": "88646d32-a1aa-4e1a",
    15.              "replicas": [{
    16.                  "containerId": 1,
    17.                  "datanodeHost": "localhost-1",
    18.                  "firstReportTimestamp": 1605722960125,
    19.                  "lastReportTimestamp": 1605731230509
    20.              }, 
    21.             ...
    22.             ]
    23.          },
    24.         ...
    25.         ]
    26.      }

    Parameters

    • batchNum (optional)

      The batch number (like “page number”) of results to return. Passing 1, will return records 1 to limit. 2 will return limit + 1 to 2 * limit, etc.

    • limit (optional)

      Only returns the limited number of results. The default limit is 1000.

    Returns the UnhealthyContainerMetadata objects for the containers in the given state. Possible unhealthy container states are MISSING, MIS_REPLICATED,UNDER_REPLICATED, OVER_REPLICATED. The response structure is same as /containers/unhealthy.

    Namespace Metadata (admin only)

    GET /api/v1/namespace/summary

    Parameters

    • path

      The path request in string without any protocol prefix.

    Returns

    Returns a basic summary of the path, including entity type and aggregate count of objects under the path.

    status is OK if path exists, PATH_NOT_FOUND otherwise.

    Example: /api/v1/namespace/summary?path=/

    1.     {
    2.       "status": OK,
    3.       "type": ROOT,
    4.       "numVolume": 10,
    5.       "numBucket": 100,
    6.       "numDir": 1000,
    7.       "numKey": 10000
    8.     }

    Example: /api/v1/namespace/summary?path=/volume1

    Example: /api/v1/namespace/summary?path=/volume1/bucket1

    1.     {
    3.       "type": BUCKET,
    4.       "numVolume": -1,
    5.       "numBucket": -1,
    6.       "numDir": 50,
    7.       "numKey": 500
    8.     }

    Example: /api/v1/namespace/summary?path=/volume1/bucket1/dir

    1.     {
    2.       "status": OK,
    3.       "type": DIRECTORY,
    4.       "numVolume": -1,
    5.       "numBucket": -1,
    6.       "numDir": 10,
    7.       "numKey": 100
    8.     }

    Example: /api/v1/namespace/summary?path=/volume1/bucket1/dir/nestedDir

    1.     {
    2.       "status": OK,
    3.       "type": DIRECTORY,
    4.       "numVolume": -1,
    5.       "numBucket": -1,
    6.       "numDir": 5,
    7.       "numKey": 50
    8.     }

    If any num field is -1, the path request is not applicable to such an entity type.

    GET /api/v1/namespace/du

    Parameters

    • path

      The path request in string without any protocol prefix.

    • files (optional)

      A boolean with a default value of false. If set to true, computes disk usage for keys under the path.

    • replica (optional)

      A boolean with a default value of false. If set to true, computes disk usage with replicated size of keys.

    Returns

    Returns disk usage of all sub-paths under the path. Normalizes path fields, returns total size of keys directly under the path as sizeDirectKey, and returns size/sizeWithReplica in number of bytes.

    status is OK if path exists, PATH_NOT_FOUND otherwise.

    Example: /api/v1/namespace/du?path=/vol1/bucket1&files=true&replica=true

    1.     {
    2.       "status": OK,
    3.       "path": "/vol1/bucket1",
    4.       "size": 100000,
    5.       "sizeWithReplica": 300000,
    6.       "subPathCount": 4,
    7.       "subPaths": [
    8.         {
    9.           "path": "/vol1/bucket1/dir1-1",
    10.           "size": 30000,
    11.           "sizeWithReplica": 90000,
    12.           "isKey": false
    13.         },
    14.         {
    15.           "path": "/vol1/bucket1/dir1-2",
    16.           "size": 30000,
    17.           "sizeWithReplica": 90000,
    18.           "isKey": false
    19.         },
    20.         {
    21.           "path": "/vol1/bucket1/dir1-3",
    22.           "size": 30000,
    23.           "sizeWithReplica": 90000,
    24.           "isKey": false
    25.         },
    26.         {
    27.           "path": "/vol1/bucket1/key1-1",
    28.           "size": 10000,
    29.           "sizeWithReplica": 30000,
    30.           "isKey": true
    31.         }
    32.       ],
    33.       "sizeDirectKey": 10000
    34.     }

    If files is set to false, sub-path /vol1/bucket1/key1-1 is omitted. If replica is set to false, sizeWithReplica returns -1. If the path’s entity type cannot have direct keys (Root, Volume), sizeDirectKey returns -1.

    GET /api/v1/namespace/quota

    Parameters

    • path

      The path request in string without any protocol prefix.

    Returns

    Returns the quota allowed and used under the path. Only volumes and buckets have quota. Other types are not applicable to the quota request.

    status is OK if the request is valid, PATH_NOT_FOUND if path doesn’t exist, TYPE_NOT_APPLICABLE if path exists, but the path’s entity type is not applicable to the request.

    Example: /api/v1/namespace/quota?path=/vol

    1.     {
    2.       "status": OK,
    3.       "allowed": 200000,
    4.       "used": 160000

    If quota is not set, allowed returns -1. More on [Quota in Ozone] ()

    GET /api/v1/namespace/dist

    Parameters

    • path

    Returns

    Returns the file size distribution of all keys under the path.

    status is OK if the request is valid, PATH_NOT_FOUND if path doesn’t exist, TYPE_NOT_APPLICABLE if path exists, but the path is a key, which does not have a file size distribution.

    Example: /api/v1/namespace/dist?path=/

    Recon keeps track of all keys with size from 1 KB to . For keys smaller than 1 KB, map to the first bin (index); for keys larger than 1 PB, map to the last bin (index).

    Each index of dist is mapped to a file size range (e.g. 1 MB - 2 MB).

    Parameters

    No parameters.

    Returns

    Returns a summary of the current state of the Ozone cluster.

    1.      {
    2.          "pipelines": 5,
    3.          "totalDatanodes": 4,
    4.          "healthyDatanodes": 4,
    5.          "storageReport": {
    6.              "capacity": 1081719668736,
    7.              "used": 1309212672,
    8.              "remaining": 597361258496
    9.          },
    10.          "containers": 26,
    11.          "volumes": 6,
    12.          "buckets": 26,
    13.          "keys": 25
    14.      }

    Datanodes

    GET /api/v1/datanodes

    Parameters

    No parameters.

    Returns

    Returns all the datanodes in the cluster.

    1.      {
    2.          "totalCount": 4,
    3.          "datanodes": [{
    4.              "uuid": "f8f8cb45-3ab2-4123",
    5.              "hostname": "localhost-1",
    6.              "state": "HEALTHY",
    7.              "lastHeartbeat": 1605738400544,
    8.              "storageReport": {
    9.                  "capacity": 270429917184,
    10.                  "used": 358805504,
    11.                  "remaining": 119648149504
    12.              },
    13.              "pipelines": [{
    14.                  "pipelineID": "b9415b20-b9bd-4225",
    15.                  "replicationType": "RATIS",
    16.                  "replicationFactor": 3,
    17.                  "leaderNode": "localhost-2"
    18.              }, {
    19.                  "pipelineID": "3bf4a9e9-69cc-4d20",
    20.                  "replicationType": "RATIS",
    21.                  "replicationFactor": 1,
    22.                  "leaderNode": "localhost-1"
    23.              }],
    24.              "containers": 17,
    25.              "leaderCount": 1
    26.          },
    27.         ...
    28.         ]
    29.      }

    GET /api/v1/pipelines

    Parameters

    No parameters.

    Returns

    Returns all the pipelines in the cluster.

    1.      {
    2.          "totalCount": 5,
    3.          "pipelines": [{
    4.              "pipelineId": "b9415b20-b9bd-4225",
    5.              "status": "OPEN",
    6.              "leaderNode": "localhost-1",
    7.              "datanodes": ["localhost-1", "localhost-2", "localhost-3"],
    8.              "lastLeaderElection": 0,
    9.              "duration": 23166128,
    10.              "leaderElections": 0,
    11.              "replicationType": "RATIS",
    12.              "replicationFactor": 3,
    13.              "containers": 0
    14.          },
    15.         ...
    16.         ]
    17.      }

    Tasks

    GET /api/v1/task/status

    Parameters

    No parameters.

    Returns

    Returns the status of all the Recon tasks.

    1.      [
    2.        {
    3.          "taskName": "OmDeltaRequest",
    4.          "lastUpdatedTimestamp": 1605724099147,
    5.          "lastUpdatedSeqNumber": 186
    6.        },
    7.        ...
    8.      ]

    GET /api/v1/utilization/fileCount

    Parameters

    • volume (optional)

      Filters the results based on the given volume name.

    • bucket (optional)

      Filters the results based on the given bucket name.

    • fileSize (optional)

      Filters the results based on the given fileSize.

    Returns

    Returns the file counts within different file ranges with fileSize in the response object being the upper cap for file size range.

    1.      [{
    2.          "volume": "vol-2-04168",
    3.          "bucket": "bucket-0-11685",
    4.          "fileSize": 1024,
    5.          "count": 1
    6.      }, {
    7.          "volume": "vol-2-04168",
    8.          "bucket": "bucket-1-41795",
    9.          "fileSize": 1024,
    10.          "count": 1
    11.      }, {
    12.          "volume": "vol-2-04168",
    13.          "bucket": "bucket-2-93377",
    14.          "fileSize": 1024,
    15.          "count": 1
    16.      }, {
    17.          "volume": "vol-2-04168",
    18.          "bucket": "bucket-3-50336",
    19.          "fileSize": 1024,
    20.          "count": 2

    Metrics

    Parameters

    Refer to Prometheus HTTP API Reference for complete documentation on querying.

    Returns

    This is a proxy endpoint for Prometheus and returns the same response as the prometheus endpoint. Example: /api/v1/metrics/query?query=ratis_leader_election_electionCount