Storage Container Manager

    Storage container manager provides multiple critical functions for the Ozone cluster. SCM acts as the cluster manager, Certificate authority, Block manager and the Replica manager.

    1. SCM is the block manager. SCM allocates blocks and assigns them to data nodes. Clients read and write these blocks directly.

    2. SCM keeps track of all the block replicas. If there is a loss of data node or a disk, SCM detects it and instructs data nodes make copies of the missing blocks to ensure high availability.

    For a detailed view of Storage Container Manager this section gives a quick overview about the provided network services and the stored persisted data.

    • Pipelines: List/Delete/Activate/Deactivate
      • Raft groups are planned by SCM
    • Containers: Create / List / Delete containers
    • Admin related requests
    • Safemode status/modification
    • Replication manager start / stop
    • CA authority service
    • Required by other sever components
    • Datanode HeartBeat protocol
      • From Datanode to SCM (30 sec by default)
      • SCM can add commands to the response

    Note: client doesn’t connect directly to the SCM

    Persisted state

    • Pipelines
      • Replication group of servers. Maintained to find a group for new container/block allocations.
    • Containers
      • Containers are the replication units. Data is required to act in case of data under/over replicated.
    • Deleted blocks
      • Block data is deleted in the background. Need a list to follow the progress.
    • Valid cert, Revoked certs
    • Used by the internal Certificate Authority to authorize other Ozone services