S3 Protocol

    S3 buckets are stored under the volume.

    S3 Gateway is a separated component which provides the S3 compatible APIs. It should be started additional to the regular Ozone components.

    You can start a docker based cluster, including the S3 gateway from the release package.

    Go to the compose/ozone directory, and start the server:

    You can access the S3 gateway at http://localhost:9878

    URL Schema

    Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. ) and the path-style addresses (eg. http://host:9878/bucketname)

    By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your ozone-site.xml:

    1.    <name>ozone.s3g.domain.name</name>
    2.    <value>s3g.internal</value>
    3. </property>

    For example the content of the ‘testbucket’ could be checked from the browser using the URL

    Implemented REST endpoints

    Operations on S3Gateway service:

    Operations on Bucket:

    Operation on Objects:

    If security is not enabled, you can use any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

    If security is enabled, you can get the key and the secret with the ozone s3 getsecret command (*kerberos based authentication is required).

    1. kinit -kt /etc/security/keytabs/testuser.keytab testuser/scm@EXAMPLE.COM
    2. ozone s3 getsecret
    4. awsSecret=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999

    Now, you can use the key and the secret to access the S3 endpoint:

    1. ozone s3 revokesecret
    2. Enter 'y' to confirm S3 secret revocation for 'testuser/scm@EXAMPLE.COM': y

    Ozone Manager administrators can run ozone s3 getsecret and ozone s3 revokesecret command with -u parameter to specify another users.

    1. # Obtained Kerberos TGT for testuser/scm@EXAMPLE.COM with kinit,
    2. # testuser/scm@EXAMPLE.COM is an OM admin.
    3. ozone s3 getsecret -u om/om@EXAMPLE.COM
    5. awsSecret=1e9379d0424cce6669b1a501ff14834e46dee004ee868b41a313b49eabcfb68f
    7. ozone s3 revokesecret -u om/om@EXAMPLE.COM -y
    8. S3 secret revoked.

    Expose any volume

    Ozone has one more element in the name-space hierarchy compared to S3: the volumes. By default, all the buckets of the /s3v volume can be accessed with S3 interface but only the (Ozone) buckets of the /s3v volumes are exposed.

    To make any other buckets available with the S3 interface a “symbolic linked” bucket can be created:

    This example expose the /vol1/bucket1 Ozone bucket as an S3 compatible common-bucket via the S3 interface.

    (Note: the implementation details of the bucket-linking feature can be found in the design doc)

    aws CLI could be used by specifying the custom REST endpoint.

    1. aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest

    Or

      S3 Fuse driver (goofys)

      Next >>