Swift

    This guide describes how to configure Alluxio with an under storage system supporting the Swift API.

    The Alluxio binaries must be on your machine. You can either , or download the binaries locally.

    Basic Setup

    A Swift bucket can be mounted to the Alluxio either at the root of the namespace, or at a nested directory.

    Configure Alluxio to use under storage systems by modifying . If it does not exist, create the configuration file from the template.

    Modify conf/alluxio-site.properties to include:

    1. alluxio.master.mount.table.root.ufs=swift://<bucket>/<folder>
    2. alluxio.master.mount.table.root.option.fs.swift.user=<swift-user>
    3. alluxio.master.mount.table.root.option.fs.swift.tenant=<swift-tenant>
    4. alluxio.master.mount.table.root.option.fs.swift.password=<swift-user-password>
    5. alluxio.master.mount.table.root.option.fs.swift.auth.url=<swift-auth-url>
    6. alluxio.master.mount.table.root.option.fs.swift.auth.method=<swift-auth-model>

    Replace <bucket>/<folder> with an existing Swift bucket location. Possible values of <swift-use-public> are true, false. Possible values of are keystonev3, keystone, tempauth, swiftauth.

      On the successful authentication, Keystone will return two access URLs: public and private. If Alluxio is used inside company network and Swift is located on the same network it is advised to set value of <swift-use-public> to false.

      An Swift location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio’s Command Line Interface can be used for this purpose.

      Using the Swift module makes and IBM SoftLayer Object Storage as under storage options for Alluxio. To use Ceph, the module must be deployed.

      Running Alluxio Locally with Swift

      Start an Alluxio cluster:

      1. $ ./bin/alluxio format
      2. $ ./bin/alluxio-start.sh local

      This should start an Alluxio master and an Alluxio worker. You can see the master UI at .

      Run a simple example program:

      1. $ ./bin/alluxio runTests

      To stop Alluxio, you can run:

      1. $ ./bin/alluxio-stop.sh local

      The following command can be used to test if the given Swift credentials are valid. Developers can also use it to run functional tests against a Swift endpoint to validate the contract between Alluxio and Swift.

      2.   -Dfs.swift.user=<SWIFT_USER> \
      3.   -Dfs.swift.tenant=<SWIFT_TENANT> \
      4.   -Dfs.swift.password=<SWIFT_PASSWORD> \
      5.   -Dfs.swift.auth.method=<AUTH_METHOD>

      Advanced Setup

      If Alluxio security is enabled, Alluxio enforces the access control inherited from underlying object storage.

      The Swift credentials specified in Alluxio (fs.swift.user, fs.swift.tenant and fs.swift.password) represents a Swift user. Swift service backend checks the user permission to the container. If the given Swift user does not have the right access permission to the specified container, a permission denied error will be thrown. When Alluxio security is enabled, Alluxio loads the container ACL to Alluxio permission on the first time when the metadata is loaded to Alluxio namespace.

      If you want to share the Swift mount point with other users in Alluxio namespace, you can enable alluxio.underfs.object.store.mount.shared.publicly.

      In addition, chown, chgrp, and to Alluxio directories and files do NOT propagate to the underlying Swift buckets nor objects.