The Center for High Performance Computing includes an object store as a low-cost option for archiving data. Although there is a fee for using the object store, the cost of the storage is heavily subsidized by the university, according to current plans. Each HPC user can request private space on the object store for personal use, for example to back up their home directory on Grace's cluster storage, or to hold research data sets when they are not being actively used. In addition, labs, departments, or other entities can request shared space that can be utilized by all of their users.
The object store currently is implemented with a Dell EMC Elastic Cloud Storage (ECS) appliance. The ECS system interchangeably supports both the S3 and Swift protocols for object access, as well as the NFS and HDFS protocols for file-system-like access to objects kept in the store. If you really insist, we also have a way to support SMB through a gateway node (good for Windows-based file shares). Although we haven't benchmarked it, we suspect that SMB access would be slower than other styles of access due to the extra hop through the SMB gateway node. The ECS system also supports the EMC ATMOS and CAS (from the EMC Centera world) protocols; however, the UAMS HPC management team is not actively supporting those protocols. Users who chose to utilize those protocols are on their own. Please consult the ECS Data Access Guide for details about the object access APIs supported by our object store. In most cases, the ECS Data Access Guide merely outlines the differences between the APIs as implemented in the ECS system and the reference APIs. Hence, a user may also need to consult the upstream API documentation to get more details about the protocols.
Some grasp of ECS terminology and concepts is useful for understanding how to interact with the ECS system. The ECS system divides the storage into Namepspaces. An Object User is assigned to a particular Namespace, which is the default Namespace used when no specific Namespace is identified in a RESTful API call. An Object User cannot be assigned to multiple Namespaces. A person who needs to be an Object User in multiple Namespaces with be given multiple Object User IDs, one for each Namespace. Each Namespace holds a set of Buckets, which in turn each hold a set of Objects. An Object User may create as many buckets with whatever characteristics as they like within the Namespace to which the Object User belongs. Each Bucket in a namespace belongs to a Bucket Owner in that Namespace, which by default is the Object User who created that Bucket. The Bucket Owner can set an Access Control List (ACL) on their Buckets that dictate which other Object Users may search or modify that Bucket, including who can create, read, update, or delete Objects held within that Bucket. The Bucket Owner can also set ACLs for each individual object, as desired, to allow other Object Users to access those objects.
It is possible for Object Users that belong to one Namespace to access Buckets and Objects in other Namespaces. To do so, the Object User must have been given permission (e.g. through an ACL attached to the Object or the Bucket in which the Object resides) by the owner of the Object. The Object User must also identify the other namespace in the RESTful API call, for example using the x-emc-namespace element in the HTTP header of the RESTful call.
For the object store at UAMS, each HPC user who requests personal space in the object store is given their own Namespace tied to their account on Grace. When referencing their account on Grace the HPC user must append "@hpc.uams.edu" to their Grace username. In other words, the name of HPC user's personal Namespace in the object store is their HPC username (the same as their home directory name on Grace), and the sole Object User within that namespace is identified as username@hpc.uams.edu.
Using their Grace login information in the form of username@hpc.uams.edu, a user may access the ECS management GUI via a web browser, where the user has limited rights to manage their personal Namespace (e.g. create or replace access keys, create/delete Buckets, set Bucket attributes, set up NFS or HDFS shares on buckets, etc.). Please note that this GUI allows you to fetch your secret key for accessing S3, or to set your Swift password.
Departments, labs, or other entities may arrange to have shared Namespaces in the object store. The department, lab, or other entity designates one or more persons to manage that Namespace. These departmental Namespace Administrators would access the ECS management GUI using their UAMS credentials in the form of "username@uams.edu" (i.e. not their HPC credentials). Note that each UAMS user can only be the Namespace Administrator of one shared Namespace (not counting the user's personal Namespace tied to their HPC account on Grace). There is a workaround for this rule, but it requires multiple IDs, one for each Namespace. From the management GUI the Namespace Administrators can either add ACLs to Buckets or Objects to allow users to access data in the Namespace, or can create local Object Users (Object Users who are not tied to any particular domain) within the Namespace.
We are working with UAMS IT to get more user friendly addressing options. For the moment, we only allow for path-based addressing of the object store, and depend on the HTTPS ports outlined in the ECS Data Access Guide. Only the HTTPS protocols are allowed. For example, to access a Bucket named "foo" using the S3 protocol the URL would be https://uamsecs1.ad.uams.edu:9021/foo. If the bucket is not in the Object User's namespace, the RESTful call must also include x-emc-namespace in the HTTPS header.
A popular method for accessing the object store from the command line is the s3curl command. Since our object store is based on Dell EMC ECS technology, one should use the Dell EMC ECS variant of s3curl, found at https://github.com/EMCECS/s3curl.
There are a lot of other options for accessing the object store, including both free and paid software. We do not endorse any particular tool. An Object User can also set up a Bucket with File Access enabled at Bucket creation time. Then, using the ECS management GUI the Namespace Administrator for the Namespace in which the Bucket resides can set up, for example, an NFS share using that Bucket along with user ID mapping. Users can then mount and access the bucket using the NFS protocol. We have not benchmarked access to the object store via NFS, but suspect that it will be slower than access using the object protocols (e.g. S3 or Swift). If a user desires high speed NFS storage, UAMS IT offers at cost a research NAS that the researcher may use. Please contact UAMS IT directly if you would like Research NAS storage.