ROSS, the Research Object Store System currently is implemented with a Dell EMC Elastic Cloud Storage (ECS) appliance. The ECS system interchangeably supports both the S3 and Swift protocols for object access, as well as the NFS and HDFS protocols for file-system-like access to objects kept in the store. If you really insist, we also have a way to support SMB through a gateway node (good for Windows-based file shares). Although we haven't benchmarked it, we suspect that SMB access would be slower than other styles of access due to the extra hop through the SMB gateway node. The ECS system also supports the EMC ATMOS and CAS protocols (from the EMC Centera world); however, the UAMS HPC management team is not actively supporting those protocols. Users who chose to utilize those protocols are on their own. Please consult the ECS Data Access Guide for details about the object access APIs supported by our object store. In most cases, the ECS Data Access Guide merely outlines the differences between the APIs as implemented in the ECS system and the reference APIs. Hence, a user may also need to consult the upstream API documentation to get more details about the protocols.
...
Using their Grace login information in the form of username@hpc.uams.edu, a user may access the ECS management GUI Portal via a web browser, where the user has limited rights as a Namespace Administrator to manage their personal Namespace (e.g. create or replace access keys, create/delete Buckets, set Bucket attributes, set up NFS or HDFS shares on buckets, etc.). Please note that this GUI allows you to fetch your secret key for accessing S3, or to set your Swift password.
Departments, labs, or other entities may arrange to have shared group Namespaces in the object store. The department, lab, or other entity designates one or more persons to manage that Namespace. These group Namespace Administrators normally would access the ECS management GUI Portal using their UAMS credentials in the form of "username@uams.edu" (i.e. not their HPC credentials). Note that each UAMS user can only be the Namespace Administrator of one shared Namespace (not counting the user's personal Namespace tied to their HPC account on Grace). There is a workaround for this rule, but it requires multiple IDs, one for each Namespace. From the management GUI ECS Portal the Namespace Administrators can either add ACLs to Buckets or Objects to allow users to access data in the Namespace, or can create local Object Users (Object Users who are not tied to any particular domain) within the Namespace., and can configure buckets for object users.
S3 Object Users can choose between We are working with UAMS IT to get more user friendly addressing options. For the moment, we only allow for path-based addressing of the object store , and depend on the HTTPS ports outlined in the ECS Data Access Guide. Only the HTTPS protocols are allowedor virtual host based addressing. Only the HTTPS protocols should be used to maintain security.
In S3 path-based addressing, the bucket name is the first path element in the URL. For example, to access a Bucket bucket named " foo" using the S3 protocol , the URL would be be https://uamsecs1s3.ross.adhpc.uams.edu:9021/foo/. If the bucket is not in the Object User's namespace, the RESTful call must also include x-emc-namespace in the HTTPS header. By default, without the x-emc-namespace key/value pair in the HTTP header, ECS assumes that the Namespace that the bucket foo resides in is the Namespace to which the Object User belongs. Object Users can only belong to one Namespace. A particular person could have multiple Object User IDs if they work in multiple namespaces, and adding the header is inconvenient.
In S3 virtual host based addressing, the namespace and optionally the bucket can be prepended to the base host name of s3.ross.hpc.uams.edu. For example, to access a bucket named foo in a namespace named bar, the URL would be https://foo.bar.s3.ross.hpc.uams.edu or alternatively https://bar.s3.ross.hpc.uams.edu/foo/. With virtual host based addressing, there is no need to include the x-emc-namespace key/value pair in the header, since the Namepace is inferred from the virtual host name.
If using the Swift protocol, the base URL to access the API would be https://swift.ross.hpc.uams.edu/. The Namespace and the Bucket names are included in the path of the Swift API URL, as defined by the Swift API. The Namespace is equivalent to the <account> path component in the API descriptions in the ECS Data Access Guide. The Bucket is equivalent to <container>in the Swift API
The ECS Data Access Guide also outlines alternate ports for reaching the respective APIs. These ports are enabled on the https://ross.hpc.uams.edu. There is no need to prepend the API type (e.g. s3 or swift) to the host name since the port takes care of identifying the API. For example, to access a Bucket named "foo" using the S3 protocol the URL would be https://ross.hpc.uams.edu:9021/foo. When using the alternate ports, only path-based S3 addressing is allowed.
A popular method for accessing the object store from the command line is the s3curl command. Since our object store is based on Dell EMC ECS technology, one should use the Dell EMC ECS variant of s3curl, found at https://github.com/EMCECS/s3curl.
...
There are a lot of other options for accessing the object store, including both free and paid software. UAMS IT has a limited number of licenses for S3 Browser, which is a GUI-based Windows tool for access accessing ROSS. We do not endorse any particular tool. Users may pick the tool that they feel most comfortable with.
An Object User A Namespace Administrator can also set up a Bucket with File Access enabled at Bucket creation time using the ECS Portal. Then, using the ECS management GUIAfter creating the file-enabled Bucket, the Namespace Administrator for the Namespace in which the Bucket resides can set up, for example, an NFS share using that Bucket along with user ID mapping. Users can then mount and access the bucket using the NFS protocol, similar to any other network file share. Enabling a bucket for file access does seem to add a tiny bit of overhead to that bucket, and does disable a few advanced features such as S3 object versioning. And accessing a bucket via NFS is slower than accessing using the object protocols (e.g. S3 or Swift). If an Object User needs file access, they might consider using using rclone mounting instead of the NFS mounting, as it might be faster.
...