UAMS runs a object storage system for archiving research data, based on the Dell EMC ECS (Elastic Cloud Storage) system. The ECS system is now available for use, and is jointly managed by the HPC administration team and UAMS IT (i.e. UAMS IT runs the data center infrastructure in which it sits and the networking infrastructure needed to access it, while the HPC team manages the ECS access itself). Click here for technical info about the object store.
- Archival storage may be purchased for $70 per TB, with an expected lifespan of 5 years. There is no free storage available. Note that this cost is considerably less expensive than commercial-cloud-based archival storage (e.g. Amazon Glacial storage).
- The $70/TB charge is for the amount of storage reserved (i.e. the quota limit), and is not based on actual use.
- Unlike some cloud providers, UAMS does not impose additional data transfer (access, egress or networking) fees for accessing the archival storage.
- Please coordinate Robin at DBMI to arrange an IDT to the storage core to pay for storage quota requests.
- Once you’ve picked the size of your storage pool and have arranged the financial details with Robin, one of the administrators will set up a namespace and quotas for you.
- As part of this setup, you will designate one or more namespace administrators who would manage users of and set permissions for your namespace.
- Objects in the archive are stored in buckets that belong to namespaces. Your namespace administrator may create as many buckets as desired within your namespace. Note that it is recommended that a namespace have no more than 1000 buckets.
- A bucket has a native format of either S3 or Swift, your choice. However either object storage API (S3 or Swift) can be used to access a bucket, subject to the limitations of the native format (cross-head support).
- On bucket creation, your namespace administrator can also configure a bucket for file access (in addition to object access) using either the NFS or HDFS protocol. Changing the file access option after bucket creation currently requires re-creating the bucket and copying data from old to new bucket. Note that you do not lose object access by enabling file access, but enabling file access on a bucket may have some minor impacts on object access.
- The ECS system also offers EMC-proprietary bucket formats (Atmos or CAS) which we are not actively supporting, and which do not offer cross-head support (i.e. you can’t access them with other protocols like S3 or Swift). They are there for compatibility purposes for older systems/software.
- It is also possible to enable CIFS, or SMB access, (often used for Windows shares) to a bucket set up for file access. But since CIFS/SMB access goes through a secondary server, performance likely suffers, so we are not recommending it for heavy use.
- Once your administrator sets up the bucket, object users designated by your namespace adiministrator may use the object APIs, and if you enabled file access, mount the buckets to any system inside the UAMS firewall.
- It currently is not possible to access the EMC ECS system from outside the UAMS firewall.
- Grace, the HPC, has data movers that can assist in staging data to/from Grace’s cluster storage for running HPC jobs. If you plan on using this feature, please discuss with HPC staff to get it set up for you.
- Although the EMC ECS storage pools have 12+4 error coding (i.e. redundantly stores data) to protect data against failures, there currently is no offsite backup.
- Users who need offsite backup could, for example, send backup copies to Amazon Glacier or similar systems, with the hope of never having to ever retrieve them except in dire circumstances. However, the users are responsible for the off site backup costs.
- We are looking into an option that would allow researchers to send copies of their archival data to the NFS-sponsored OURRStore project, a write once, read seldom research archive at the University of Oklahoma in Norman. If this pans out, users would only be charged for the LTO-7/8 tapes (at least 2, preferably 3). Tapes with 9 TB capacity currently run $60 to $80 each (i.e. $120 to $240 for a set of 2-3, or about $13-26 per TB unformatted). However, OURRStore will not be in production mode until late 2019, according to the current schedule.
- Users who need automatic offsite backups or better file performance can still request space on the Research NAS that UAMS IT manages (i.e. the EMC ECS system is not the only game in town).