Types of Storage
We have three file systems available to NICS users: home directories, system global temporary directories, and HPSS directories.
Home Directories
Home directories for each user are mounted on NICS systems via the Network File Service. This is online disk space intended for permanent, frequently used storage and is backed up on a daily basis. This file system does not generally provide the input/output (I/O) performance required by most jobs. The system global area is recommended for all I/O-intensive work.
System Global Temporary Directories (Work Directories)
Separate system global temporary directories are provided on each system. On Kraken, this is the Lustre file system. This is a large, fast disk area intended for global access to temporary storage, for example, to hold output generated by a user’s job. It is currently not backed up, and old files are automatically deleted every night. Any file older than 14 days is subject to deletion. Our intention is to maintain these file systems so there is always a large amount of scratch space available for executing jobs. Users should move files that are not actively being used to HPSS.
HPSS Directories
HPSS is the archival storage system. It consists of 12 TB of disk cache and approximately 620 TB of user data stored in robotic tape storage libraries. HPSS may be accessed by HSI (hsi).
Projects should not duplicate executables or data sets in each user’s area but instead should set privileges to share a master copy. Contact the User Assistance Center for help in setting privileges to facilitate sharing. Files not in immediate use, such as source code and output data sets, should be moved to HPSS.
Space Allocations
Home Directories
A research account is entitled to 2GB in its home directory. For projects with special requirements, we will consider extensions of this limit on a case-by-case basis.
System Global Temporary Directories (Work Directories)
Data in a system global file system should be present if needed or produced by a running, queued, or recently completed job. Files should not be retained in this file system and should be migrated to HPSS as soon as the files are not actively being used. Because of the working-directory character of the system global file system, files are automatically purged on a regular basis. Files older than
14 days are eligible for purging.
HPSS Directories
Space on HPSS is for files that are not immediately needed. We have not yet set a space limit for HPSS usage. Even though HPSS is a very large storage system, space is not unlimited. Users must not store files unrelated to their NICS projects on HPSS. They must also periodically review their files and remove unneeded ones.
General
If a file system runs out of or gets low on space, user data on that file system will be subject to involuntary deletion without prior notice.
Consequences of Abuse
Storage usage will periodically be monitored. If time permits, first offenders will be warned to clean up their space. Ignoring these warnings will result in loss of access privileges.
