Two filesystems are available to ACF users for storing user files: the Network File System (NFS) and Lustre Haven. NFS contains and is used for home directories and Lustre Haven contains and is used for project and scratch directories. Table 1.1 summarizes the available filesystems.
|NFS Home Directory||/nics/[a,b,c,d]/home/<username>||10GB||Not Purged|
|Lustre Scratch Directory||/lustre/haven/user/<username>||No Quota||Purged|
|Lustre Project Directory||/lustre/haven/proj/<project>||1 TB default
more by request
Please note that while both NFS and Lustre are reliable filesystems, errors and corruptions can still occur. It is each user's responsibility to back up your data. To learn about data transfer to/from the ACF, please review the Data Transfer document.
For more information on the Lustre file system, please refer to the Lustre User Guide.
On the ACF, fifteen terabytes of storage space is available via NFS for home directories. Home directories on NFS are periodically backed up for disaster recovery. Each new account on the ACF receives a home directory on NFS. This is each account's personal storage location for a small amount of files. Here you can store job scripts, virtual environments, and other types of files and data. In a Linux environment, you can refer to your home directory with the environment variable $HOME or with the tilde (~) character.
By default, your home directory is limited to 10GB of storage space. It is not intended to store large amounts of project or job-related data. For job-related data, please use your scratch directory. For project data that you do not want to be purged request and use project space.
To determine how much storage space you have consumed in your home directory, execute the
quota -s command. Figure 2.1 shows the possible output of this command. Of interest are the first “space,” “quota,” and “limit” fields. The “space” field shows how much storage space is currently in use. The “quota” field displays the soft quota placed on your home directory. The “limit” field defines the hard quota, which is the absolute maximum storage space you can consume. When you exceed the quota, you will start a grace period that gives you time to reduce your storage space usage. If you do not reduce your storage space usage during this period, the soft limit defined by the “quota” field will be enforced.
All users are initially put into one of the ACF institutional projects: ACF-UTK0011 for UTK users and ACF-UTHSC0001 for UTHSC users. These projects do not have a project space. All courses and research projects requested by a University faculty are created with a default 1 terabyte project space. For graduate student Master's Thesis or Dissertations it is suggested that a project should be requested with the graduate student's advisor as the project principal investigator to obtain a project directory. If a 1 terabyte project space is not sufficient please submit a project directory request on the portal to request more project storage space. Specify all the information requested on the form in order to process the request. Lustre project directories will be accessible under the /lustre/haven/courses and /lustre/haven/proj/ paths.
Lustre file systems are protected by RAID and are generally reliable. However, if requested Lustre project directories can be backed up for a fee. Submit a service request if you want this service.
Quotas on Lustre project directories will be set to the total storage space requested (default 1 terabyte) by the PI of the project.
Note: Lustre project directories are NOT purged.
Scratch directories are available for all users on the Lustre Haven filesystem. Approximately 2.7 petabytes of Lustre storage space is available on /lustre/haven which is shared with scratch directories and project directories.
Important: Lustre scratch directories are NOT backed up.
Important Purging Notice: Lustre Haven scratch space is purged monthly on approximately the 3rd Monday of each month. Files in Lustre Haven scratch directories are deleted by the purging process if they have not been accessed or modified within 180 days. Over the course of 2020, the purging timeperiod will be gradually reduced to 90 days. See the purging schedule below. In general, users have many temporary files that are no longer needed once a job completes and results are returned. Many times these files and other orphaned and unneeded files are not deleted by end users and they accumulate in scratch directories and can fill the file system which is detrimental to all users. Purging email notices are sent to all active users at least once at the beginning of each month to remind users that purging of scratch space will take place on the 3rd Monday of each month. This notice explains the purging process and how to request a temporary purge exemption or the process to request a project space (project directories are exempt from purging). To request a temporary purge exemption submit a service request with "temporary purge exemption request" in the subject. Project requests along with their corresponding project directory can be made at the portal project request page.
|Purge Estimated Date||Path||Files Purge if not accessed or modified in|
|Jul 20, 2020||/lustre/haven/user/<username>||<180 days|
|Aug 17, 2020||/lustre/haven/user/<username>||<180 days|
|Sep 21, 2020||/lustre/haven/user/<username>||<180 days|
|Oct 19, 2020||/lustre/haven/user/<username>||<180 days|
|Nov 16, 2020||/lustre/haven/user/<username>||<180 days|
|Dec 21, 2020||/lustre/haven/user/<username>||<180 days|
To transfer data out of your scratch space see the ACF Data Transfer documenation.
Each user has access to a scratch directory in Lustre which is located at /lustre/haven/user/<username>. For convenience, use the $SCRATCHDIR environment variable to refer to your Lustre scratch directory.
If you wish to determine which files are eligible to be purged from Lustre space, execute the
lfs find $SCRATCHDIR -mtime +180 -type f command. Files that will be purged from Lustre space are those that are not modified or accessed for 180 days. If you wish to view your total usage of Lustre space, execute the
lfs quota -u <user> /lustre/haven command.
Any attempts to circumvent purging, such as using the touch command on all files in a users scratch directory, will be considered a violation of the ACF acceptable use policy. Instead of taking the time to circumvent purging, why not request a project with corresponding project space. As we are all Tennessee Volunteers, our research community will be improved with positive user actions and behaviors, such as cleaning up unneeded files or requesting a project, instead of circumventing ACF file purging policy. This will result in less wasted staff support time.
Last Updated: 01 / 29 / 2020