Kraken will be officially retired and no longer accessible on August 27, 2014. For more information see Kraken Decommission FAQs.
Kraken will be officially retired and no longer accessible on August 27, 2014. For more information see Kraken Decommission FAQs.
The National Institute for Computational Sciences

File Systems

Network File System (NFS)

The NFS server contains user's home directories, project directories, and software directories. Home and project directories can be used to store frequently used items such as source code, binaries, and scripts. NFS is accessible from all NICS resources.

Home Directories

A user's home directory is found at:

/nics/[a-d]/home/$USER

The environment variable $HOME contains this path. For home directories, each user receives a default storage limit of 2 GB. To request an increase in this limit please sumbit a request to help@xsede.org. Home directories are regularly backed up to tape.

Lustre File System

The scratch disk space for NICS resources utilizes Lustre, a massively parallel distributed file system. The following Lustre scratch areas are available:

PathMounted onSize
/lustre/medusa/$USER Darter, Nautilus, Keeneland, Beacon 1.3 PB

For convenience, the environment variable $SCRATCHDIR points to the user's corresponding Lustre directory for each machine.

Important Points for Users

  • The Lustre filesystem is scratch space intended for production work and not for long term storage. Files in scratch directories are not backed up and are subject to being purged after 30 days. It is the user's responsibility to back up all important data to HPSS or other storage resources.

    The Lustre find command can be used to locate any of your files that have been around for 30 days or longer:

    > lfs find /lustre/medusa/$USER -mtime +30 -type f
    
  • This will recursively list all regular files in your Lustre scratch area that are in danger of being purged.

  • Striping is an important concept with Lustre—basically, the ability to break files into chunks and spread them across multiple storage targets (called OSTs). The striping defaults set up for NICS resources are usually fine but may need to be altered in certain use cases, like when dealing with very large files. Please see our Lustre Striping Guide for details.

  • Beware of using normal Linux commands for inspecting and managing your files and directories in Lustre scratch space. Using ls -l can cause undue load and may hang because it necessitates access to all OSTs holding your files. Make sure that your ls is not aliased to ls -l.

  • Use lfs quota to see your total usage on the Lustre system. You must specify your username and the Lustre path with this command, for example:

    > lfs quota -u <username> /lustre/medusa
    

For more detailed information regarding Lustre usage, see the following pages:

Project Directories

For sharing data among a research group, NICS provides project directories on NFS or Lustre. For information on requesting and managing a project directory see here.

High Performance Storage System (HPSS)

HPSS is the archival mass storage resource jointly operated by NICS and the Oak Ridge Leadership Computing Facility (OLCF). For long-term storage, user's can transfer data to HPSS from Kraken or Nautilus. Please see the HPSS documentation on using the HSI and HTAR utilities for storage in HPSS.