The National Institute for Computational Sciences


HPSS (Darter and Nautilus): Is the HPSS system able to be accessed by more than one process at a time?

There is nothing that should prevent you from running a script that creates multiple simultaneous connections to HPSS. The HPSS system administrator recommends that you should not create more than 1 or 2 connections at a time. Every time you introduce a new instance, the performance of the overall system is degraded.

HPSS (Darter and Nautilus): What is the best way to transfer a large number of small files to HPSS?

HPSS performance is greatly improved when the transfer size is between 8 GB and 256 GB. For that reason, users with large numbers of relatively small files should combine those files into one or a few 8 GB to 256 GB files and then transfer the larger files. The files can be combined with tar on the HPC system, or they can be created on the fly with a command similar to tar cvf some_dir -|hsi put - : somedir.tar. This command will tar all files in the some_dir subdirectory into a file named somedir.tar on HPSS.


Subscribe to RSS - HPSS