|
Data Transfer |
|
|---|---|
NICS provides several ways for transferring files to/from all of our HPC resources, that can benefit from the high speed network connections with the XSEDE network and Internet2.
GridFTP
This is the preferred method to transfer files. All XSEDE resources have GridFTP installed. Check for local availability at other locations. Before using GridFTP, check out the Getting Started with Globus page. For more detailed information about how to use it, please visit the dedicated GridFTP page. This method can be used via the command line or a web interface called Globus Online.
SCP, SFTP, rsync
For smaller data transfers, you can use the standard transfer utilities, SCP, SFTP and rsync, which are usually available on all Linux/Unix machines. There are command line and graphical clients available. These utilities are ill suited for transferring large amounts of data. More information on these utilities can be found on the transfer utilities page.
BBCP
This is another tool to securely and quickly copy data from source to target. The BBCP utility is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data faster than single-streaming utilities such as SCP and SFTP. Check for local availability, but if it is not available the source code can be obtained from its homepage. Several examples about how to use it, can be found at its dedicated BBCP page.
HSI/HTAR
These tools are the only way to transfer files to/from the HPSS archival system. In order to archive files, it is required to login with the RSA tokens. HSI can be used to transfer single files, whereas HTAR offers a convenient way to bundle and archive multiple files in one step. More information can be found on the HSI page.
XSEDE File Manager
The XSEDE File Manager is a java applet available through the XUP. It enables access to desktop files, remote XSEDE systems, cloud storage systems (specifically Eucalyptus and Amazon's S3), and external systems defined by users. It supports third-party transfers, bookmarking, and remote searching. The TGUP File Manager was designed to be a common, intuitive interface for both novice and expert users. Visit XSEDE File Manager for more information.
Globus Online
NICS users can use the web Globus Online tool to perform that makes it it quite easy to move, back up or restore relevant data using a visual interface. To get you started visit the Globus Online website.