• National Institute for Computational Sciences is a UT/ORNL Partnership
Home

Data Transfer

  Data Transfer

 

NICS maintains high speed network connections with the XSEDE network and Internet2. NICS currently maintains the following options for file transfer:

GridFTP

When available, the preferred method for file transfer is GridFTP. XSEDE resources should have GridFTP installed, and it may be installed at other locations, check for local availability. Before using GridFTP, check out Getting Started with Globus. GSI is not currently installed on Nautilus, however GridFTP will be available for file transfers.

SCP, SFTP, rsync

The standard transfer utilities, SCP, SFTP and rsync can be used to transfer files to and from NICS systems. These utilities are usually already installed on Linux/Unix machines, and there are many command and graphical clients available. Due to familiarity and ease, these may be the best choice for transferring scripts and small files, however, these options can be slow in comparison, and may be ill suited for transferring large amounts of data. More information on these utilities can be found on the transfer utilities page.

BBCP

If GridFTP is not available, the multistreaming transfer utility BaBar copy (BBCP) may be used. The BBCP utility is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data faster than single-streaming utilities such as SCP and SFTP. Use of BBCP requires the installation of BBCP on both host systems involved in the transfer. Examples may be found on the BBCP page.

HSI/HTAR

For data transfers between Kraken and Nautilus users may stage their data through HPSS. This has the added benefit of also creating a backup copy of the data in the HPSS archive. Access to the HPSS is accomplished through the Hierarchical Storage Interface (HSI) utility. More information can be found on the HSI page.

XSEDE File Manager

The XSEDE File Manager can be used for data transfers.

Globus Online

NICS users can use the Globus Online tool to perform large file transfers, for “drag and drop archiving” to move data between its long-time archival storage and compute systems, making it quite easy to move, back up or restore relevant data using a visual interface. To get you started visit: https://www.globusonline.org/.