Category: Program Dev-Data Movement
Allows the user to securely and quickly copy a file from source to destination. It is ideally suite to transfer large files with its multistreaming transfer abilites. bbcp is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data much faster than single-streaming utilities such as scp and sftp.
We are using a modified version of bbcp that incorporates the
(recursive) option to support tranfer of entire directory trees. Until
SLAC incorporates this update into its source code and binaries, be aware
that binaries downloaded from SLAC will not support recursive
More information on bbcp can be found by typing
bbcp -h on the NICS
systems as well as on the
Before you can use the bbcp utility, it must be installed on both the local and remote systems. It is currently available on Nautilus as part of each user's default environment. Several precompiled binaries as well as the source can be downloaded from the Stanford Linear Accelerator Center (SLAC) bbcp download page.
There is a man page at bbcp for detailed documentation.
To transfer the local file
/local/path/largefile.tar to the remote system
/remote/path/largefile.tar, use the following:
> bbcp -P 2 -V -w 8m -s 16 /local/path/largefile.tar remotesystem:/remote/path/largefile.tarwhere
-P 2produces progress messages every 2 seconds,
-Vproduces verbose output, including detailed transfer-speed statistics,
-w 8msets the size of the disk input/output (I/O) buffers, and
-s 16sets the number of parallel network streams to 16.
> ssh remotesystem which bbcpIf this is not the case, the
-Toption can be used to specify how to start bbcp on the remote system. For example, you could use the following:
> bbcp -P 2 -V -w 8m -s 16 -T 'ssh -x -a -oFallBackToRsh=no %I -l %U %H /remote/path/to/bbcp' /local/path/largefile.tar remotesystem:/remote/path/largefile.tarOften, during large transfers the connection between the transferring systems is lost. The
-aoptions gives bbcp the ability to pick up where it left off. For example, you could use the following:
> bbcp -k -a /remotesystem/homedir/.bbcp/ -P 2 -V -w 8m -s 16 /local/path/largefile.tar remotesystem:/remote/path/largefile.tarTo transfer an entire directory tree, use the following:
> bbcp -r -P 2 -V -w 8m -s 16 /local/path/* remotesystem:/remote/path
Copyright © 2002, Board of Trustees of the Leland Stanford, Jr. University.
Produced under contract DE-AC03-76-SF00515 with the US Department of Energy.
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
a. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
b. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
c. Neither the name of the Leland Stanford, Jr. University nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
This package has the following support level : Supported