The National Institute for Computational Sciences

General: How do I change the striping in Lustre?

A user can change the striping settings for a file or directory in Lustre by using the lfs command. The usage for the lfs command is

lfs setstripe  -s  -i  -c 

where

size - the number of bytes on each OST (0 indicating default of 1 MB) specified with k, m, or g to indicate units of KB, MB, or GB, respectively.
index - the OST index of first stripe (-1 indicating default)
count - the number of OSTs to stripe over (0 indicating default of 4 and -1 indicating all OSTs [limit of 160]).

NOTE: If you change the settings for existing files, the file will get the new settings only if it is recreated.

To change the settings for an existing directory, you will need to rename the directory, create a new directory with the proper settings, and then copy (not move) the files to the new directory to inherit the new settings.

If your application is the type in which each separate process writes to its own file, then we believe that the best option is to not use striping. This can be set by using this command:

> lfs setstripe  -c 1

Then we see that

> lfs find -v testdirectory
OBDS:
0: ost1_UUID ACTIVE
--snip--
testdirectory/
default stripe_count: 1 stripe_size: 0 stripe_offset: -1

This shows we have a stripe count of 1 (no striping), the stripe size is set to 0 (which means use the default), and the stripe offset is set to -1 (which means to round-robin the files across the OSTs).

NOTE: You should always use -1 for stripe_offset.

The stripe count and stripe size are something you can tweak for performance. If your application writes very large files, then we believe that the best option is to stripe across all or a subset of the OSTs on the file system. Striping across all OSTs can be set by using this command:

> lfs setstripe  -c -1

Caution: Not striping large files may cause a write error if the file's size is larger than the space on a single OST. Each OST has a finite size which is smaller than the total Lustre area of all OSTs.