The National Institute for Computational Sciences

Darter: Can I use MPI_Alltoall with MPI_IN_PLACE?

The MPI_IN_PLACE option causes communication on an intra-communicator to happen in place, rather than being copied into buffers. This reduces the required number of operations (it is only possible within a node, not between nodes).

In order to use this option with MPI_Alltoall, you need to disable Cray's optimization for that call:

export MPICH_COLL_OPT_OFF=mpi_alltoall