我想弄清楚为什么我的 OpenMPI 1.6 版本不起作用。我在 CentOS 6.6 上使用 gcc-4.7.2。给定一个玩具程序(即 hello.c)
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
int main(int argc, char * argv[])
{
int taskID = -1;
int NTasks = -1;
/* MPI Initializations */
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &taskID);
MPI_Comm_size(MPI_COMM_WORLD, &NTasks);
printf("Hello World from Task %i\n", taskID);
MPI_Finalize();
return 0;
}
并使用
mpicc hello.c 编译并运行 mpirun -np 8 ./a.out ，我收到错误:--------------------------------------------------------------------------
WARNING: No preset parameters were found for the device that Open MPI
detected:
Local host: qmaster02.cluster
Device name: mlx4_0
Device vendor ID: 0x02c9
Device vendor part ID: 4103
Default device parameters will be used, which may result in lower
performance. You can edit any of the files specified by the
btl_openib_device_param_files MCA parameter to set values for your
device.
NOTE: You can turn off this warning by setting the MCA parameter
btl_openib_warn_no_device_params_found to 0.
--------------------------------------------------------------------------
Hello World from Task 4
Hello World from Task 7
Hello World from Task 5
Hello World from Task 0
Hello World from Task 2
Hello World from Task 3
Hello World from Task 6
Hello World from Task 1
[headnode.cluster:22557] 7 more processes have sent help message help-mpi-btl-openib.txt / no device params found
[headnode.cluster:22557] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
如果我使用 mvapich2-2.1 和 gcc-4.7.2 运行它，我只会得到
Hello World from Task N 而没有任何这些错误/警告。查看链接到
a.out 的库，我得到:$ ldd a.out
linux-vdso.so.1 => (0x00007fff05ad2000)
libmpi.so.1 => /act/openmpi-1.6/gcc-4.7.2/lib/libmpi.so.1 (0x00002b0f8e196000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003954800000)
libm.so.6 => /lib64/libm.so.6 (0x0000003955400000)
librt.so.1 => /lib64/librt.so.1 (0x0000003955c00000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003965000000)
libutil.so.1 => /lib64/libutil.so.1 (0x0000003964c00000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003955000000)
libc.so.6 => /lib64/libc.so.6 (0x0000003954c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003954400000)
如果我用 mvapic2 重新编译它，
$ ldd a.out
linux-vdso.so.1 => (0x00007fffcdbcb000)
libmpi.so.12 => /act/mvapich2-2.1/gcc-4.7.2/lib/libmpi.so.12 (0x00002af3be445000)
libc.so.6 => /lib64/libc.so.6 (0x0000003954c00000)
libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x000000395e800000)
libibmad.so.5 => /usr/lib64/libibmad.so.5 (0x0000003955400000)
librdmacm.so.1 => /usr/lib64/librdmacm.so.1 (0x0000003146400000)
libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x0000003955800000)
libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x0000003956000000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003954800000)
librt.so.1 => /lib64/librt.so.1 (0x0000003955c00000)
libgfortran.so.3 => /act/gcc-4.7.2/lib64/libgfortran.so.3 (0x00002af3beaf6000)
libm.so.6 => /lib64/libm.so.6 (0x00002af3bee0a000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003955000000)
libgcc_s.so.1 => /act/gcc-4.7.2/lib64/libgcc_s.so.1 (0x00002af3bf08e000)
libquadmath.so.0 => /act/gcc-4.7.2/lib64/libquadmath.so.0 (0x00002af3bf2a4000)
/lib64/ld-linux-x86-64.so.2 (0x0000003954400000)
libz.so.1 => /lib64/libz.so.1 (0x00002af3bf4d9000)
libnl.so.1 => /lib64/libnl.so.1 (0x0000003958800000)
这里有什么问题？这是因为在 openmpi 情况下没有链接到 infiniband 库吗？
最佳答案
默认情况下，Open MPI 1.6 不附带部件 ID 为 4103 的 Mellanox ConnectX HCA 的设备参数，可以轻松修复。在 [Mellanox Hermon] 中找到 $PREFIX/share/openmpi/mca-btl-openib-device-params.ini 部分，并将 4103 附加到部件 ID 列表的末尾:
[Mellanox Hermon]
vendor_id = 0x2c9,0x5ad,0x66a,0x8f1,0x1708,0x03ba,0x15b3,0x119f
vendor_part_id = 25408,25418,25428,...<skipped>...,26488,4099,4103
use_eager_rdma = 1 ^^^^^
mtu = 2048
max_inline_data = 128
将
$PREFIX 替换为 Open MPI 安装的路径。在您的情况下，这将是 /act/openmpi-1.6/gcc-4.7.2 。关于mpi - 玩具程序使用 OpenMPI 1.6 失败，但适用于 Mvapich2，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/34029657/