本文共 7670 字,大约阅读时间需要 25 分钟。
参考http://blog.csdn.net/icefromair/article/details/51524476
1、系统环境
ubuntu server 16.04
软件:
hpl-2.2.tar.gzl_mkl_2018.1.163.tgzl_mpi_2018.1.163.tgz
2、解压安装mkl
tar命名解压安装即可,然后进入目录,直接执行install.sh文件,根据提示安装即可。
node1@node1:~$ tar -zxvf l_mkl_2018.1.163.tgznode1@node1:~$ cd l_mkl_2018.1.163/node1@node1:~/l_mkl_2018.1.163$ ./install.sh#后面根据提示进行就可以了
3、解压安装mpi
同mkl一样解压安装即可。
node1@node1:~$ tar -zxvf l_mpi_2018.1.163.tgz node1@node1:~$ cd l_mpi_2018.1.163/node1@node1:~/l_mpi_2018.1.163$ ./install.sh #同样根据提示进行就可以
4、编译安装hpl
4.1、解压hpl文件
node1@node1:~$ tar -zxvf hpl-2.2.tar.gznode1@node1:~$ cd hpl-2.2/
node1@node1:~/hpl-2.2$ cp setup/Make.Linux_PII_CBLAS .
把setup目录里面的Make.Linux_PII_CBLAS文件复制到top目录也就是setup的上级目录,hpl-2.2目录下,然后命名一下
我的命名是Make.ubuntu_mkl_intel,文件里面arch=Make.ubuntu_mkl_intel,这两个是一样的。
修改后文件主要有下面内容(去掉了注释行):
SHELL = /bin/shCD = cdCP = cpLN_S = ln -sMKDIR = mkdirRM = /bin/rm -fTOUCH = touchARCH = ubuntu_mkl_intelTOPdir = $(HOME)/hpl-2.2INCdir = $(TOPdir)/includeBINdir = $(TOPdir)/bin/$(ARCH)LIBdir = $(TOPdir)/lib/$(ARCH)HPLlib = $(LIBdir)/libhpl.a MPdir = /usr/local/intel/impi/2018.1.163/intel64MPinc = -I$(MPdir)/includeMPlib = $(MPdir)/lib/libmpi.aLAdir = -L/usr/local/intel/mkl/lib/intel64 -L/usr/local/intel/lib/intel64#这里也可以通过导入链接库路径也最好这样,LAinc = -I/usr/local/intel/mkl/include #不然像我这样,最后执行xhpl文件会找不到链接库,要添加到LD_LIBRARY_PATH中才可以LAlib = -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lm#F2CDEFS =HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAdir) $(LAinc) $(MPinc) -lpthreadHPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) -lpthreadHPL_OPTS = -DHPL_CALL_CBLASHPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)CC = /usr/local/intel/impi/2018.1.163/bin64/mpicc -lpthreadCCNOOPT = $(HPL_DEFS)CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loopsLINKER = /usr/local/intel/impi/2018.1.163/bin64/mpicc -lpthreadLINKFLAGS = $(CCFLAGS)ARCHIVER = arARFLAGS = rRANLIB = echo
接下来编译:
node1@node1:~/hpl-2.2$ make arch=ubuntu_mkl_intel#如果没有报错,在当前目录下有个bin目录会看到ubuntu_mkl_intel目录,里面就有可以这行的文件了
5、运行测试:
node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ mpirun -n 4 ./xhpl ./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory./xhpl: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
出现了我说上面说的问题,没有添加没有把共享库路径写进LD_LIBRARY_PATH,因为这个共享库没有在默认搜索路径下,所以现在添加到LD_LIBRARY_PATH就可以。
node1@node1:/etc/profile.d$ sudo vim mkl.sh#写入export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/intel/mkl/lib/intel64:/usr/local/intel/lib/intel64然后重新加载这个文件node1@node1:/etc/profile.d$ source mkl.sh#你也可以不写在这里。能加入环境变量即可
重新运行文件:
node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ ldd xhpl#可以看到动态链接库都找得到了node1@node1:~/hpl-2.2/bin/ubuntu_mkl_intel$ mpirun -n 4 ./xhpl#就可以看到一大堆的运行结果了,部分如下--------------------------------------------------------------------------------||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)= 0.0167027 ...... PASSED================================================================================T/V N NB P Q Time Gflops--------------------------------------------------------------------------------WR00R2R4 35 4 4 1 0.00 2.786e-01HPL_pdgesv() start time Wed Mar 7 22:46:45 2018HPL_pdgesv() end time Wed Mar 7 22:46:45 2018--------------------------------------------------------------------------------||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)= 0.0234356 ...... PASSED================================================================================Finished 864 tests with the following results: 864 tests completed and passed residual checks, 0 tests completed and failed residual checks, 0 tests skipped because of illegal input values.--------------------------------------------------------------------------------End of Tests.===============================================================================
6、总结
这几天装各种不同数学库和mpi版本出现各种问题,希望借此能帮助一些同样装hpl遇到困难的人。
7、附带一些关于mpich+atlas、mpich+openblas、mpich+gotoblas2数学库的配置文件
MPICH + atlas
#MPICH+AtlasSHELL = /bin/shCD = cdCP = cpLN_S = ln -sMKDIR = mkdirRM = /bin/rm -fTOUCH = touchARCH = ubuntu_atlas_mpichTOPdir = $(HOME)/hpl-2.2INCdir = $(TOPdir)/includeBINdir = $(TOPdir)/bin/$(ARCH)LIBdir = $(TOPdir)/lib/$(ARCH)HPLlib = $(LIBdir)/libhpl.a MPdir = /usr/local/mpichMPinc = -I$(MPdir)/includeMPlib = $(MPdir)/lib/libmpi.aLAdir = /usr/local/atlas/libLAinc =LAlib = $(LAdir)/libcblas.a $(LAdir)/libatlas.aF2CDEFS =HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthreadHPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) -lpthreadHPL_OPTS = -DHPL_CALL_CBLASHPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)CC = /usr/local/mpich/bin/mpicc -lpthreadCCNOOPT = $(HPL_DEFS)CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loopsLINKER = /usr/local/mpich/bin/mpiccLINKFLAGS = $(CCFLAGS)ARCHIVER = arARFLAGS = rRANLIB = echo
MPICH + openblas
SHELL = /bin/shCD = cdCP = cpLN_S = ln -sMKDIR = mkdirRM = /bin/rm -fTOUCH = touchARCH = ubuntu_open_mpichTOPdir = $(HOME)/hpl-2.2INCdir = $(TOPdir)/includeBINdir = $(TOPdir)/bin/$(ARCH)LIBdir = $(TOPdir)/lib/$(ARCH)HPLlib = $(LIBdir)/libhpl.a MPdir = /usr/local/mpichMPinc = -I$(MPdir)/includeMPlib = $(MPdir)/lib/libmpi.aLAdir = /usr/local/openblas/libLAinc =LAlib = $(LAdir)/libopenblas.aF2CDEFS =HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthreadHPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) -lpthreadHPL_OPTS = -DHPL_CALL_CBLASHPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)CC = /usr/local/mpich/bin/mpicc -lpthreadCCNOOPT = $(HPL_DEFS)CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loopsLINKER = /usr/local/mpich/bin/mpiccLINKFLAGS = $(CCFLAGS)ARCHIVER = arARFLAGS = rRANLIB = echoMPI + GotoBlas2
SHELL = /bin/shCD = cdCP = cpLN_S = ln -sMKDIR = mkdirRM = /bin/rm -fTOUCH = touchARCH = ubuntu_goto_mpichTOPdir = $(HOME)/hpl-2.2INCdir = $(TOPdir)/includeBINdir = $(TOPdir)/bin/$(ARCH)LIBdir = $(TOPdir)/lib/$(ARCH)HPLlib = $(LIBdir)/libhpl.a MPdir = /usr/local/mpichMPinc = -I$(MPdir)/includeMPlib = $(MPdir)/lib/libmpi.aLAdir = $(HOME)/GotoBLAS2LAinc =LAlib = $(LAdir)/libgoto2.aF2CDEFS =HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -lpthreadHPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) -lpthreadHPL_OPTS = -DHPL_CALL_CBLASHPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)CC = /usr/local/mpich/bin/mpicc -lpthreadCCNOOPT = $(HPL_DEFS)CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loopsLINKER = /usr/local/mpich/bin/mpiccLINKFLAGS = $(CCFLAGS)ARCHIVER = arARFLAGS = rRANLIB = echo