Code Aster 15.4 MPI outside Singularity

未分類
sponsored

Preparataion

$sudo apt-get install  gfortran g++ python-dev python-numpy liblapack-dev libblas-dev tcl tk zlib1g-dev bison flex checkinstall openmpi-bin libx11-dev cmake grace gettext libboost-all-dev swig

And install superlu package.

$sudo apt-get install libopenmpi-dev python3-mpi4py cython

And I build Code aster 15.2.0 sequential version with –prefix=/opt/aster/15.2.

Code Aster 15.4

Download the source code from sourceforge and put into ~/aster-full-src-15.2.0/SRC/aster-15.4.0. And add pkginfo.py in the following folder.

OpenBLAS

To get source code from author’s HP. Then execute following procedure.

$ tar xfvz OpenBLAS-0.2.20.tar.gz
$ cd OpenBLAS-0.2.20
$ make NO_AFFINITY=1 USE_OPENMP=1
$ make PREFIX=/opt/OpenBLAS install
$ echo /opt/OpenBLAS/lib | sudo tee -a /etc/ld.so.conf.d/openblas.conf
$ sudo ldconfig

Scalapack

Download “scalapack_installer.tgz and unpack.

$./setup.py --lapacklib=/opt/OpenBLAS/lib/libopenblas.a --mpicc=mpicc --mpif90=mpif90 --mpiincdir=/usr/lib/x86_64-linux-gnu/openmpi/include --ldflags_c=-fopenmp --ldflags_fc=-fopenmp --prefix=/opt/scalapack-n --ccflag=-fPIC --fcflag=-fPIC 

Parmetis

$ tar xvf parmetis-4.0.3.tar.gz
$ cd parmetis-4.0.3

Integer size is 8 byte in this work. #define IDXTYPEWIDTH 64″ and “#define REALTYPEWIDTH 64” is written in ~/parmetis-4.0.3/metis/include/metis.h.

$ make config shared=1 prefix=/opt/parmetis-4.0.3
$ make

and install.

$ export LD_LIBRARY_PATH=/opt/parmetis-4.0.3/lib/:$LD_LIBRARY_PATH
$ make install

Scotch and Ptscotch

Unpack scotch-6.0.4-aster5.tar.gz which is included the package of code_aster in /opt.

$ cd /opt/scotch-6.0.4/src

Makefile.inc is here

$ make scotch esmumps ptscotch ptesmumps CCD=mpicc

check.

$ make check
$ make ptcheck

MUMPS

Pick mumps-5.2.1-aster2.tar.gz from the package of code_aster out and unpack it. The name of folder is changed to mumps-5.2.1_mob.

$ cd /opt/mumps-5.2.1_mob
$ make all

Next, to go to ~/examples and check the MUMPS.

Petsc

petsc 3.12.3.tar.gz is downloaded from author’s HP. Then unpack it in /opt. First , a few line (line 43 to 48) is commented out in /opt/petsc-3.12.3/config/BuildSystem/config/packages/metis.py.

def configureLibrary(self):
config.package.Package.configureLibrary(self)
oldFlags = self.compilers.CPPFLAGS
self.compilers.CPPFLAGS += ' '+self.headers.toString(self.include)
# if not self.checkCompile('#include "metis.h"', '#if (IDXTYPEWIDTH != '+ str(self.getDefaultIndexSize())+')\n#error incompatible IDXTYPEWIDTH\n#endif'):
# if self.defaultIndexSize == 64:
# msg= '--with-64-bit-indices option requires a metis build with IDXTYPEWIDTH=64.\n'
# else:
# msg= 'IDXTYPEWIDTH=64 metis build appears to be specified for a default 32-bit-indices build of PETSc.\n'
# raise RuntimeError('Metis specified is incompatible!\n'+msg+'Suggest using --download-metis for a compatible metis')

And when openmpi is used, path to openmpi library is added to LD_LIBRARY_PATH. Next is a sample.

$ export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/openmpi/lib/:$LD_LIBRARY_PATH

configure.

$ ./configure --with-debugging=0 COPTFLAGS='-O -fPIC' CXXOPTFLAGS='-O -fPIC' FOPTFLAGS='-O -fPIC' --with-shared-libraries=yes --with-scalapack-dir=/opt/scalapack-n --PETSC_ARCH=linux-metis-mumps --with-metis-dir=/opt/aster/public/metis-5.1.0 --with-ptscotch-dir=/opt/scotch-6.0.4 --LIBS="-lgomp" --with-mumps-dir=/opt/mumps-5.2.1_mob -with-x=0 --with-blas-lapack-lib=[/opt/OpenBLAS/lib/libopenblas.a]  --force --download-petsc4py=yes --download-hypre=yes --download-ml=yes --download-superlu=yes
$ make PETSC_DIR=/opt/petsc-3.12.3 PETSC_ARCH=linux-metis-mumps all

Code Aster 15.4 MPI

Go to source code folder. 

$ cd ..Downloads/aster-full-src-15.2.0/SRC
/aster-15.4.0

a part of ~/waftools/mathematics.py is modified to skip the checking of blacs.

# program testing a blacs call, output is 0 and 1
blacs_fragment = r"""
program test_blacs
    integer iam, nprocs
#    call blacs_pinfo (iam, nprocs)
#    print *,iam
#    print *,nprocs
end program test_blacs
"""

Modify profile.sh in /opt/aster/15.2/share/aster.

$ source /opt/aster/15.2/share/aster/profile.sh

Download following files and put into the source code folder.

$ ./waf clean
$ ./waf configure --use-config-dir=/opt/aster/15.2/share/aster --use-config=Ubuntu_gnu_mpi --prefix=/opt/aster/15.4MP
$ ./waf build --jobs=1
$ ./waf install

When build is finished, sign up the parallel version in /opt/aster/etc/codeaster/aster.

Testing

In case of petsc01a

コメント

タイトルとURLをコピーしました