This page is a note of the work to build parallel version.
The linux box is xUbuntu 18.04LTS on VMware.
０．preparation
Preparation is same as the case of CodeAster 14.4.
Some packages are installed as follows.
$sudo apt-get install gfortran g++ python-dev python-numpy liblapack-dev libblas-dev tcl tk zlib1g-dev bison flex checkinstall openmpi-bin libx11-dev cmake grace gettext libboost-all-dev swig
And install superlu package.
$sudo apt-get install libsuperlu-dev
１．OpenBLAS
To get source code from author’s HP. Then execute following procedure.
$ tar xfvz OpenBLAS-0.2.20.tar.gz
$ cd OpenBLAS-0.2.20
$ make NO_AFFINITY=1 USE_OPENMP=1
$ make PREFIX=/opt/OpenBLAS install
$ echo /opt/OpenBLAS/lib | sudo tee -a /etc/ld.so.conf.d/openblas.conf
$ sudo ldconfig
２．Code_Aster eith OpenBLAS
First, the source code of code_aster is unpacked. Next, “setup.py” is modified according to the reference.
$ cd aster-full-src-14.4.0
$ sed -i "s:PREFER_COMPILER\ =\ 'GNU':PREFER_COMPILER\ =\'GNU_without_MATH'\nMATHLIB=\ '/opt/OpenBLAS/lib/libopenblas.a':g" setup.cfg
And
$ python3 setup.py install
After the build complete, to make host file for parallel calculation.
$ echo "$HOSTNAME cpu=$(cat /proc/cpuinfo | grep processor | wc -l)" > /opt/aster/etc/codeaster/mpi_hostfile
３．ScaLAPACK
$ tar xfvz scalapack_installer.tgz
$ cd scalapack_installer
$ ./setup.py --lapacklib=/opt/OpenBLAS/lib/libopenblas.a --mpicc=mpicc --mpif90=mpif90 --mpiincdir=/usr/lib/x86_64-linux-gnu/openmpi/include --ldflags_c=-fopenmp --ldflags_fc=-fopenmp --prefix=/opt/scalapack-n
Over.
４．Parmetis
Get source code and unpack it. Integer size is 8 byte in this work.
#define IDXTYPEWIDTH 64″ and “#define REALTYPEWIDTH 32” is written in ~/parmetis-4.0.3/metis/include/metis.h. or add followings to metis.h
#ifdef INTSIZE32
#define IDXTYPEWIDTH 32
#else
#define IDXTYPEWIDTH 64
#endif
then
$ tar xvf parmetis-4.0.3.tar.gz
$ cd parmetis-4.0.3
$ make config prefix=/opt/parmetis-4.0.3
$ make
Next, checking.
$ cd Graphs
$ mpirun -np 8 ptest rotor.graph rotor.graph.xyz
When any error is reported, go to~/parmetis-4.0.3 and execute follows:
$ make install
This part is finished.
５．Scotch and Ptscotch
Unpack scotch-6.0.4-aster5.tar.gz which is included the package of code_aster in /opt.
$ cd /opt/scotch-6.0.4/src
Makefile.inc is here
$ make scotch esmumps ptscotch ptesmumps CCD=mpicc
After building, check it.
$ make check
$ make ptcheck
This part is finised.
６．MUMPS
Pick mumps-5.1.2-aster6.tar.gz from the package of code_aster out and unpack it. The name of folder is changed to mumps-5.1.2_mob.
$ cd /opt/mumps-5.1.2_mob
Makefile.inc is.here.
$ make all
Next, to go to ~/examples and check the MUMPS.
７．Petsc
petsc-3.9.4.tar.gz is downloaded from author’s HP. Then unpack it in /opt. First , a few line (line 43 to 48) is commented out metis.py in /opt/petsc-3.9.4/config/BuildSystem/config/packages.
def configureLibrary(self):
config.package.Package.configureLibrary(self)
oldFlags = self.compilers.CPPFLAGS
self.compilers.CPPFLAGS += ' '+self.headers.toString(self.include)
# if not self.checkCompile('#include "metis.h"', '#if (IDXTYPEWIDTH != '+ str(self.getDefaultIndexSize())+')\n#error incompatible IDXTYPEWIDTH\n#endif'):
# if self.defaultIndexSize == 64:
# msg= '--with-64-bit-indices option requires a metis build with IDXTYPEWIDTH=64.\n'
# else:
# msg= 'IDXTYPEWIDTH=64 metis build appears to be specified for a default 32-bit-indices build of PETSc.\n'
# raise RuntimeError('Metis specified is incompatible!\n'+msg+'Suggest using --download-metis for a compatible metis')
And when openmpi is used, path to openmpi library is added to LD_LIBRARY_PATH. Next is a sample.
$ export LD_LIBRARY_PATH =/usr/lib/x86_64-linux-gnu/openmpi/lib/:$LD_LIBRARY_PATH
Then, “configure” is done.
$./configure --with-debugging=0 COPTFLAGS=-O CXXOPTFLAGS=-O FOPTFLAGS=-O --with-shared-libraries=0 --with-scalapack-dir=/opt/scalapack-n --PETSC_ARCH=linux-metis-mumps --with-metis-dir=/opt/aster/public/metis-5.1.0 --with-parmetis-dir=/opt/parmetis-4.0.3 --with-ptscotch-dir=/opt/scotch-6.0.4 --LIBS="-lgomp" --with-mumps-dir=/opt/mumps-5.1.2_mob -with-x=0 --with-blas-lapack-lib=[/opt/OpenBLAS/lib/libopenblas.a] --download-hypre=yes --download-ml=yes
and “make”
$ make PETSC_DIR=/opt/petsc-3.9.4 PETSC_ARCH=linux-metis-mumps all
８．Parallel version Code_Aster
Go to the folder stored the source code of code_aster. And unpack aster-14.4.0.
$ cd ~/Install_Files
$ cd aster-full-src-14.4.0/SRC
$ tar xfvz aster-14.4.0.tgz
$ cd aster-14.4.0
Next a part of ~/waftools/mathematics.py is modified to skip the checking of blacs.
# program testing a blacs call, output is 0 and 1
blacs_fragment = r"""
program test_blacs
integer iam, nprocs
# call blacs_pinfo (iam, nprocs)
# print *,iam
# print *,nprocs
end program test_blacs
"""
Then copy Ubuntu_gnu.py and Ubuntu_gnu_mpi.py to the folder. Now preparation of build is ready.
$ export ASTER_ROOT=/opt/aster
$ export PYTHONPATH=/$ASTER_ROOT/lib/python3.6/site-packages/:$PYTHONPATH
$ ./waf configure --use-config-dir=$ASTER_ROOT/14.4/share/aster --use-config=Ubuntu_gnu_mpi --prefix=$ASTER_ROOT/PAR14.4MUPT
$ ./waf install -p --jobs=1
When build is finished. add register the parallel version to /opt/aster/etc/codeaster/aster.
# Code_Aster versions
# versions can be absolute paths or relative to ASTER_ROOT
# examples : NEW11, /usr/lib/codeaster/NEW11
# default version (overridden by --vers option)
default_vers : stable
# available versions
# DO NOT EDIT FOLLOWING LINE !
#?vers : VVV?
vers : stable:/opt/aster/14.4/share/aster
vers : 13.6MUPT:/opt/aster/PAR14.4MUPT/share/aster
Whole work is finished.
I got many information from Code_aster forum. I appreciate the form members.
