Compiling

Building DREAM should be relatively straightforward once all external dependencies are installed on your system.

Preliminaries

In order to build DREAM, you must first make sure that the following software is installed on your system:

  • CMake version 3.12 or later

  • A C++17 compliant compiler and standard library (i.e. gcc >= 7 or equivalent)

  • GNU Scientific Library version 2.0 or later

  • HDF5

  • PETSc

  • OpenMP

  • MPI (for PETSc)

  • Python 3 (required for generating ADAS data)

If you are running Linux, these tools are usually available directly in the package repository for your Linux distribution. On distributed computers, you can usually install these programs using a module system (e.g. module).

Additionally, the SOFT support library softlib is required, but is automatically downloaded and installed if you have git installed.

Python

To use the DREAM Python interface, you will need Python 3 installed on your system. Additionally, the following packages are needed (all of which should be installable via pip (https://pypi.org/):

  • h5py

  • matplotlib

  • numpy

  • packaging

  • scipy

Installing dependencies with pip:

$ pip install h5py matplotlib numpy packaging scipy

Building PETSc

Tip

If you already have an installation of PETSc available (as is the case on many public computer systems) it should not be necessary to rebuild PETSc. Instead, you can proceed to building DREAM directly.

DREAM uses PETSc, the Portable, Extensible Toolkit for Scientific Computation, for working with sparse matrices. Much of the execution time is spent in PETSc or the external packages it calls during a DREAM simulation, and it is therefore useful to pay extra attention to the configuration of the library.

General steps

PETSc can be downloaded using git:

$ git clone -b release https://gitlab.com/petsc/petsc.git petsc

We generally recommend using the latest version available of PETSc (which the above command will give you). The last argument to git above, petsc, specifies the location to download PETSc to. As specified above, PETSc is downloaded to a subdirectory called petsc located in the current working directory. A common place to put PETSc is either in your home directory (~/petsc) or, if you are the system administrator, in /opt/petsc.

Once PETSc is downloaded, you will need to configure PETSc. This is achieved by going into the PETSc directory and running ./configure:

$ ./configure

Note

Generally you will also want to specify a number of configuration flags. See for example Recommended configuration and External packages for details.

After configuration has finished successfully, you can compile using the command

$ make all

After this command finishes successfully, you can proceed to compiling DREAM.

Tip

To reduce the amount of typing when compiling DREAM, you can export appropriate values for the PETSC_DIR and PETSC_ARCH environment variables in your ~/.bashrc file:

...
export PETSC_DIR="/path/to/petsc"
export PETSC_ARCH=linux-c-opt

The values to use for PETSC_DIR and PETSC_ARCH are given at the end of the PETSc configuration.

External packages

PETSc provides an interface a large number of external linear solver packages. In DREAM we have added explicit support for a few of them, and we generally recommend using one of the packages over the default built-in PETSc sparse LU factorization algorithm. In our experience, the fastest and most reliable linear solver when used in conjunction with DREAM is Intel MKL’s PARDISO solver.

Intel MKL PARDISO

The Intel Math Kernel Library (MKL) contains the PARDISO linear solver which can be used by PETSc. To include support for PARDISO in PETSc, configure it with:

$ ./configure --with-mkl_pardiso-dir=/path/to/mkl --with-blaslapack-dir=/path/to/mkl

where /path/to/mkl is the path to where the Intel MKL library is installed.

The solver can be installed along with the rest of Intel MKL and is available in the package repositories of many popular Linux distributions (including Ubuntu 20.04+ and Arch Linux). To install on recent versions of Ubuntu, simply run

sudo apt install intel-mkl

If Intel MKL is not available in the package repositories of your Linux distribution, you can download it from the official Intel MKL website.

SuperLU

To add support for the SuperLU linear solver to PETSc, configure with the command:

$ ./configure --download-superlu

Building DREAM

Summary

The whole build process is described in more detail below, but can be summarised using the following chain of commands:

$ cd /path/to/DREAM
$ mkdir build
$ cd build
$ cmake ..
$ make -j NTHREADS

where NTHREADS is the number of CPU threads available on your computer. (This number if optional, and there is not really any harm in specifying the “wrong” number; in general, the more “correct” it is, the faster the compilation will go).

If the PETSC_DIR and PETSC_ARCH environment variables are not exported in your ~/.bashrc, you may also need to give them explicitly to CMake:

$ cmake .. -DPETSC_DIR=/path/to/petsc -DPETSC_ARCH=linux-c-opt

(note that in order for the variables to be defined, you must restart bash after exporting them in your ~/.bashrc).

Troubleshooting

PETSC_EXECUTABLE_RUNS missing

On some systems, particularly Ubuntu, you will need to override the PETSC_EXECUTABLE_RUNS CMake variable:

$ cmake .. -DPETSC_EXECUTABLE_RUNS=YES

“PETSc was configured with MPICH but now appears to be compiling using a non-MPICH mpi.h”

This error can occur if you have installed MPI while configuring PETSc, or if you have multiple MPI implementations (e.g. MPICH and OpenMPI) installed alongside each other on your system. If you installed MPICH automatically during the configuration of PETSc you should run CMake with the flag

$ cmake .. -DMPI_CXX_COMPILER=/path/to/petsc/$PETSC_ARCH/bin/mpicxx

Alternatively, if you compiled PETSc with a system-wide MPICH installation you should specify

$ cmake .. -DMPI_EXECUTABLE_SUFFIX=.mpich

or, you use OpenMPI

cmake .. -DMPI_EXECUTABLE_SUFFIX=.openmpi