4. Compile from Source#

4.1. When to Compile g-learn#

Generally, it is not required to compile g-learn as the installation through pip and conda contains most of its features, including support for GPU devices. You may compile g-learn if you want to:

  • modify g-learn.

  • or, build this documentation.

Otherwise, install g-learn through the Python Wheels.

This section walks you through the compilation process.

4.2. Install C++ Compiler (Required)#

Compile g-learn with either of GCC, Clang/LLVM, or Intel C++ compiler on UNIX operating systems. For Windows, compile g-learn with Microsoft Visual Studio (MSVC) Compiler for C++.

Install GNU GCC Compiler

sudo apt install build-essential

sudo yum group install "Development Tools"

sudo dnf group install "Development Tools"

sudo brew install gcc libomp

Then, export C and CXX variables by

export CC=/usr/local/bin/gcc
export CXX=/usr/local/bin/g++

Install Clang/LLVN Compiler

sudo apt install clang

sudo yum install yum-utils
sudo yum-config-manager --enable extras
sudo yum makecache
sudo yum install clang

sudo dnf install yum-utils
sudo dnf config-manager --enable extras
sudo dnf makecache
sudo dnf install clang

sudo brew install llvm libomp-dev

Then, export C and CXX variables by

export CC=/usr/local/bin/clang
export CXX=/usr/local/bin/clang++

Install Intel oneAPI Compiler

To install Intel Compiler see Intel oneAPI Base Toolkit.

4.3. Install OpenMP (Required)#

OpenMP comes with the C++ compiler installed. However, you may alternatively install it directly on UNIX. Install OpenMP library on UNIX as follows:

sudo apt install libgomp1 -y

sudo yum install libgomp -y

sudo dnf install libgomp -y

sudo brew install libomp

Note

In macOS, for libomp versions 15 and above, Homebrew installs OpenMP as keg-only. To utilize the OpenMP installation, you should establish the following symbolic links:

libomp_dir=$(brew --prefix libomp)
ln -sf ${libomp_dir}/include/omp-tools.h  /usr/local/include/omp-tools.h
ln -sf ${libomp_dir}/include/omp.h        /usr/local/include/omp.h
ln -sf ${libomp_dir}/include/ompt.h       /usr/local/include/ompt.h
ln -sf ${libomp_dir}/lib/libomp.a         /usr/local/lib/libomp.a
ln -sf ${libomp_dir}/lib/libomp.dylib     /usr/local/lib/libomp.dylib

4.4. Configure Compile-Time Environment Variables (Optional)#

Set the following environment variables as desired to configure the compilation process.

CYTHON_BUILD_IN_SOURCE#

By default, this variable is set to 0, in which the compilation process generates source files outside of the source directory, in /build directry. When it is set to 1, the build files are generated in the source directory. To set this variable, run

export CYTHON_BUILD_IN_SOURCE=1

$env:CYTHON_BUILD_IN_SOURCE = "1"

Hint

If you generated the source files inside the source directory by setting this variable, and later you wanted to clean them, see Clean Compilation Files.

CYTHON_BUILD_FOR_DOC#

Set this variable if you are building this documentation. By default, this variable is set to 0. When it is set to 1, the package will be built suitable for generating the documentation. To set this variable, run

export CYTHON_BUILD_FOR_DOC=1

$env:CYTHON_BUILD_FOR_DOC = "1"

Warning

Do not use this option to build the package for production (release) as it has a slower performance. Building the package by enabling this variable is only suitable for generating the documentation.

Hint

By enabling this variable, the build will be in-source, similar to setting CYTHON_BUILD_IN_SOURCE=1. To clean the source directory from the generated files, see Clean Compilation Files.

DEBUG_MODE#

By default, this variable is set to 0, meaning that g-learn is compiled without debugging mode enabled. By enabling debug mode, you can debug the code with tools such as gdb. Set this variable to 1 to enable debugging mode by

export DEBUG_MODE=1

$env:DEBUG_MODE = "1"

Attention

With the debugging mode enabled, the size of the package will be larger and its performance may be slower, which is not suitable for production.

4.5. Compile and Install#

repo-size

Get the source code of g-learn from the GitHub repository by

git clone https://github.com/ameli/glearn.git
cd glearn

To compile and install, run

python -m pip install .

The above command may need sudo privilege.

A Note on Using sudo

If you are using sudo for the above command, add -E option to sudo to make sure the environment variables (if you have set any) are accessible to the root user. For instance

export CUDA_HOME=/usr/local/cuda
export USE_CUDA=1
export CUDA_DYNAMIC_LOADING=1

sudo -E python -m pip install .

$env:CUDA_HOME = "/usr/local/cuda"
$env:USE_CUDA = "1"
$env:CUDA_DYNAMIC_LOADING = "1"

sudo -E python -m pip install .

Once the installation is completed, check the package can be loaded by

cd ..  # do not load glearn in the same directory of the source code
python -c "import glearn; glearn.info()"

The output to the above command should be similar to the following:

glearn version  : 0.17.0
imate version   : 0.18.0
processor       : Intel(R) Xeon(R) CPU E5-2623 v3 @ 3.00GHz
num threads     : 8
gpu device      : 'GeForce GTX 1080 Ti'
num gpu devices : 4
cuda version    : 11.2.0
nvidia driver   : 460.84
process memory  : 1.7 (Gb)

Attention

Do not load g-learn if your current working directory is the root directory of the source code of g-learn, since python cannot load the installed package properly. Always change the current directory to somewhere else (for example, cd .. as shown in the above).

Cleaning Compilation Files

If you set CYTHON_BUILD_IN_SOURCE or CYTHON_BUILD_FOR_DOC to 1, the output files of Cython’s compiler will be generated inside the source code directories. To clean the source code from these files (optional), run the following:

python setup.py clean