The python package pygenten provides python bindings for the GenTen package. GenTen is a tool for computing Canonical Polyadic (CP, also called CANDECOMP/PARAFAC) decompositions of tensor data. It is geared towards analysis of extreme-scale data and implements several CP decomposition algorithms that are parallel and scalable, including:
- CP-ALS: The workhorse algorithm for Gaussian sparse or dense tensor data.
- CP-OPT: CP decomposition of (sparse or dense) Gaussian data using a quasi-Newton optimization algorithm incorporating possible upper and lower bound constraints.
- GCP: Generalized CP supporting arbitrary loss functions (Gaussian, Poisson, Bernoulli, ...), solved using quasi-Newton (dense tensors) or stochastic gradient descent (sparse or dense tensors) optimization methods.
- Streaming GCP: A GCP algorithm that incrementally updates a GCP decomposition as new data is observed, suitable for in situ analysis of streaming data.
- Federated GCP: A federated learning algorithm for GCP supporting asynchronous parallel communication.
GenTen builds on Kokkos and Kokkos Kernels to support shared memory parallel programming models on a variety of contemporary architectures, including:
- OpenMP for CPUs.
- CUDA for NVIDIA GPUs.
- HIP for AMD GPUs.
- SYCL for Intel GPUs.
GenTen also supports distributed memory parallelism using MPI.
There are two general approaches for building pygenten:
- Enable python in the generic CMake build process described here.
- Install using pip which automates the CMake build to some degree.
pygenten has experimental support for installation using pip from the source distribution on pypi. Furthermore, binary wheels are provided in the following limited circumstances, enabling immediate installation:
| OS | Arch | Kokkos Backend | BLAS/LAPACK |
|---|---|---|---|
| Linux | x86_64 | OpenMP | OpenBLAS |
| Macos-14 (Sonoma) | arm64 | OpenMP | Accelerate |
| Macos-15 (Sequoia) | arm64 | OpenMP | Accelerate |
(in the future, Windows binaries as well as GPU backends are expected to be provided). The pip installation leverages scikit-build-core to provide a CMake build backend for pip, which allows the user to provide CMake defines that control the pygenten build process and determine which architectures/parallel programming models are enabled. We thus recommend becoming familiar with the CMake build process for GenTen in general as described here before continuing. In particular, the user must have BLAS and LAPACK libraries available in their build environment that can either be automatically discovered by CMake or manually specified through LAPACK_LIBS.
A basic installation of pygenten can be done simply by:
pip install pygenten
This will install the binary wheel if it is available, and if it isn't, build GenTen and pygenten for a CPU architecture using OpenMP parallelism using a default compiler from the user's path. During the build of pygenten, CMake will attempt to locate valid BLAS and LAPACK libraries in the user environment. If these cannot be found, the user can customize the build by specifying LAPACK_LIBS as described below.
To customize the GenTen/pygenten build, you must first instruct pip to compile from source by adding the --no-binary pygenten command-line argument. The can then be customized by passing CMake defines to specify compilers, BLAS/LAPACK libraries, host/device architectures, and enabled programming models. This is done by adding command-line arguments to pip of the form
--config-settings=cmake.define.SOME_DEFINE=value
Any CMake define accepted by GenTen/Kokkos/KokkosKernels can be passed this way. Since this is fairly verbose and GenTen can require several defines, several meta-options are provided to enable supported parallel programming models:
| CMake Define | What it enables |
|---|---|
| PYGENTEN_MPI | Enable distributed parallelism with MPI. Sets the execution space to Serial by default. |
| PYGENTEN_OPENMP | Enable shared memory host parallelism using OpenMP |
| PYGENTEN_SERIAL | No host shared memory parallelism. Useful for builds targeting GPU architectures or distributed memory parallelism |
| PYGENTEN_CUDA | Enable CUDA parallelism for NVIDIA GPU architectures |
| PYGENTEN_HIP | Enable HIP parallelism for AMD GPU architectures |
| PYGENTEN_SYCL | Enable SYCL parallelism for Intel GPU architectures |
When enabling GPU architectures, one also needs to specify the corresponding architecture via Kokkos_ARCH_* defines described here.
For example, an MPI+CUDA build for a Volta V100 GPU architecture can be obtained with
pip install -v --no-binary pygenten --config-settings=cmake.define.PYGENTEN_CUDA=ON --config-settings=cmake.define.Kokkos_ARCH_VOLTA70=ON --config-settings=cmake.define.PYGENTEN_MPI=ON pygenten
For MPI builds, pygenten assumes the MPI compiler wrappers mpicxx and mpicc are available in the user's path. If this is not correct, the user can specify the appropriate compiler by setting the appropriate CMake define, e.g., CMAKE_CXX_COMPILER. Furthermore, for CUDA builds, pygenten will build with the nvcc_wrapper script as the compiler as required by Kokkos, which calls g++ as the host compiler by default. This can be changed by setting the NVCC_WRAPPER_DEFAULT_COMPILER environment variable. Moreover, for MPI+CUDA, pygenten will set environment variables to override the compiler wrapped by mpicxx to use nvcc_wrapper, which currently works only with OpenMPI and MPICH. Finally, for MPI+HIP or MPI+SYCL builds, pygenten assumes the compiler wrappers call the appropriate device-enabled compiler, e.g., hipcc for AMD and icpx for Intel.
pygenten relies on numpy, both of which are compiled extension libraries leveraging OpenMP and BLAS/LAPACK. Therefore, module import errors can occur if pygenten and numpy are compiled in very different environments, due to, e.g., symbol conflicts in libstdc++. This typically happens when pygenten is compiled with a much newer compiler than what was used to compile the numpy wheel. Futhermore, we have observed slower performance in pygenten in some cases when numpy is imported before pygenten, which we believe is due to inconsistent OpenMP and/or BLAS/LAPACK libraries between the two packages. Thus, the most robust way to use pygenten is to also install numpy from source, using the same build environment as pygenten. This can be done in a similar manner as pygenten by providing configure options to numpy through pip, e.g.,
pip install --no-binary numpy -Csetup-args=-Dblas=my_blas -Csetup-args=-Dlapack=my_lapack numpy
to specify the appropriate BLAS and LAPACK libraries (called my_blas and my_lapack in this case). Compilers can be specified through the CC, CXX, and FC environment variables. More details can be found here. However, we only recommend doing this if you see errors when importing pygenten or you observe slower performance than what would be observed with the genten command-line tool.