Please add standard Cmake support for MKL

[VictorEijkhout]

CMake Error at CMakeLists.txt:367 (message):
  BLAS++ requires a BLAS library and none was found.  Ensure that it is
  accessible in environment variables $CPATH, $LIBRARY_PATH, and
  $LD_LIBRARY_PATH.

I believe CPATH is a Gnu extension. Package discovery in Cmake is done through CMAKE_MODULE_PATH or (as in the case of MKL) PGK_CONFIG_PATH.

Could you add support for the latter?

[mgates3]

Please always include your command line input (and any relevant environment variables) to help us replicate what you did.

I'm not opposed to PKG_CONFIG_PATH, but I'm not quite sure how to support it. For instance, with MKL there are actually 17 different packages:

sh leconte 2023.0.0> ls -l $MKLROOT/lib/pkgconfig
total 68
-rw-r--r-- 1 jenkins jenkins 1570 Jan  2 12:40 mkl-dynamic-ilp64-gomp.pc
-rw-r--r-- 1 jenkins jenkins 1569 Jan  2 12:40 mkl-dynamic-ilp64-iomp.pc
-rw-r--r-- 1 jenkins jenkins 1538 Jan  2 12:40 mkl-dynamic-ilp64-seq.pc
-rw-r--r-- 1 jenkins jenkins 1564 Jan  2 12:40 mkl-dynamic-ilp64-tbb.pc
-rw-r--r-- 1 jenkins jenkins 1556 Jan  2 12:40 mkl-dynamic-lp64-gomp.pc
-rw-r--r-- 1 jenkins jenkins 1555 Jan  2 12:40 mkl-dynamic-lp64-iomp.pc
-rw-r--r-- 1 jenkins jenkins 1524 Jan  2 12:40 mkl-dynamic-lp64-seq.pc
-rw-r--r-- 1 jenkins jenkins 1550 Jan  2 12:40 mkl-dynamic-lp64-tbb.pc
-rw-r--r-- 1 jenkins jenkins 1488 Jan  2 12:40 mkl-sdl.pc
-rw-r--r-- 1 jenkins jenkins 1612 Jan  2 12:40 mkl-static-ilp64-gomp.pc
-rw-r--r-- 1 jenkins jenkins 1630 Jan  2 12:40 mkl-static-ilp64-iomp.pc
-rw-r--r-- 1 jenkins jenkins 1599 Jan  2 12:40 mkl-static-ilp64-seq.pc
-rw-r--r-- 1 jenkins jenkins 1634 Jan  2 12:40 mkl-static-ilp64-tbb.pc
-rw-r--r-- 1 jenkins jenkins 1598 Jan  2 12:40 mkl-static-lp64-gomp.pc
-rw-r--r-- 1 jenkins jenkins 1616 Jan  2 12:40 mkl-static-lp64-iomp.pc
-rw-r--r-- 1 jenkins jenkins 1585 Jan  2 12:40 mkl-static-lp64-seq.pc
-rw-r--r-- 1 jenkins jenkins 1620 Jan  2 12:40 mkl-static-lp64-tbb.pc

How would you see the user specifying which of those packages to use?
Is there an existing package that uses pkg-config for MKL to expose these different options?

MKL adds itself to CPATH, etc. when running setvars.sh.

CPATH is supported by a variety of compilers including GNU, Clang, and Intel compilers.

You can also specify the include path via CXXFLAGS instead of CPATH. The include path is needed only for the tester, not for the BLAS++ library itself.

You can also specify the library path via LDFLAGS instead of LIBRARY_PATH. For example:

unset CPATH
unset LIBRARY_FLAGS
unset LD_LIBRARY_FLAGS
export LDFLAGS=-L${MKLROOT}/lib/intel64
export CXXFLAGS=-I${MKLROOT}/include    # needed only for tester
cmake ..

Alternatively, you can use CMake's BLAS search. E.g., set BLA_VENDOR=Intel10_64lp

cmake -DBLA_VENDOR=Intel10_64lp ..

In some cases, that has fewer options, e.g., it has only one IBMESSL version, not specific ones for 32-bit/64-bit integers, sequential and multi-threaded.

Note both MKL and OpenBLAS have pkg-config and cmake files. Haven't looked at other libraries yet.

[ax3l]

X-ref: icl-utk-edu/lapackpp#7

@mgates3 note that we try to build a CUDA-enabled BLAS++ here (target are TACC Lonestar6 A100 GPUs with CUDA 11.4).

[VictorEijkhout]

Please always include your command line input

Oh right. Cmake doesn't even record how it was invoked. Trust me, if I ask for cmake supoprt it's not because I'm a fan.

How would you see the user specifying which of those packages to use?

Eh, you're the user. The PKG_CONFIG tells you were the pc files are, and then you pick the single/multi-threaded / whatever you need. No?

CPATH is supported by a variety of compilers

1. The whole cmake ecosystem seems an exercise in buck-passing. Does no every package start setting CPATH in addition to generating a .pc and .cmake file?

2. The standard cmake mechanisms define a bunch of convenient macros that can be used in the CMakeLists. CPATH is sort of bypassing cmake: it's a way to directly inject into the compiler. That's good for makefiles, but cmake has more sophisticated mechanisms.

You can also specify the include path via CXXFLAGS instead of CPATH.

And I can use autoconf and makefiles....

Note both MKL and OpenBLAS have pkg-config and cmake files. Haven't looked at other libraries yet.

BLIS does too. So use them?

V.

[mgates3]

My question is for an end-user of BLAS++, what should that look like? Should they say:

blaspp/build>  cmake -Dblas=mkl ..

and then BLAS++ should figure out which pkg-config file to load? Or they should say specifically which MKL variant they want, like:

blaspp/build>  cmake -Dblas=mkl-dynamic-ilp64-tbb ..

When there's only one choice of a package, say I want to load FFTW, then it's clear how to use pkg-config. When there are many choices — Intel MKL with its many variants, OpenBLAS, BLIS, IBM ESSL with its variants, Cray libsci, Apple Accelerate, etc., some that have pkg-config or cmake configs, some that don't — it's less clear to me what that looks like.

Our early hope was that environment modules would deal with this for us. The user would load whatever modules they want, then MKL, OpenBLAS, etc. are in their various paths. This doesn't turn out to work very well — in many cases, module load foo for some library foo doesn't add it to all the necessary paths.

[VictorEijkhout]

Got it. How about

cmake -Dblas=mkl -DBLAS_INT_SIZE=64 -DBLAS_THREADING=sequential

?
That should also make it orthogonal if you support other blasses.

Environment modules are a mechanism, not a standard vocabulary. They differ greatly between systems. On my system I provide

module load petsc/3.18
module load petsc/3.18-i64
module load petsc/3.18-complex

et cetera. Different module for each configuration.

If a module doesn't set all paths, that's on the module writer. We have an mkl module to be loaded when the compiler is gcc. I've just asked the module maintainer to add PKG_CONFIG_PATH to that module. (Would be nice if you started using that otherwise I look silly having asked for that :-)

[wavefunction91]

@mgates3 Reading through this, we've solved the majority of these problems in
https://github.com/wavefunction91/linalg-cmake-modules

When we include blaspp via FetchContent we do the following

# Download linalg-cmake-modules
# Append proper directories to CMAKE_MODULE_PATH

find_package( BLAS REQUIRED ) # This calls our `FindBLAS` overload and caches `BLAS_LIBRARIES` appropriately

# Download blaspp
FetchContent_MakeAvailable( blaspp )

The same (general) procedure works for find_package(blaspp) as well since BLAS_LIBRARIES persists into your blaspp-config calls.

This is a really long way of saying that we have tested that blaspp (and lapackpp for that matter) are compatible with our modules. The analogous call for the MKL discovery that @VictorEijkhout wants would be

set( BLAS_THREAD_LAYER "sequential" )
set( BLAS_PREFERENCE_LIST "IntelMKL") # Not required based on default preference, but it would cause a failure if MKL wasn't found
find_package( BLAS REQUIRED COMPONENTS ilp64 ) # ILP64 == INT64 BLAS

The set calls can be replaced by cmake -DVAR=VAL, and the ilp64 spec can get replaced by something BLAS_INTEGER_TYPE that is passed in, we just treat it like a component since we typically populate all MKL linkers and only pass back the one that was requested to the BLAS_XYZ interfaces.