oneAPI Math Kernel Library (oneMKL) Interfaces

oneMKL Interfaces is an open-source implementation of the oneMKL Data Parallel C++ (DPC++) interface according to the oneMKL specification. It works with multiple devices (backends) using device-specific libraries underneath.

oneMKL is part of oneAPI.

User Application	oneMKL Layer	Third-Party Library	Hardware Backend
oneMKL interface	oneMKL selector	Intel(R) oneAPI Math Kernel Library for x86 CPU	x86 CPU
		Intel(R) oneAPI Math Kernel Library for Intel GPU	Intel GPU
		NVIDIA cuBLAS for NVIDIA GPU	NVIDIA GPU
		NVIDIA cuSOLVER for NVIDIA GPU	NVIDIA GPU
		NVIDIA cuRAND for NVIDIA GPU	NVIDIA GPU
		NVIDIA cuFFT for NVIDIA GPU	NVIDIA GPU
		NETLIB LAPACK for x86 CPU	x86 CPU
		AMD rocBLAS for AMD GPU	AMD GPU
		AMD rocSOLVER for AMD GPU	AMD GPU
		AMD rocRAND for AMD GPU	AMD GPU
		AMD rocFFT for AMD GPU	AMD GPU
		portBLAS	x86 CPU, Intel GPU, NVIDIA GPU, AMD GPU
		portFFT	x86 CPU, Intel GPU, NVIDIA GPU, AMD GPU

Table of Contents

• Support and Requirements
• Documentation
• FAQs
• Legal Information

---

Support and Requirements

Supported Usage Models:

Host API

There are two oneMKL selector layer implementations:

• Run-time dispatching: The application is linked with the oneMKL library and the required backend is loaded at run-time based on device vendor (all libraries should be dynamic).

Example of app.cpp with run-time dispatching:

#include "oneapi/mkl.hpp"

...
cpu_dev = sycl::device(sycl::cpu_selector());
gpu_dev = sycl::device(sycl::gpu_selector());

sycl::queue cpu_queue(cpu_dev);
sycl::queue gpu_queue(gpu_dev);

oneapi::mkl::blas::column_major::gemm(cpu_queue, transA, transB, m, ...);
oneapi::mkl::blas::column_major::gemm(gpu_queue, transA, transB, m, ...);

How to build an application with run-time dispatching:

if OS is Linux, use icpx compiler. If OS is Windows, use icx compiler. Linux example:

$> icpx -fsycl –I$ONEMKL/include app.cpp
$> icpx -fsycl app.o –L$ONEMKL/lib –lonemkl

• Compile-time dispatching: The application uses a templated backend selector API where the template parameters specify the required backends and third-party libraries and the application is linked with the required oneMKL backend wrapper libraries (libraries can be static or dynamic).

Example of app.cpp with compile-time dispatching:

#include "oneapi/mkl.hpp"

...
cpu_dev = sycl::device(sycl::cpu_selector());
gpu_dev = sycl::device(sycl::gpu_selector());

sycl::queue cpu_queue(cpu_dev);
sycl::queue gpu_queue(gpu_dev);

oneapi::mkl::backend_selector<oneapi::mkl::backend::mklcpu> cpu_selector(cpu_queue);

oneapi::mkl::blas::column_major::gemm(cpu_selector, transA, transB, m, ...);
oneapi::mkl::blas::column_major::gemm(oneapi::mkl::backend_selector<oneapi::mkl::backend::cublas> {gpu_queue}, transA, transB, m, ...);

How to build an application with compile-time dispatching:

$> clang++ -fsycl –I$ONEMKL/include app.cpp
$> clang++ -fsycl app.o –L$ONEMKL/lib –lonemkl_blas_mklcpu –lonemkl_blas_cublas

Refer to Selecting a Compiler for the choice between icpx/icx and clang++ compilers.

Device API

Header-based and backend-independent Device API can be called within sycl kernel or work from Host code (device-rng-usage-model-example). Currently, the following domains support the Device API:

• RNG. To use RNG Device API functionality it's required to include oneapi/mkl/rng/device.hpp header file.

Supported Configurations:

Supported domains: BLAS, LAPACK, RNG, DFT, SPARSE_BLAS

Linux*

Domain	Backend	Library	Supported Link Type	Supported Compiler
BLAS	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*, hipSYCL
	Intel GPU		Dynamic, Static	DPC++
	NVIDIA GPU	NVIDIA cuBLAS	Dynamic, Static	LLVM*, hipSYCL
	x86 CPU	NETLIB LAPACK	Dynamic, Static	DPC++, LLVM*, hipSYCL
	AMD GPU	AMD rocBLAS	Dynamic, Static	LLVM*, hipSYCL
	x86 CPU, Intel GPU, NVIDIA GPU, AMD GPU	portBLAS	Dynamic, Static	DPC++, LLVM*
LAPACK	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*
	Intel GPU		Dynamic, Static	DPC++
	NVIDIA GPU	NVIDIA cuSOLVER	Dynamic, Static	LLVM*
	AMD GPU	AMD rocSOLVER	Dynamic, Static	LLVM*
RNG	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*, hipSYCL
	Intel GPU		Dynamic, Static	DPC++
	NVIDIA GPU	NVIDIA cuRAND	Dynamic, Static	LLVM*, hipSYCL
	AMD GPU	AMD rocRAND	Dynamic, Static	LLVM*, hipSYCL
DFT	Intel GPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++
	x86 CPU		Dynamic, Static	DPC++
	NVIDIA GPU	NVIDIA cuFFT	Dynamic, Static	DPC++
	AMD GPU	AMD rocFFT	Dynamic, Static	DPC++
	x86 CPU, Intel GPU, NVIDIA GPU, AMD GPU	portFFT (limited API support)	Dynamic, Static	DPC++
SPARSE_BLAS	Intel GPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++
	x86 CPU		Dynamic, Static	DPC++

Windows*

Domain	Backend	Library	Supported Link Type	Supported Compiler
BLAS	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*
	Intel GPU		Dynamic, Static	DPC++
	x86 CPU	NETLIB LAPACK	Dynamic, Static	DPC++, LLVM*
LAPACK	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*
	Intel GPU		Dynamic, Static	DPC++
RNG	x86 CPU	Intel(R) oneAPI Math Kernel Library	Dynamic, Static	DPC++, LLVM*
	Intel GPU		Dynamic, Static	DPC++

* LLVM - Intel project for LLVM* technology with support for NVIDIA CUDA

---

Hardware Platform Support

• CPU
  • Intel Atom(R) Processors
  • Intel(R) Core(TM) Processor Family
  • Intel(R) Xeon(R) Processor Family
• Accelerators
  • Intel(R) Arc(TM) A-Series Graphics
  • Intel(R) Data Center GPU Max Series
  • NVIDIA(R) A100 (Linux* only)
  • AMD(R) GPUs see here tested on AMD Vega 20 (gfx906)

---

Supported Operating Systems

Linux*

Operating System	CPU Host/Target	Integrated Graphics from Intel (Intel GPU)	NVIDIA GPU
Ubuntu	18.04.3, 19.04	18.04.3, 19.10	18.04.3, 20.04
SUSE Linux Enterprise Server*	15	Not supported	Not supported
Red Hat Enterprise Linux* (RHEL*)	8	Not supported	Not supported
Linux* kernel	N/A	4.11 or higher	N/A

Windows*

Operating System	CPU Host/Target	Integrated Graphics from Intel (Intel GPU)
Microsoft Windows*	10 (64-bit version only)	10 (64-bit version only)
Microsoft Windows* Server	2016, 2019	Not supported

---

Software Requirements

What should I download?

General:

Using Conan	Using CMake Directly
	Functional Testing	Build Only	Documentation
Linux* : GNU* GCC 5.1 or higher Windows* : MSVS* 2017 or MSVS* 2019 (version 16.5 or newer)
Python 3.6 or higher	CMake
	Ninja (optional)
Conan C++ package manager	GNU* FORTRAN Compiler	-	Sphinx
	NETLIB LAPACK	-	-

Hardware and OS Specific:

Operating System	Device	Package	Installed by Conan
Linux*/Windows*	x86 CPU	Intel(R) oneAPI DPC++ Compiler or Intel project for LLVM* technology	No
		Intel(R) oneAPI Math Kernel Library	Yes
	Intel GPU	Intel(R) oneAPI DPC++ Compiler	No
		Intel GPU driver	No
		Intel(R) oneAPI Math Kernel Library	Yes
Linux* only	NVIDIA GPU	Intel project for LLVM* technology or hipSYCL with CUDA backend and dependencies	No
	AMD GPU	Intel project for LLVM* technology or hipSYCL with ROCm backend and dependencies	No

If Building with Conan, above packages marked as "No" must be installed manually.

If Building with CMake, above packages must be installed manually.

Notice for Use of Conan Package Manager

LEGAL NOTICE: By downloading and using this container or script as applicable (the "Software Package") and the included software or software made available for download, you agree to the terms and conditions of the software license agreements for the Software Package, which may also include notices, disclaimers, or license terms for third party software (together, the "Agreements") included in this README file.

If the Software Package is installed through a silent install, your download and use of the Software Package indicates your acceptance of the Agreements.

Product and Version Information:

Product	Supported Version	Installed by Conan	Conan Package Source	Package Install Location on Linux*	License
Python	3.6 or higher	No	N/A	Pre-installed or Installed by user	PSF
Conan C++ Package Manager	1.24 or higher	No	N/A	Installed by user	MIT
CMake	3.13 or higher	Yes (3.15 or higher)	conan-center	~/.conan/data or $CONAN_USER_HOME/.conan/data	The OSI-approved BSD 3-clause License
Ninja	1.10.0	Yes	conan-center	~/.conan/data or $CONAN_USER_HOME/.conan/data	Apache License v2.0
GNU* FORTRAN Compiler	7.4.0 or higher	Yes	apt	/usr/bin	GNU General Public License, version 3
Intel(R) oneAPI DPC++ Compiler	latest	No	N/A	Installed by user	End User License Agreement for the Intel(R) Software Development Products
hipSYCL	later than 2cfa530	No	N/A	Installed by user	BSD-2-Clause License
Intel project for LLVM* technology binary for x86 CPU	Daily builds	No	N/A	Installed by user	Apache License v2
Intel project for LLVM* technology source for NVIDIA and AMD GPUs	Daily source releases	No	N/A	Installed by user	Apache License v2
Intel(R) oneAPI Math Kernel Library	latest	Yes	apt	/opt/intel/inteloneapi/mkl	Intel Simplified Software License
NVIDIA CUDA SDK	10.2	No	N/A	Installed by user	End User License Agreement
AMD rocBLAS	4.5	No	N/A	Installed by user	AMD License
AMD rocRAND	5.1.0	No	N/A	Installed by user	AMD License
AMD rocSOLVER	5.0.0	No	N/A	Installed by user	AMD License
AMD rocFFT	rocm-5.4.3	No	N/A	Installed by user	AMD License
NETLIB LAPACK	3.7.1	Yes	conan-community	~/.conan/data or $CONAN_USER_HOME/.conan/data	BSD like license
Sphinx	2.4.4	Yes	pip	~/.local/bin (or similar user local directory)	BSD License
portBLAS	0.1	No	N/A	Installed by user	Apache License v2.0
portFFT	0.1	No	N/A	Installed by user or automatically by cmake	Apache License v2.0

conan-center: https://api.bintray.com/conan/conan/conan-center

conan-community: https://api.bintray.com/conan/conan-community/conan

---

Documentation

• Contents
• About
• Get Started
  • Selecting a Compiler
  • Building the Project
• Developer Reference
  • oneMKL Defined Datatypes
  • Dense Linear Algebra
• Integrating a Third-Party Library

---

Contributing

See CONTRIBUTING for more information.

---

License

Distributed under the Apache license 2.0. See [LICENSE](LICENSE) for more

information.

---

FAQs

oneMKL

1. What is the difference between the following oneMKL items?
   • The oneAPI Specification for oneMKL
   • The oneAPI Math Kernel Library (oneMKL) Interfaces Project
   • The Intel(R) oneAPI Math Kernel Library (oneMKL) Product

Answer:

• The oneAPI Specification for oneMKL defines the DPC++ interfaces for performance math library functions. The oneMKL specification can evolve faster and more frequently than implementations of the specification.

• The oneAPI Math Kernel Library (oneMKL) Interfaces Project is an open source implementation of the specification. The project goal is to demonstrate how the DPC++ interfaces documented in the oneMKL specification can be implemented for any math library and work for any target hardware. While the implementation provided here may not yet be the full implementation of the specification, the goal is to build it out over time. We encourage the community to contribute to this project and help to extend support to multiple hardware targets and other math libraries.

• The Intel(R) oneAPI Math Kernel Library (oneMKL) product is the Intel product implementation of the specification (with DPC++ interfaces) as well as similar functionality with C and Fortran interfaces, and is provided as part of Intel® oneAPI Base Toolkit. It is highly optimized for Intel CPU and Intel GPU hardware.

2. I'm trying to use oneMKL Interfaces in my project using FetchContent, but I keep running into ONEMKL::SYCL::SYCL target was not found problem when I try to build the project. What should I do?

Answer:

Make sure you set the compiler when you configure your project. E.g. cmake -Bbuild . -DCMAKE_CXX_COMPILER=icpx.

3. I'm trying to use oneMKL Interfaces in my project using find_package(oneMKL). I set oneMKL/oneTBB and Compiler environment first, then I built and installed oneMKL Interfaces, and finally I tried to build my project using installed oneMKL Interfaces (e.g. like this cmake -Bbuild -GNinja -DCMAKE_CXX_COMPILER=icpx -DoneMKL_ROOT=<path_to_installed_oneMKL_interfaces> .) and I noticed that cmake includes installed oneMKL Interfaces headers as a system include which ends up as a lower priority than the installed oneMKL package includes which I set before for building oneMKL Interfaces. As a result, I get conflicts between oneMKL and installed oneMKL Interfaces headers. What should I do?

Answer:

Having installed oneMKL Interfaces headers as -I instead on system includes (as -isystem) helps to resolve this problem. We use INTERFACE_INCLUDE_DIRECTORIES to add paths to installed oneMKL Interfaces headers (check oneMKLTargets.cmake in lib/cmake to find it). It's a known limitation that INTERFACE_INCLUDE_DIRECTORIES puts headers paths as system headers. To avoid that:

• Option 1: Use CMake >=3.25. In this case oneMKL Interfaces will be built with EXPORT_NO_SYSTEM property set to true and you won't see the issue.
• Option 2: If you use CMake < 3.25, set PROPERTIES NO_SYSTEM_FROM_IMPORTED true for your target. E.g: set_target_properties(test PROPERTIES NO_SYSTEM_FROM_IMPORTED true).

Conan

1. I am behind a proxy. How can Conan download dependencies from external network?

   • ~/.conan/conan.conf has a [proxies] section where you can add the list of proxies. For details refer to Conan proxy settings.

2. I get an error while installing packages via APT through Conan.

   dpkg: warning: failed to open configuration file '~/.dpkg.cfg' for reading: Permission denied
   Setting up intel-oneapi-mkl-devel (2021.1-408.beta07) ...
   E: Sub-process /usr/bin/dpkg returned an error code (1)
   

   • Although your user session has permissions to install packages via sudo apt, it does not have permissions to update debian package configuration, which throws an error code 1, causing a failure in conan install command.
   • The package is most likely installed correctly and can be verified by:
     1. Running the conan install command again.
     2. Checking /opt/intel/inteloneapi for mkl and/or tbb directories.

---

Legal information