XRootD tests are divided into two main categories: unit and integration tests that can be run directly with CTest, and containerized tests that are required to be run from within a container built with docker or podman. This document describes how to run the former, that is, the tests that are run just with CTest. This document assumes you are already familiar with how to build XRootD from source. If you need instructions on how to do that, please see the INSTALL.md file. There you will also find a full list of optional features and which dependencies are required to enable them.
XRootD unit and integration tests are enabled via the CMake configuration option
-DENABLE_TESTS=ON. Unit and integration tests may depend on GoogleTest.
Therefore, the development packages for GoogleTest (i.e. gtest-devel or
equivalent) are needed in order to enable all available tests. Here we discuss
how to use the test.cmake CTest script to run all steps to
configure and build the project, then run all tests using CTest. The script
test.cmake can be generically called from the top directory of
the repository as shown below
xrootd $ ctest -V -S test.cmake
-- Using CMake cache file config.cmake
Run dashboard with model Experimental
Source directory: xrootd
Build directory: xrootd/build
Reading ctest configuration file: xrootd/CTestConfig.cmake
Site: example.cern.ch (Linux - x86_64)
Build name: Linux GCC 12.3.1 RelWithDebInfo
Use Experimental tag: 20230622-0712
Updating the repository: xrootd
Use GIT repository type
Old revision of repository is: 6fce466a5f9b369f45ef2592c2ae246de1f13103
New revision of repository is: 6fce466a5f9b369f45ef2592c2ae246de1f13103
Gathering version information (one . per revision):
Configure project
Each . represents 1024 bytes of output
..... Size of output: 4K
Build project
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
.................................................. Size: 49K
.. Size of output: 52K
0 Compiler errors
0 Compiler warnings
Test project xrootd/build
Start 1: XrdCl::URLTest.LocalURLs
1/23 Test #1: XrdCl::URLTest.LocalURLs ..................... Passed 0.01 sec
Start 2: XrdCl::URLTest.RemoteURLs
2/23 Test #2: XrdCl::URLTest.RemoteURLs .................... Passed 0.12 sec
Start 3: XrdCl::URLTest.InvalidURLs
3/23 Test #3: XrdCl::URLTest.InvalidURLs ................... Passed 0.01 sec
Start 4: XrdHttpTests.checksumHandlerTests
4/23 Test #4: XrdHttpTests.checksumHandlerTests ............ Passed 0.01 sec
Start 5: XrdHttpTests.checksumHandlerSelectionTest
5/23 Test #5: XrdHttpTests.checksumHandlerSelectionTest .... Passed 0.01 sec
Start 6: XrdCl::Poller
6/23 Test #6: XrdCl::Poller ................................ Passed 5.01 sec
Start 7: XrdCl::Socket
7/23 Test #7: XrdCl::Socket ................................ Passed 0.02 sec
Start 8: XrdCl::Utils
8/23 Test #8: XrdCl::Utils ................................. Passed 8.01 sec
Start 9: XrdEc::AlignedWriteTest
9/23 Test #9: XrdEc::AlignedWriteTest ...................... Passed 0.06 sec
Start 10: XrdEc::SmallWriteTest
10/23 Test #10: XrdEc::SmallWriteTest ........................ Passed 0.06 sec
Start 11: XrdEc::BigWriteTest
11/23 Test #11: XrdEc::BigWriteTest .......................... Passed 0.05 sec
Start 12: XrdEc::VectorReadTest
12/23 Test #12: XrdEc::VectorReadTest ........................ Passed 0.06 sec
Start 13: XrdEc::IllegalVectorReadTest
13/23 Test #13: XrdEc::IllegalVectorReadTest ................. Passed 0.06 sec
Start 14: XrdEc::AlignedWrite1MissingTest
14/23 Test #14: XrdEc::AlignedWrite1MissingTest .............. Passed 0.06 sec
Start 15: XrdEc::AlignedWrite2MissingTest
15/23 Test #15: XrdEc::AlignedWrite2MissingTest .............. Passed 0.05 sec
Start 16: XrdEc::AlignedWriteTestIsalCrcNoMt
16/23 Test #16: XrdEc::AlignedWriteTestIsalCrcNoMt ........... Passed 0.06 sec
Start 17: XrdEc::SmallWriteTestIsalCrcNoMt
17/23 Test #17: XrdEc::SmallWriteTestIsalCrcNoMt ............. Passed 0.06 sec
Start 18: XrdEc::BigWriteTestIsalCrcNoMt
18/23 Test #18: XrdEc::BigWriteTestIsalCrcNoMt ............... Passed 0.06 sec
Start 19: XrdEc::AlignedWrite1MissingTestIsalCrcNoMt
19/23 Test #19: XrdEc::AlignedWrite1MissingTestIsalCrcNoMt ... Passed 0.06 sec
Start 20: XrdEc::AlignedWrite2MissingTestIsalCrcNoMt
20/23 Test #20: XrdEc::AlignedWrite2MissingTestIsalCrcNoMt ... Passed 0.06 sec
Start 21: XRootD::start
21/23 Test #21: XRootD::start ................................ Passed 0.01 sec
Start 23: XRootD::smoke-test
22/23 Test #23: XRootD::smoke-test ........................... Passed 1.63 sec
Start 22: XRootD::stop
23/23 Test #22: XRootD::stop ................................. Passed 1.00 sec
100% tests passed, 0 tests failed out of 23
Total Test time (real) = 16.55 secFor full verbose output, use -VV instead of -V. We recommend using at least -V
to add some verbosity. The output is too terse to be useful otherwise.
Since the script is targeted for usage with continuous integration, it tries to
load a configuration file from the .ci subdirectory in the source directory.
The default configuration is in the config.cmake file. This file is used to
pre-load the CMake cache. If found, it is passed to CMake during configuration
via the -C option. This file is a CMake script that should only contain CMake
set() commands using the CACHE option to populate the cache. Some effort is
made to detect and use a more specific configuration file than the generic
config.cmake that is used by default. For example, on Ubuntu, a file named
ubuntu.cmake will be used if present. The script also tries to detect the
version of the OS and use a more specific file if found for that version. For
example, on Alma Linux 8, one could use almalinux8.cmake which would have
higher precedence than almalinux.cmake. The default config.cmake file will
enable as many options as possible without failing if the dependencies are not
installed, so it should be sufficient in most cases.
The behavior of the test.cmake script can also be influenced
by environment variables like CC, CXX, CXXFLAGS, CMAKE_ARGS, CMAKE_GENERATOR,
CMAKE_BUILD_PARALLEL_LEVEL, CTEST_PARALLEL_LEVEL, and CTEST_CONFIGURATION_TYPE.
These are mostly self-explanatory and can be used to override the provided defaults.
For example, to build with clang and use ninja as CMake generator, one can run:
xrootd $ env CC=clang CXX=clang++ CMAKE_GENERATOR=Ninja ctest -V -S test.cmakeFor performance analysis and profiling with perf, we recommend building with
xrootd $ CXXFLAGS='-fno-omit-frame-pointer' ctest -V -C RelWithDebInfo -S test.cmakeFor enabling link-time optimizations (LTO), we recommend using
CXXFLAGS='-flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing'
This turns some important warnings into errors to avoid potential runtime issues
with LTO. Please see GCC's manual page for descriptions of each of the warnings
above. XRootD also support using address and thread sanitizers, via the options
-DENABLE_ASAN=ON and -DENABLE_TSAN=ON, respectively. These should be enabled
using CMAKE_ARGS, as shown below
$ env CMAKE_ARGS="-DENABLE_TSAN=1" ctest -V -S test.cmakeNote that options passed by setting CMAKE_ARGS in the environment have higher
precedence than what is in the pre-loaded cache file, so this method can be used
to override the defaults without having to edit the pre-loaded cache file.
The test.cmake has are several options that allow the developer to customize the build being tested. The main options are shown in the table below:
| Option | Description |
|---|---|
| -DCOVERAGE=ON | Enables test coverage analysis with gcov |
| -DMEMCHECK=ON | Enables memory checking with valgrind |
| -DSTATIC_ANALYSIS=ON | Enables static analysis with clang-tidy |
| -DINSTALL=ON | Enables an extra step to call make install |
When enabling coverage, a report is generated by default in the html/
directory inside the build directory. The results can then be viewed by
opening the file html/coverage_details.html in a web browser. The report
generation step can be disabled by passing the option -DGCOVR=0 to the
script. If gcovr is not found, the step will be skipped automatically.
It is recommended to use a debug build to generate the coverage analysis.
The CMake build type can be specified directly on the command line with the
-C option of ctest. For example, to run a coverage build in debug mode,
showing test output when a test failure happens, one can run:
xrootd $ ctest -V --output-on-failure -C Debug -DCOVERAGE=ON -S test.cmakeMemory checking is performed with valgrind when it is enabled. In this case,
the tests are run twice, once as usual, and once with valgrind. The output
logs from running the tests with valgrind can be found in the build directory
at build/Testing/Temporary/MemoryChecker.<#>.log where <#> corresponds to
the test number as shown when running ctest.
Static analysis requires clang-tidy and is enabled by setting CMAKE_CXX_CLANG_TIDY
for the build. If clang-tidy is not in the standard PATH, then it may be
necessary to set it manually instead of using the option -DSTATIC_ANALYSIS=ON.
For the moment XRootD does not provide yet its own configuration file for
clang-tidy, so the defaults will be used for the build. Warnings and errors
coming from static analysis will be shown as part of the regular build, so it is
important to enable full verbosity when enabling static analysis to be able to
see the output from clang-tidy.
The option -DINSTALL=ON will enable a step to perform a so-called staged
installation inside the build directory. It can be used to check if the
installation procedure is working as expected, by inspecting the contents of the
install/ directory inside the build directory after installation:
xrootd $ ctest -DINSTALL=1 -S test.cmake
Each . represents 1024 bytes of output
..... Size of output: 4K
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
.................................................. Size: 49K
.. Size of output: 52K
Each symbol represents 1024 bytes of output.
'!' represents an error and '*' a warning.
............................................*!.... Size: 49K
. Size of output: 50K
Error(s) when building project
xrootd $ tree -L 3 -d build/install
build/install
└── usr
├── bin
├── include
│ └── xrootd
├── lib
│ └── python3.11
├── lib64
└── share
├── man
└── xrootd
11 directoriesNote that, as shown above, CTest erroneously shows build errors when
installing XRootD with this command. This is because of a deprecation
warning emitted by pip while installing the Python bindings and can be
safely ignored.
The test.cmake script may also need some extra dependencies for
some of its features. For memory checking, valgrind is needed, and for static
analysis, clang-tidy is needed:
dnf install \
clang \
clang-tools-extra \
valgrindFor coverage, you need to install gcovr. It is not available via yum/dnf,
but can be installed with pip. See https://gcovr.com/en/stable/installation.html
for more information.
Dependencies to run containerized tests with podman on RHEL 8/9 and derivatives
can be installed with dnf groupinstall 'Container Management'. A quick test to check
if everything is correctly setup is to try to start a busybox image with:
podman run --rm -it busyboxOn Debian, Ubuntu, and derivatives, The extra dependencies to use with the test.cmake script can be installed with:
apt install clang clang-tidy valgrind gcovrDependencies to run containerized tests can be installed with
apt install podmanIf you would like to run XRootD tests on other platforms, you can use
the xrd-docker script and associated Dockerfiles in the docker/
subdirectory. The steps needed are described below.
The first thing that needs to be done is packaging a tarball with the version
of XRootD to be used to build in the container image. The command xrd-docker package
by default creates a tarball named xrootd.tar.gz in the current directory using the
HEAD of the currently checked branch. We recommend changing directory to the docker/
directory in the XRootD git repository in order to run these commands. Suppose
we would like to run the tests for release v5.6.4. Then, we would run
$ xrd-docker package v5.6.4to create the tarball that will be used to build the container image. The
tarball created by this command is a standard tarball created with git archive.
Inside it, the VERSION file contains the expanded version which is used by the
new spec file to detect the version of XRootD being built. You can also create a
source RPM with such tarballs, but they must be built with rpmbuild --with git
as done in the CI builds and the Dockerfiles in the docker/build/ subdirectory.
The next step is to build the container image for the desired OS. It can be built
with either docker or podman. The xrd-docker script has the build command
to facilitate this. Currently, supported OSs for building are AlmaLinux 8, 9 and
10, Fedora, Alpine, Debian, and Ubuntu. The command to build the image is simply
$ xrd-docker build <OS>where <OS> is one of alma9 (default), alma8, fedora, alpine, debian,
or ubuntu. The name simply chooses which Dockerfile is used from the build/
directory, as they are named Dockerfile.<OS> for each suported OS. It is possible
to add new Dockerfiles following this same naming scheme to support custom setups
and still use xrd-docker build command to build an image. The images built with
xrd-docker build are named simply xrootd (latest being a default tag added
by docker), and an extra xrootd:<OS> tag is added to allow having it built for
multiple OSs at the same time. The current Dockerfiles use the spec file and
build the image using the RPM packaging workflow, installing dependencies as
declared in the spec file, in the first stage, building the RPMs in a second
stage, then, in a third stage starting from a fresh image, the RPMs built in
stage 2 are copied over and installed with yum or dnf.
The xrd-docker script takes either docker or podman if available, in this
order. If you have only one of the two installed, everything should work without
any extra setup, but if you have both installed and would like to use podman
instead of docker for building the images, it can be done by exporting an
environment variable:
$ export DOCKER=$(command -v podman)
$ xrd-docker build # uses podman from now on...Unlike docker, podman may not work out of the box after installing it. If it
doesn't, make sure that you have subuids and subgids setup for your user by
running, for example, the two commands below:
$ sudo usermod --add-subuids 1000000-1000999999 $(id -un)
$ sudo usermod --add-subgids 1000000-1000999999 $(id -un)You may also have to ensure that container registries in
/etc/containers/registries.conf. Usually a usable configuration can be renamed
from /etc/containers/registries.conf.example.
Finally, you may want to try container runtimes other than the default. If you
still have problems getting started, podman's documentation can be found at
https://podman.io/docs.