nextest-filtering 0.2.0

Added

• The expression language supports several new predicates:
  • kind(name-matcher): include all tests in binary kinds (e.g. lib, test, bench) matching name-matcher.
  • binary(name-matcher): include all tests in binary names matching name-matcher.
  • platform(host) or platform(target): include all tests that are built for the host or target platform, respectively.

• It is now possible to evaluate a query without knowing the name of the test. The result is evaluated as a three-valued logic (Kleene K3), and is returned as an Option<bool> where None indicates the unknown state.

Changed

• The evaluator now takes a TestQuery struct, making it easier to add more parameters in the future.
• MSRV updated to Rust 1.59.

cargo-nextest 0.9.25

This is a major release with several new features.

Filter expressions

Filter expressions are now ready for production. For example, to run all tests in nextest-runner and all its transitive dependencies within the workspace:

cargo nextest run -E 'deps(nextest-runner)'

This release includes a number of additions and changes to filter expressions.

Added

• The expression language supports several new predicates:
  • kind(name-matcher): include all tests in binary kinds (e.g. lib, test, bench) matching name-matcher.
  • binary(name-matcher): include all tests in binary names matching name-matcher.
  • platform(host) or platform(target): include all tests that are built for the host or target platform, respectively.

Changed

• If a filter expression is guaranteed not to match a particular binary, it will not be listed by nextest. (This allows platform(host) and platform(target) to work correctly.)

• If both filter expressions and standard substring filters are passed in, a test must match filter expressions AND substring filters to be executed. For example:

cargo nextest run -E 'package(nextest-runner)' test_foo test_bar

This will execute only the tests in nextest-runner that match test_foo or test_bar.

Per-test overrides

Nextest now supports per-test overrides. These overrides let you customize settings for subsets of tests. For example, to retry tests that contain the substring test_e2e 3 times:

[[profile.default.overrides]]
filter = "test(test_e2e)"
retries = 3

Currently, only retries are supported. In the future, more kinds of customization will be added.

Other changes

• A new environment variable NEXTEST_RETRIES controls the number of retries tests are run with. In terms of precedence, this slots in between the command-line --retries option and per-test overrides for retries.
• cargo nextest list now hides skipped tests and binaries by default. To print out skipped tests and binaries, use cargo nextest list --verbose.
• The Machine-readable output for cargo nextest list now contains a new "status" key. By default, this is set to "listed", and for binaries that aren't run because they don't match expression filters this is set to "skipped".
• The --platform-filter option is deprecated, though it will keep working for all versions within the nextest 0.9 series. Use -E 'platform(host)' or -E 'platform(target)' instead.
• cargo nextest run -- --skip and --exact now suggest using a filter expression instead.

nextest-runner 0.15.0

Added

• New config option profile.<profile-name>.test-threads controls the number of tests run simultaneously. This option accepts either an integer with the number of threads, or the string "num-cpus" (default) for the number of logical CPUs.

Fixed

• Within JUnit XML, test failure descriptions (text nodes for <failure> and <error> tags) now have invalid ANSI escape codes stripped from their output.

cargo-nextest 0.9.24

Added

• New config option profile.<profile-name>.test-threads controls the number of tests run simultaneously. This option accepts either an integer with the number of threads, or the string "num-cpus" (default) for the number of logical CPUs. As usual, this option is overridden by --test-threads and NEXTEST_TEST_THREADS, in that order.
• The command-line --test-threads option and the NEXTEST_TEST_THREADS environment variable now accept num-cpus as their argument.
• nextest now works with cargo binstall (#332). Thanks [Remoun] for your first contribution!

Fixed

• Within JUnit XML, test failure descriptions (text nodes for <failure> and <error> tags) now have invalid ANSI escape codes stripped from their output.

nextest-runner 0.14.0

Added

• On Windows, nextest now detects tests that abort due to e.g. an access violation (segfault) and prints their status as "ABORT" rather than "FAIL", along with an explanatory message on the next line.
• Improved JUnit support: nextest now heuristically detects stack traces and adds them to the text node of the <failure> element (#311).

nextest-metadata 0.4.3

Added

• New documented exit code WRITE_OUTPUT_ERROR.

cargo-nextest 0.9.23

Added

• On Windows, nextest now detects tests that abort due to e.g. an access violation (segfault) and prints their status as "ABORT" rather than "FAIL", along with an explanatory message on the next line.
• Improved JUnit support: nextest now heuristically detects stack traces and adds them to the text node of the <failure> element (#311).

Changed

• Errors that happen while writing data to the output now have a new documented exit code: WRITE_OUTPUT_ERROR: https://docs.rs/nextest-metadata/latest/nextest_metadata/enum.NextestExitCode.html#associatedconstant.WRITE_OUTPUT_ERROR.

quick-junit 0.2.0

Changed

• quick-xml updated to 0.23.0.
• The error type is now defined by quick-junit, so that future breaking changes to quick-xml will not necessitate a breaking change to this crate.
• MSRV bumped to Rust 1.59.

nextest-runner 0.13.0

Added

• Benchmarks are now treated as normal tests. (#283, thanks @tabokie for your contribution!).

  Note that criterion.rs benchmarks are currently incompatible with nextest (#96) -- this change doesn't have any effect on that.

Changed

• If nextest's output is colorized, it no longer strips ANSI escape codes from test runs.
• quick-junit updated to 0.2.0.

cargo-nextest 0.9.22

Added

• Benchmarks are now treated as normal tests. (#283, thanks @tabokie for your contribution!).

  Note that criterion.rs benchmarks are currently incompatible with nextest (#96) -- this change doesn't have any effect on that.

• Added -F as a shortcut for --features, mirroring an upcoming addition to Cargo 1.62 (#287, thanks Alexendoo for your first contribution!)

Changed

• If nextest's output is colorized, it no longer strips ANSI escape codes from test runs.