value-provider is a free library that facilitates writing realistic test data and in turn better tests for your Java application.
It works best in conjunction with reusable test data factories that encapsulate creating valid instances for your data objects.
value-provider consists of two major parts:
- the ValueProvider class which populates properties of test data objects with random values
- infrastructure for reproducing said random data in case of test failures (JUnit5 extension, JUnit4 rules)
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update the tests as appropriate.
For further information, please refer to CONTRIBUTING.md. For technical details, you may find the sequence diagrams in the doc directory helpful.
value-provider has the following prerequisites:
- Java 8 and above
- JUnit 5.5 and above for the JUnit5 infrastructure
- JUnit 4.12 and above for the JUnit4 infrastructure
// core library
testImplementation 'com.tngtech.valueprovider:value-provider-core:1.2.3'
// infrastructure
// for JUnit 5
testImplementation 'com.tngtech.valueprovider:value-provider-junit5:1.2.3'
// alternatively, for JUnit 4
testImplementation 'com.tngtech.valueprovider:value-provider-junit4:1.2.3'<!-- ... -->
<dependencies>
<!-- ... -->
<!-- core library -->
<dependency>
<groupId>com.tngtech.valueprovider</groupId>
<artifactId>value-provider-core</artifactId>
<version>1.2.3</version>
<scope>test</scope>
</dependency>
<!-- infrastructure -->
<!-- for JUnit 5 -->
<dependency>
<groupId>com.tngtech.valueprovider</groupId>
<artifactId>value-provider-junit5</artifactId>
<version>1.2.3</version>
<scope>test</scope>
</dependency>
<!-- alternatively, for JUnit 4 -->
<dependency>
<groupId>com.tngtech.valueprovider</groupId>
<artifactId>value-provider-junit4</artifactId>
<version>1.2.3</version>
<scope>test</scope>
</dependency>
<!-- ... -->
</dependencies>We strongly recommend implementing reusable test data factories that encapsulate creating valid instances for your test data objects.
Consider a simple Product1...:
// ...
@Getter
@ToString
@EqualsAndHashCode
@RequiredArgsConstructor(staticName = "of")
public class Product {
@NonNull
private final ProductCategory category;
@NonNull
private final String name;
@NonNull
private final String description;
}... the test data factory would look like... (see also ProductTestDataFactory):
import com.tngtech.valueprovider.ValueProvider;
import static com.tngtech.valueprovider.ValueProviderFactory.createRandomValueProvider;
public class ProductTestDataFactory {
private ProductTestDataFactory() {
}
public static Product createProduct() {
return createProduct(createRandomValueProvider());
}
public static Product createProduct(ValueProvider values) {
return Product.of(
values.oneOf(ProductCategory.class),
values.fixedDecoratedString("name"),
values.fixedDecoratedString("description"));
}
}The ValueProvider is used to
- select a ProductCategory at random
- populate the
nameanddescriptionproperties with a so called decorated string
What is a decorated string? Let's have a look at the following example output when invoking ProductTestDataFactory.createProduct() multiple times:
Product(category=CAR, name=nameaPr, description=descriptionaPr)
Product(category=COMPUTER, name=nameyBp, description=descriptionyBp)
Product(category=COMPUTER, name=namejeM, description=descriptionjeM)
The decoration is simply a 3 letter suffix that is appended to the base string provided as parameter to the fixedDecoratedString() method.
Note that the suffix of all String properties of one Product instance stays the same, since the values are populated using the
same ValueProvider instance. Conversely, different Product
instances contain properties with different suffixes. This is achieved by invoking createRandomValueProvider(), when calling
the ProductTestDataFactory.
Having seen the basics, let's move on to a more complex example. Consider an Order:
// ...
@Getter
@ToString
@EqualsAndHashCode
@Builder(toBuilder = true)
public class Order {
@Singular
@NonNull
private final ImmutableList<OrderItem> orderItems;
@NonNull
private final Customer customer;
@NonNull
private final Address shippingAddress;
@NonNull
private final Optional<Address> billingAddress;
public Address getBillingAddress() {
return billingAddress.orElse(shippingAddress);
}
// ...
}This time, let's start with the example output of invoking OrderTestDataFactory.createOrder():
Order(
orderItems=[
OrderItem(product=Product(category=COMPUTER, name=A-nameFhG, description=A-descriptionFhG), quantity=66),
OrderItem(product=Product(category=FOOD, name=B-nameFhG, description=B-descriptionFhG), quantity=8),
OrderItem(product=Product(category=COMPUTER, name=C-nameFhG, description=C-descriptionFhG), quantity=35),
OrderItem(product=Product(category=BOOK, name=D-nameFhG, description=D-descriptionFhG), quantity=89)],
customer=Customer(firstName=firstNameFhG, lastName=lastNameFhG, birthDate=1975-05-06),
shippingAddress=Address(zip=96874, city=S-cityFhG, street=S-streetFhG, number=315),
billingAddress=Address(zip=32924, city=B-cityFhG, street=B-streetFhG, number=120))
Order(
orderItems=[
OrderItem(product=Product(category=MAGIC_EQUIPMENT, name=A-namerwk, description=A-descriptionrwk), quantity=1),
OrderItem(product=Product(category=MAGIC_EQUIPMENT, name=B-namerwk, description=B-descriptionrwk), quantity=69),
OrderItem(product=Product(category=COMPUTER, name=C-namerwk, description=C-descriptionrwk), quantity=85)],
customer=Customer(firstName=firstNamerwk, lastName=lastNamerwk, birthDate=2002-06-08),
shippingAddress=Address(zip=08583, city=cityrwk, street=streetrwk, number=86),
billingAddress=Address(zip=08583, city=cityrwk, street=streetrwk, number=86))
Order(
orderItems=[
OrderItem(product=Product(category=COMPUTER, name=A-namekwh, description=A-descriptionkwh), quantity=73)],
customer=Customer(firstName=firstNamekwh, lastName=lastNamekwh, birthDate=1929-08-23),
shippingAddress=Address(zip=81571, city=S-citykwh, street=S-streetkwh, number=174),
billingAddress=Address(zip=71331, city=B-citykwh, street=B-streetkwh, number=169))
Note that the 3 letter suffix that we already saw in the Product example is shared for the entire hierarchy of objects that comprise an order. It therefore eases recognizing objects that belong together.
The output also demonstrates further aspects of randomization in test data factories. The 3 order objects all have a different number of order items, and e.g., the shipping and billing addresses have random zip codes or house numbers.
Last but not least, note the second aspect of string decoration. The products in the order items have an additional prefix in their string properties (e.g., 'A-', 'B-', ...). This is required to differentiate multiple objects of the same kind. The same applies to the shipping and billing addresses. They have different prefixes, if they differ ('S-' vs. 'B-'), but no prefix, if they are the same.
Let's take a look at OrderTestDataFactory - how to achieve all this:
import com.tngtech.valueprovider.example.Order.OrderBuilder;
import com.tngtech.valueprovider.ValueProvider;
import static com.tngtech.valueprovider.example.AddressTestDataFactory.createAddress;
import static com.tngtech.valueprovider.example.CustomerTestDataFactory.createCustomer;
import static com.tngtech.valueprovider.example.OrderItemTestDataFactory.createOrderItem;
import static com.tngtech.valueprovider.ValueProviderFactory.createRandomValueProvider;
public final class OrderTestDataFactory {
private OrderTestDataFactory() {
}
public static Order createOrder() {
return createOrder(createRandomValueProvider());
}
public static Order createOrder(ValueProvider values) {
return createOrderBuilder(values).build();
}
public static OrderBuilder createOrderBuilder() {
return createOrderBuilder(createRandomValueProvider());
}
public static OrderBuilder createOrderBuilder(ValueProvider values) {
OrderBuilder builder = Order.builder()
.customer(createCustomer(values));
setAddress(builder, values);
addItems(builder, values);
return builder;
}
private static void setAddress(OrderBuilder builder, ValueProvider values) {
boolean useDifferentBillingAddress = values.booleanValue();
if (useDifferentBillingAddress) {
builder
.shippingAddress(createAddress(values.copyWithChangedPrefix("S-")))
.billingAddress(createAddress(values.copyWithChangedPrefix("B-")));
} else {
builder
.shippingAddress(createAddress(values));
}
}
private static void addItems(OrderBuilder builder, ValueProvider values) {
int numOrderItems = values.intNumber(1, 5);
for (int i = 0; i < numOrderItems; i++) {
char prefix = (char) ('A' + i);
ValueProvider prefixedProvider = values.copyWithChangedPrefix("" + prefix + "-");
builder.orderItem(createOrderItem(prefixedProvider));
}
}
}Sharing the same suffix for the entire object hierarchy is easy, just pass the ValueProvider instance to each invoked test data factory as in:
// ...
public final class OrderTestDataFactory {
// ...
public static OrderBuilder createOrderBuilder(ValueProvider values) {
OrderBuilder builder = Order.builder()
.customer(createCustomer(values));
// ...
return builder;
}
// ...
}If you need multiple objects of the same type, add a prefix, like for the order item:
// ...
public final class OrderTestDataFactory {
// ...
private static void addItems(OrderBuilder builder, ValueProvider values) {
int numOrderItems = values.intNumber(1, 5);
for (int i = 0; i < numOrderItems; i++) {
char prefix = (char) ('A' + i);
ValueProvider prefixedProvider = values.copyWithChangedPrefix("" + prefix + "-");
builder.orderItem(createOrderItem(prefixedProvider));
}
}
// ...
}The copyWithChangedPrefix() method takes the suffix of the ValueProvider for which it is called, and creates a new instance with the passed
prefix. Like the suffix, the prefix remains the same for the lifetime of the ValueProvider.
A final aspect that is related to using lombok:
As opposed to a Product, creating an Order is done via a
builder rather than via a factory method. The OrderTestDataFactory therefore has 4 methods, a pair
of createOrder() and a pair of createOrderBuilder()
methods. Again, one of each pair has the ValueProvider as parameter to allow passing it on to invoked test data factories. The other
one without parameter creates a new random ValueProvider.
Now comes the easy part: If you need a valid data object for your test, but don't care about its content, create one. If you need more than one data object with different but valid data, create another one:
import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrder;
// ...
class MyOrderTest {
@Test
void do_something_with_a_single_order() {
Order anOrder = createOrder();
// ...
}
@Test
void do_something_with_two_different_orders() {
Order anOrder = createOrder();
Order anotherOrder = createOrder();
// ...
}
}If you want to control specific aspects of a data object that are important for your test, restrict your test code to only these aspects:
import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrderBuilder;
// ...
class MyOrderTest {
// ...
@Test
void shipping_address_is_used_as_default_for_billing_address() {
Order useShippingAddressAsBillingAddress = createOrderBuilder()
.billingAddress(empty())
.build();
// ...
}
}Please refer to OrderTest for more examples, and to the ValueProvider and its Javadoc to learn more about the methods it offers to populate the properties of your data objects.
As you have learned by now, using randomness helps minimize the code for creating test data. However, this comes at a price: If you want to reproduce test failures that might be related to random data, especially fom your CI suite of hundreds or even thousands of tests, it is vital to use the same data.
value-provider supports this use case out of the box by providing infrastructure for reproducing test failures.
For JUnit5, use the ValueProviderExtension:
import com.tngtech.valueprovider.ValueProviderExtension;
// ...
@ExtendWith(ValueProviderExtension.class)
class MyOrderTest {
// ...
}If your test class is derived from a base class, make sure to specify the ValueProviderExtension in the base class of the inheritance hierarchy.
For JUnit4, use the ValueProviderRule:
import com.tngtech.valueprovider.ValueProviderRule;
// ...
public class MyOrderTest {
@Rule
public ValueProviderRule valueProviderRule = new ValueProviderRule();
// ...
}If your test uses static test data created by using a ValueProvider, use the ValueProviderClassRule in addition:
import com.tngtech.valueprovider.ValueProviderClassRule;
import com.tngtech.valueprovider.ValueProviderRule;
// ...
import static com.tngtech.valueprovider.example.OrderTestDataFactory.createOrder;
// ...
public class MyOrderTest {
@ClassRule
public static final ValueProviderClassRule staticProviders = new ValueProviderClassRule();
@Rule
public ValueProviderRule instanceProviders = new ValueProviderRule();
// static test data
private static final Order DEFAULT_ORDER = createOrder();
// ...
}If your test class is derived from a base class, make sure to specify the rule(s) in the base class of the inheritance hierarchy. Otherwise, your test (or other tests in a CI suite) may fail.
If a test using the infrastructure fails, it provides information about the seed values used for generating the random data as shown in the following example:
org.junit.ComparisonFailure:
Expected :"testUserName1"
Actual :"testUserName"
<Click to see difference>
...
Suppressed: com.tngtech.valueprovider.ValueProviderException: If the failure is related to random ValueProviders, specify the following system properties for the JVM to reproduce:
-Dvalue.provider.factory.test.class.seed=0
-Dvalue.provider.factory.test.method.seed=-1608847119246027406
-Dvalue.provider.factory.reference.date.time=2021-06-04T15:28:34.004
at com.tngtech.valueprovider.ValueProviderRule.handleFailure(ValueProviderRule.java:57)
at com.tngtech.valueprovider.ValueProviderRule.access$100(ValueProviderRule.java:17)
at com.tngtech.valueprovider.ValueProviderRule$1.evaluate(ValueProviderRule.java:38)
...
If the failure is related to random data, you can easily reproduce it. Just specify the above shown JVM system properties in the command line when you re-run the test, e.g., in your IDE:
-Dvalue.provider.factory.test.class.seed=0
-Dvalue.provider.factory.test.method.seed=-1608847119246027406
-Dvalue.provider.factory.reference.date.time=2021-06-04T15:28:34.004
Note that seed values relate to individual test classes and methods, even if they have been run in a CI build together with other tests. Thus, it is sufficient to rerun the individual test in order to reproduce the failure.
The above example code always used ValueProviderFactory.createRandomValueProvider()
to create a ValueProvider that, in turn, generates random data. To be more precise,
the ValueProvider
is initialized with a random seed and will generate exactly the same data, if it is initialized with the same seed, and the same sequence of method invocations is executed. As you may have guessed
already, reproducing test failures is based on this functionality.
So, if you need reproducible test data, e.g., to test a transformation of Java data to XML by verifying against a previously stored XML file,
use ValueProviderFactory.createReproducibleValueProvider()
and provide a seed value of your choice to create the ValueProvider. Your test data factories will accept this ValueProvider just
as any other one.
The ValueProvider offers a considerable amount of common methods to fill properties of test data objects. Sooner or later however, the need will arise to add project specific functionality.
We advise the following approach:
- Create your own ValueProvider class. Let it extend the AbstractValueProvider provided by this library, and add the methods you need
- Create your own ValueProviderFactory class and implement 2 static methods that create instances of
your derived AbstractValueProvider class
createRandomValueProvider()createReproducibleValueProvider()
- Use your own classes instead of the ones provided by this library in your test data factories and other test code.
Refer to CustomValueProvider , CustomValueProviderFactory , and CustomValueProviderFactoryTest for a fully functional example.
The infrastructure uses thread-local data to store the seed values and can therefore be used in parallel CI builds without any problems.
However, reproducing test failures is not possible for multithreaded test code. Likewise, a ValueProvider initialized with a fixed seed value will not necessarily generate the same sequence of data if it is used by multiple threads, as the sequence of method invocations from different threads is not reproducible, and neither the ValueProvider nor the infrastructure provide any synchronisation.
value-provider is published under the Apache License 2.0, see license file for details.
1Please note that we use lombok and immutable data objects in our examples for convenience, but this is not a requirement for using value-provider.