Unit Testing

Mocks in unit tests

Unit tests are aimed at testing small isolated components of code that can be predictably executed any number of times to yield the same outcome.

Mocks provide a convenient mechanism to circumvent interaction with methods that are outside the scope of your unit tests. This will let you focus on testing the code that is in scope for your unit tests and not worry about external classes/libraries/dependencies.

Having said that, code should be testable without using any of the mocking libraries like Mockito, JMockit etc. "Fakes rather than Mocks". A fake object is simply a replacement for the real object. While a real object may have actual business logic, a fake object may simply return canned responses.

public interface ConfigurationService {
        Configuration readConfiguration(String configurationName);
}

public class AzureConfigurationService implements ConfigurationService {
    Configuration readConfiguration(String configurationName) {
        // read configuration from Azure configuration service
    }
}

public class MyApplication {
    private final ConfigurationService configurationService;
    public MyApplication(ConfigurationService configurationService) {
        this.configurationService = configurationService;
    }

    public String getConfigurationOwner () {
        Configuration configuration = configurationService.readConfiguration("configName");
        return configuration.getOwner();
    }
}


public class Test {
    @Test
    public void testMyApplication() {
        // unit test won't work as it requires connection to azure services
        MyApplication myApplication = new MyApplication(new AzureConfigurationService()); 
        assertEquals("foo", myApplication.getConfigurationOwner());
    }
}

Instead, you could inject a fake dependency

public class FakeConfigurationService implements ConfigurationService {
    public Configuration readConfiguration(String configurationName) {
        Configuration configuration = new Configuration();
        configuration.setOwner("foo");
        return configuration;
    }
}

public class Test {
    @Test
    public void testMyApplication() {
        // unit test will now work as the fake object requires no network connection
        MyApplication myApplication = new MyApplication(new FakeConfigurationService());
        assertEquals("foo", myApplication.getConfigurationOwner());
    }
}

However, mocking libraries help in writing your tests faster. In the previous example, a fake implementation had to be created to write unit tests. Mocks can make this faster by letting you mock the behavior of real object without writing lot of code for faking dependencies.

Good candidates for using a mock:

1. Dependencies that require network connection - Any dependency that goes over the wire like database access, REST service calls etc.
2. File System - Methods/classes that require file operations. This should be treated similar to a dependency that requires network connection. Consider the case that the file may reside on a NFS.
3. Third-party library classes - mock these judiciously. Your code depends on these libraries and it's best to not mock them unless they are libraries that internally use file or network operations. Ideally, such libraries should be wrapped behind an interface you own and the dependency should be contained.
4. Date and time tests - Tests that rely on date can be mocked to return dates in future/past. Some functionalities require execution at certain intervals of time. Injecting a mock clock object with tickers that can simulate 100 clock ticks within a second is useful rather than letting the unit test run for 100 seconds.

Static Methods

Libraries like PowerMock allow mocking static methods but it comes at a cost. It meddles with the default classloader and can lead to inconsistent behavior in tests and production runtime. Also, some of these libraries interfere with test coverage instrumentation and result in incorrect test coverage reports.

Solution: If you need to mock a static method, first evaluate if there's a way to refactor your code and eliminate the need for static method. If that's not possible, then consider isolating the static method access by wrapping it in a method that can be easily mocked.

public class ClassToTest {
    public String methodToTest() {
        String retVal = Util.notTestFriendlyStaticMethod();
        return process(retVal);
    }
}

Consider doing

public class ClassToTest {
	private UtilWrapper utilWrapper;
    
    public ClassToTest(UtilWrapper utilWrapper) {
        this.utilWrapper = utilWrapper;
    }

    public String methodToTest() {
        String retVal = utilWrapper.callStaticUtilMethod();
        return process(retVal);
    }
}

public class Test {
    @Test
    public void testMethod() {
        UtilWrapper utilWrapper = Mockito.mock(UtilWrapper.class);
        when(utilWrapper.callStaticUtilMethod()).thenReturn("ExpectedString");
        ClassToTest classToTest = new ClassToTest(utilWrapper);
        assertEquals("expectedstring", classToTest.methodToTest());
    }
}

Mocking Final classes

If you are using Mockito, it requires additional configuration to be able to mock final classes as described in Mockito 2 documentation.

Note that most of (if not all) the client classes provided in Azure SDK are final. These clients use the network to communicate with Azure services.

As a consumer of Azure SDK client libraries, your application is taking a dependency on 3rd party library that making network calls. In such scenarios, it's a good idea for your application to abstract out the dependency and hide it behind your own interface. This will contain the scope of dependency just within the implementation of the interface and your application is only using the interface you have defined. This allows your application to switch between different implementations of the interface without having to make changes to your application. The added benefit is that now your unit tests can either use a fake implementation as shown in the example at the top of this page or use a mock without requiring special configuration to enable mocking of final classes.

Mocking Azure Service Clients

Customers using the Azure SDK client libraries can mock service clients using Mockito with org.mockito.plugins.MockMaker. More information can be found at Mock the unmockable: opt-in mocking of final classes/methods.

Steps

1. Create a file src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker
2. In the file, put a single line.

• mock-maker-inline

Example

Consider an application that uses Azure Event Hubs to fetch telemetry and the end user wants to test that their TelemetryEvents class properly transforms EventData to TelemetryEvent.

Product code

public class TelemetryEvents {
    private final EventHubConsumerAsyncClient consumerClient;
    public TelemetryEvents (EventHubConsumerAsyncClient consumerClient) {
        this.consumerClient = consumerClient;
    }
    
    public Flux<TelemetryEvent> getEvents() {
        return consumerClient.receiveFromPartition("1", EventPosition.latest())
                .map(event -> new TelemtetryEvent(event));
    }
}

Test code

import reactor.test.publisher.TestPublisher;
import reactor.test.StepVerifier;

import static com.azure.messaging.eventhubs.*;
import static org.mockito.Mockito.*;

public class TelemetryEventsTest {
    @Test
    public void canGetEvents() {
        // Arrange

        // After following the instructions in "Steps" section
        EventHubConsumerAsyncClient consumerClient = mock(EventHubConsumerAsyncClient.class);

        TestPublisher<EventData> eventsPublisher = TestPublisher.createCold();
        eventsPublisher.emit(new EventData("Foo"), new EventData("Bar"));
        when(consumerClient.receiveFromPartition(eq("1"), eq(EventPosition.latest())).thenReturn(eventsPublisher.flux());

        TelemetryEvents telemetryEvents = new TelemetryEvents(consumerClient);

        // Act
        StepVerifier.create(telemetryEvents.getEvents())
            .assertNext(event -> isMatchingTelemetry(event))
            .assertNext(event -> isMatchingTelemetry(event))
            .verifyComplete();        
    }
}

Test parallelization

Unit tests, by default, when using JUnit 5 (and further discussions assume the use of JUnit 5) are configured to run in parallel to reduce the time taken to run tests, and as a side effect find parallelization bugs that may exist. Unfortunately, parallelization creates some difficulties when using our standard test dependencies in reactor-test and mockito where certain functionality isn't thread-safe, the following sections explain some well-known gotchas.

StepVerifier.setDefaultTimeout

StepVerifier.setDefaultTimeout(Duration) before tests run, and cleaning up afterwards with StepVerifier.resetDefaultTimeout(), is a great way to ensure that a test, or tests, doesn't result in an infinite wait, which is the default for .verify() and other like methods on StepVerifier. But, using this mutates a global state variable on StepVerifier which can result in other tests randomly failing with AssertionError about failing to receive a verification complete before a timeout has elapsed. For example, if two test classes are ran at the same time where one sets the global default timeout to 5s and the other is running a longer running test that takes 1 minute to complete depending on the execution order it can result in the 1 minute test throwing an exception.

So, instead of using the global StepVerifier timeout it is better to use .verify(Duration). If your tests are using the composed verifications steps, such as .verifiyComplete() or verifyError(Class<? extends Throwable>), you can transition them to the corresponding expectation APIs, .expectComplete() or .expectError(Class<? extends Throwable>), and then chain those with .verify(Duration).

StepVerifier.withVirtualTime

StepVerifier.withVirtualTime is a great way to mock execution through time, for example, mocking 30 minutes of "running" with a simple API that takes milliseconds to complete.

The base overload uses a shared global VirtualTimeScheduler to virtualize time.

When running the test, the default reactor schedulers dealing with time are replaced by shared global VirtualTimeScheduler. When tests using the shared global VirtualTimeScheduler run in parallel, it can run into states where the scheduler is shut down or isn't instantiated when the test runs. So, if you need to use this API you have to do the following:

1. Use the overload accepting scheduler and pass an instance VirtualTimeScheduler. Remember to dispose the VirtualTimeScheduler instance after the test, or else threads may be leaked.
2. Annotate the test class with @Isolated and @Execution(ExecutionMode.SAME_THREAD).
3. Annotate the test method with @Execution(ExecutionMode.SAME_THREAD).

You can refer this pull request, which follows the above three steps.

Validators hasNotDroppedElements or hasNotDroppedErrors

StepVerifier offers the option to inspect the full runtime behavior of a reactive stream, such as ensuring that element emissions (hasNotDroppedElements) or errors (hasNotDroppedErrors) aren't being dropped in the reactive call chain. These mutate the global state in Hooks, so if other tests do drop elements or errors it may be caught in your testing and result in your test failing. If these are being used, use the guidance below to ensure they run correctly.

Global mutations are required

If you must use configurations that mutate global state JUnit 5 has a class-level annotation @Isolated that ensures the test is ran while no other tests are running. Isolated test classes will run after parallel safe class complete sequentially, resulting in longer test times.

Additionally, if your tests aren't safe to run in parallel, for example mutating the global System.out which is needed in validation for each test, you can use @Execution(ExecutionMode.SAME_THREAD). This will force the test methods in the test class to run sequentially.

So, combining @Isolated and @Execution(ExecutionMode.SAME_THREAD) creates a well encapsulated test class run that can help prevent test from failing flakily.

Remote Debugging

While most the time running tests in your IDE of choice is convenient and easy occasionally you may hit bugs or issues with running tests which becomes a complete blocker. To work around this while determining the root cause you can use Maven in the command line to run and remote debug tests, and even scope those runs to specific test classes or even test methods. The most basic command is

mvn -f <specific Maven POM to test> test (optionally you can include clean to ensure the environment is wiped before running)

This will compile your project and run all tests. You can scope to specific test classes or tests by adding -Dtest=<test class>#<test method> where #<test method> is optional. Continuing from before examples of these would be

mvn -f <specific Maven POM to test> test -Dtest=<the test class I want to test>

or

mvn -f <specific Maven POM to test> test -Dtest=<the test class I want to test>#<and the specific test method in it>

So far the commands mentioned will only run tests, but as important as just running tests is debugging them. To enable test debugging you can also add -Dmaven.surefire.debug, or if running integration tests -Dmaven.failsafe.debug, which will halt test execution until a connection is made to port 5005. Continuing from before an example of this would be

mvn -f <specific Maven POM to test> test -Dtest=<the test class I want to test>#<and the specific test method in it> -Dmaven.surefire.debug

Here is documentation that explains how to remote debug in IntelliJ.

Every example above will recompile your project before running tests and this can take a while and some times you just simply want to debug the test, or tests, again. Using surefire:test or failsafe:integration-test instead of test will only run the plugin goal instead of the test lifecycle which runs all build lifecycles before it (validate and compile), do note that if you make code changes using these goals won't use the updated code and that will require using the test lifecycle goal. Continuing from before an example of this would be

mvn -f <specific Maven POM to test> surefire:test -Dtest=<the test class I want to test>#<and the specific test method in it> -Dmaven.surefire.debug

With this you should be able to use the Maven command line to run tests and remotely debug them without needing to use your IDE to run the test. If you're using PowerShell you'll need to quote -Dmaven.surefire.debug ("-Dmaven.surefire.debug") and may need to quote -Dtest if your test method has a space in it, if you don't PowerShell will incorrectly parse the command.