Use JUnit Parameterized Tests to Force Test Coverage

Pictured is author Jing Fan with a co-worker. Post title: Use JUnit Parameterized Tests to Force Test Coverage

Article summary

Background
Paramaterized Tests

During a recent software development project, I made changes to part of our codebase that programmatically generates and modifies a tree-like data structure. I wanted to highlight how some really nice unit tests using JUnit’s ParameterizedTest with a MethodSource returning Stream helped catch a couple of would-be bugs with my recent changes. This approach also helped keep our tests more manageable and readable.

[JUnit parameterized tests](https://docs.junit.org/current/user-guide/#writing-…

Pictured is author Jing Fan with a co-worker. Post title: Use JUnit Parameterized Tests to Force Test Coverage

JUnit parameterized tests allow you to run the same test case with different arguments. It’s Java/Kotlin’s equivalent to Jest’s test.each / it.each, or XUnit’s [Theory] annotations. You can provide a list of values using ValueSource, or a generator/factory function using MethodSource.

Background

First, let’s describe what the tests were trying to cover. As mentioned, the code under test is responsible for generating a tree-like data structure, which we call a Workflow. We are also making changes to the base data structure, through an Extension. So, the existing tests covered things like ensuring that the resulting data structure had valid references and that it did not contain any cycles or any unreachable nodes. Here’s what one of our tests for this functionality looks like:


@JvmStatic
fun extensionTypes(): Stream = ExtensionType.entries.stream().map {
Arguments.of(it, ExtensionData(extensionId = UUID.randomUUID()))
}

@ParameterizedTest
@MethodSource("extensionTypes")
fun `extension should have valid structure and no duplicate IDs`(
extensionType: ExtensionType,
extensionData: ExtensionData,
) {
val result = WorkflowExtensionFactory.getWorkflowExtension(extensionType = extensionType, extensionData = extensionData)
assertIdsAndReferencesValid(result, context = "extension type ${extensionType.value}")
assertDependenciesAreTraceable(result, context = "extension type ${extensionType.value}")
assertNoCyclicOutcomeBranches(result, context = "extension type ${extensionType.value}")
}

Our tests for these workflow and extension factories don’t assert about the exact equality of the created result or the specific dependencies expected. That would be a bit brittle and annoying to keep up to date. The actual movement through our tree is something that’s better covered by more system-level tests, anyway.

Using parameterized tests that assert on the validity of the resultant data structure — rather than the contents of it — strikes a happy medium of test coverage. We’re confident the data structure generated is valid, but we don’t have to invest much time keeping the expected values up to date.

Paramaterized Tests

Recently, I introduced support for a new extension type, and I accidentally omitted a dependency from a given node in the tree. This would have left the data stuck in an invalid state. We definitely would have caught this issue when we were ready to implement the ability to transition from one node to the unreachable node. However, these parameterized tests helped us catch this issue earlier on in the process (all without even writing a single line of new test code!).

Further, using .entries.stream().map() to generate arguments from our ExtensionType enum class means that any new extension types added are automatically added to this test case. While we can usually trust that a developer would remember to add test coverage for their new ExtensionType, this approach makes it impossible not to.

Even if we were testing something less repeatable that required each different enum value to have more specific assertions, we could still have a simple test case that uses .entries.stream().map() . Then we can just check that the function under tests runs without throwing (as it might if there was an exception thrown in the default block of a switch statement).