Issue
For one of my projects I'm using JUnit 5 to test reflection code, which requires a large number of classes for test cases. Throwing them all in one scope and trying to name them intelligently is nearly impossible, so I'm hoping to put both the test methods and the types being tested inside a static member class. Doing this would allow me to reuse names such as X
or Y
in each test, and would keep the types being tested near the code that tests them. (The member classes have to be static so I can add interfaces)
If I just add static classes the tests run fine out of the box, but in the final report I end up with all the member classes listed separately, so I'd like to be able to "flatten" them all into the single class in the report.
Here's an example of what I would like to achieve: (I'm actually writing the tests in Kotlin, but this is the equivalent Java code)
class MethodTests {
static class WhateverName {
interface X {}
class Y implements X {}
@Test
void something_withInterfaceSupertype_shouldReturnThing() {
// ...
}
@Test
void other_withInterfaceSupertype_shouldReturnThing() {
// ...
}
}
static class WhateverOtherName {
interface X {
void m();
}
class Y implements X {
void m() {}
}
@Test
void something_withInterfaceSupertype_andMethodImpl_shouldReturnOtherThing() {
// ...
}
}
// This might actually be even better, since I wouldn't need `WhateverName`
@Test // not valid, but maybe I could annotate the class instead of the method?
static class something_withInterfaceSupertype_shouldReturnThing_2ElectricBoogaloo {
interface X {}
class Y implements X {}
@Test
void runTest() {
// ...
}
}
}
At the moment the test report in IDEA ends up being structured like this:
- MethodTests
- someRoot_testHere
- MethodTests$WhateverName
- something_withInterfaceSupertype_shouldReturnThing
- other_withInterfaceSupertype_shouldReturnThing
- MethodTests$WhateverOtherName
- something_withInterfaceSupertype_andMethodImpl_shouldReturnOtherThing
- MethodTests$something_withInterfaceSupertype_shouldReturnThing_2ElectricBoogaloo
- runTest
But I'd like the test report to be structured like this:
- MethodTests
- someRoot_testHere
- something_withInterfaceSupertype_shouldReturnThing
- other_withInterfaceSupertype_shouldReturnThing
- something_withInterfaceSupertype_andMethodImpl_shouldReturnOtherThing
- something_withInterfaceSupertype_shouldReturnThing_2ElectricBoogaloo
I tried using @DisplayName
on the member classes but it just resulted in a duplicate name in the report. So far I think I might want to use an extension, but after doing a bit of research I haven't found any way to change the class a test is listed under in the report using them.
Solution
After more digging I was able to achieve almost exactly what I wanted using dynamic tests:
class MethodsTest {
@TestFactory
Iterator<DynamicTest> flat() {
return FlatTestScanner.scan(this);
}
@Test
void rootTest() {
}
@FlatTest
static class singleTestClass implements TestClass {
void run() {
// ...
}
}
static class Whatever {
@FlatTest
void multiTestClass_1() {
// ...
}
@FlatTest
void multiTestClass_2() {
// ...
}
}
}
The final report structure isn't quite perfect, but it's pretty close to what I was going for:
- MethodsTest
- flat()
- singleTestClass
- multiTestClass_1
- multiTestClass_2
- rootTest
Here's the code that makes this happen. It works by scanning all the declared classes for annotated methods and grabbing any that are annotated themselves, then creating dynamic tests for them, making sure to specify their source URIs. It's in Kotlin, but with a bit of work could be translated to Java:
import org.junit.jupiter.api.DisplayName
import org.junit.jupiter.api.DynamicTest
import java.net.URI
/**
* Useful for having separate class scopes for tests without having fragmented reports.
*
* @see FlatTestScanner.scan
*/
@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.FUNCTION, AnnotationTarget.CLASS)
annotation class FlatTest
object FlatTestScanner {
/**
* Returns dynamic tests to run all the flat tests declared in the passed class. This currently only works with
* static inner classes.
*
* - Annotated functions in inner classes will be run
* - Annotated inner classes will have their `run()` methods run
*
* To use this create a method in the outer class annotated with [@TestFactory][org.junit.jupiter.api.TestFactory]
* and return the result of passing `this` to this method. This will return matches from superclasses as well.
*
* ```java
* @TestFactory
* Iterator<DynamicTest> flat() {
* return FlatTestScanner.scan(this)
* }
* ```
*/
@JvmStatic
fun scan(obj: Any): Iterator<DynamicTest> {
val classes = generateSequence<Class<*>>(obj.javaClass) { it.superclass }
.flatMap { it.declaredClasses.asSequence() }
.toList()
val testMethods = classes.asSequence()
.map { clazz ->
clazz to clazz.declaredMethods.filter { m -> m.isAnnotationPresent(FlatTest::class.java) }
}
.filter { (_, methods) -> methods.isNotEmpty() }
.flatMap { (clazz, methods) ->
val instance = clazz.newInstance()
methods.asSequence().map { m ->
val name = m.getAnnotation(DisplayName::class.java)?.value ?: m.name
m.isAccessible = true
DynamicTest.dynamicTest(name, URI("method:${clazz.canonicalName}#${m.name}")) {
try {
m.invoke(instance)
} catch(e: InvocationTargetException) {
e.cause?.also { throw it } // unwrap assertion failures
}
}
}
}
val testClasses = classes.asSequence()
.filter { it.isAnnotationPresent(FlatTest::class.java) }
.map {
val name = it.getAnnotation(DisplayName::class.java)?.value ?: it.simpleName
val instance = it.newInstance()
val method = it.getDeclaredMethod("run")
method.isAccessible = true
DynamicTest.dynamicTest(name, URI("method:${it.canonicalName}#run")) {
try {
method.invoke(instance)
} catch(e: InvocationTargetException) {
e.cause?.also { throw it } // unwrap assertion failures
}
}
}
return (testMethods + testClasses).iterator()
}
}
Answered By - thecodewarrior