[FIXED] How to run multiple named tests with Gradle?

Issue

I know how to say

gradle test --tests mypackage.MyTest

but how to specify more than one?

gradle test --tests mypackage.MyTest,mypackage.model.ModelTest,BasicTests

This seemed to just run one of the tests at random.

gradle test --tests mypackage.MyTest mypackage.model.ModelTest BasicTests

This tells me

Task 'mypackage.model.ModelTest' not found in root project 'myproject'.

I read to a lot of manuals and tutorials (and a few SO posts) and found no explicit mention of the answer. But I saw someone passing --tests "somepackage.*" as an argument, so I figured that quotes around the spaces might help.

gradle test --tests "mypackage.MyTest mypackage.model.ModelTest BasicTests"

This does not fail fast, but unfortunately runs only the first test in the list.

So I guess that I have just been mislead by the plural in "--tests" and there is no way to actually give it a list, right?

Solution

The documentation article on Testing in Java & JVM projects is pretty clear on how the --tests option works. It is just an inclusive filter registering all matching tests to be executed, in the same way as using includeTestsMatching in the filter block of a Test task:

test {
    filter {
        includeTestsMatching 'org.gradle.internal.*'
    }
}

This is also the reason for the plural in the name of the parameter, as wildcards allow an arbitrary number of tests to be included.

Note that the wildcard '*' has no special understanding of the '.' package separator. It’s purely text based. So --tests *.SomeTestClass will match any package, regardless of its 'depth'.

Since Gradle 4.7 it is possible to use a simple class name without package information, if you start the filter with an uppercase letter.

To include multiple tests that don't share a common package structure, you must and can use multiple --tests parameters, as noted in the documentation:

Also note that it is possible to supply multiple --tests options, all of whose patterns will take effect.

Additional information on test filters may be found in the link above and this related question:

How to run only one test class on gradle

---

Just as additional information on why one of your approaches did not work:

Executing Gradle on the command-line conforms to the following structure. Options are allowed before and after task names.

gradle [taskName...] [--option-name...]

In your third example, the other options for the --tests option are interpreted as task names, which explains the error message, as a task with the specified name does not exist.

To prevent confusion between option values and task names and to stop any shell interference, you should define filters between quotes as recommended by Mr Haki in his blog.

Answered By - Lukas Körfer