[FIXED] Maven test command founds more test than test folder contains

Issue

I was examining a open source project and i noticed mvn test command founds more tests than test folder contains.

There are errors and failures in tests. I do not know if it is related to the problem.

mvn test command:

Tests run: 1279, Failures: 2, Errors: 918, Skipped: 2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  18.646 s
[INFO] Finished at: 2021-08-31T12:00:30+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.10:test (default-test) on project aliyun-sdk-oss: There are test failures.

If I run tests from choosing test folder in intellij ide:

So mvn test runs 1279 tests but intellij can run only 856 test. What is the reason of this ?

Solution

At first, it's common expected behavior if there are more test cases than test classes. Maven counts test cases.

public class FooTest {

    @Test
    public void first() {
        // some test and assertion
    }

    @Test
    public void second() {
        // some test and assertion
    }
}

In this case there is only one test class and two test cases, so Maven will display

Tests run: 2, ......

Then take a look to IntelliJ IDEA's output. There are four different sings at be beginning of each lines.

• Green thick means test was success
• Yellow cross means test ran but some assertion(s) failed
• Gray sing means tests are skipped (e.g. disabled on OS)
• Red exclamation mark means test didn't run.

The point is the last one. That means some prerequisites are not available or something unexpected happened during initialization. For example if the before method throws an exception, that test class (and all of test cases in that class) will not run.

Probably IDEA didn't pick all configuration option, but the command line did.

All failed tests listed on the Run tab, just scroll down a little more.

One of those failures is:

com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.

    at com.aliyun.oss.common.auth.DefaultCredentials.<init>(DefaultCredentials.java:37)
    at com.aliyun.oss.common.auth.DefaultCredentials.<init>(DefaultCredentials.java:32)
    at com.aliyun.oss.integrationtests.TestBase.getOSSClient(TestBase.java:107)
    at com.aliyun.oss.integrationtests.TestBase.deleteBucket(TestBase.java:122)
    at com.aliyun.oss.integrationtests.TestBase.tearDown(TestBase.java:98)
    at com.aliyun.oss.integrationtests.ListVersionsTest.tearDown(ListVersionsTest.java:78)
    at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
    at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:27)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Answered By - zforgo