[FIXED] JUnit's @TestMethodOrder annotation not working

Issue

I'm having a problem with following integration test

import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;

@SpringBootTest
@ActiveProfiles("test")
@TestMethodOrder(OrderAnnotation.class)
public class FooServiceIT {
    @Test
    @Order(1)
    void testUploadSuccess() { ... }
    @Test
    @Order(2)
    void testDownloadSuccess() { ... }
    @Test
    @Order(3)
    void testDeleteSuccess() { ... }
}

I'd expect when I run the test that the execution order would 1, 2, 3 but for some reason, the actual execution order is 2, 3, 1.

Tbh, I'm clueless why the annotation is not working. I'm using Spring Boot 2.1.3 with JUnit 5.4.

Solution

You need to configure correctly your IDE.

Requirements

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.4.0</version>
</dependency>

Do not use JUnit 5 that offers your IDE. If you add it as library, you will get:

No tests found for with test runner 'JUnit 5' 
==================== and this exception ===================
TestEngine with ID 'junit-vintage' failed to discover tests
java.lang.SecurityException: class "org.junit.jupiter.api.TestMethodOrder"'s signer information does not match signer information of other classes in the same package

So just include mentioned dependency only and your code will work as you expect:

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class FooServiceIT {

    @Test
    @Order(1)
    public void testUploadSuccess() {
        System.out.println("1");
    }

    @Test
    @Order(2)
    public void testDownloadSuccess() {
        System.out.println("2");
    }

    @Test
    @Order(3)
    public void testDeleteSuccess() {
        System.out.println("3");
    }
}

JUnit result:

1
2
3

Answered By - Valijon
Answer Checked By - Timothy Miller (JavaFixing Admin)