[FIXED] How can I make my JUnit tests run in random order?

Issue

I have the classical structure for tests, I have a test suite of different suites like DatabaseTests, UnitTests etc. Sometimes those suites contains other suites like SlowDatabaseTests, FastDatabaseTests etc.

What I want is to randomize the running order of tests so I will make sure they are not dependent to each other. Randomization should be at every level, like suite should shuffle test class order, and test class should shuffle test method order.

If it is possible to do this in Eclipse that will be the best.

Solution

You do have a Sortable but I can't see how you would use it.

You could extend BlockJUnit4ClassRunner and have computeTestMethods() return a randomized copy of super.computeTestMethods(). Then use the @RunWith to set that as the runner to use.

e.g.

package com.stackoverflow.mlk;

import java.util.Collections;

import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.InitializationError;

public class RandomBlockJUnit4ClassRunner extends BlockJUnit4ClassRunner {

    public RandomBlockJUnit4ClassRunner(Class<?> klass)
            throws InitializationError {
        super(klass);
    }

    protected java.util.List<org.junit.runners.model.FrameworkMethod> computeTestMethods() {
        java.util.List<org.junit.runners.model.FrameworkMethod> methods = super.computeTestMethods();
        Collections.shuffle(methods);
        return methods;
    }

}

Then

@RunWith(com.stackoverflow.mlk.RandomBlockJUnit4ClassRunner.class)
public class RandomOrder {
    @Test
    public void one() {
    }

    @Test
    public void two() {
    }

    @Test
    public void three() {
    }
}

Answered By - Michael Lloyd Lee mlk