Test execution order

Test execution order

By design, JUnit does not specify the execution order of test method invocations. Originally the methods were simply invoked in the order returned by the reflection API. However, using the JVM order is unwise since the Java platform does not specify any particular order, and in fact JDK 7 returns a more or less random order. Of course, well-written test code would not assume any order, but some do, and a predictable failure is better than a random failure on certain platforms.

From version 4.11, JUnit will by default use a deterministic, but not predictable, order. The following sections describe how to change the method execution order

OrderWith annotation

From version 4.13, to can specify a different method execution order via the @OrderWith annotation. See the example below.

The parameter to @OrderWith is an instance of Ordering.Factory. JUnit provides implementations of Ordering.Factory in org.junit.tests.manipulation. Users can create their own instances of Ordering.Factory to provide Ordering implementations that reorder tests. Implementations of Ordering.Factory should have a public constructor that takes in a Ordering.Context (see the Alphanumeric source code for an example).

The Ordering class provides some static methods that simplify creation of Ordering instances (for example, Ordering.shuffledBy(Random random) will reorder the tests using the passed-in Random instance). The Request class has been updated to have a orderWith(Ordering) method that mirrors the older sortWith(Comparator<Description> comparator) method,.

The @OrderWith annotation works with any runner that implements Orderable. The ParentRunner abstract class was been retrofitted to implement Orderable so the runners provided by JUnit support @OrderWith, and many third-party runners will support it.

Example

import org.junit.Test;
import org.junit.runner.OrderWith;
import org.junit.runner.manipulation.Alphanumeric;

@OrderWith(Alphanumeric.class)
public class TestMethodOrder {

    @Test
    public void testA() {
        System.out.println("first");
    }
    @Test
    public void testB() {
        System.out.println("second");
    }
    @Test
    public void testC() {
        System.out.println("third");
    }
}

Above code will execute the test methods in the order of their names, sorted in ascending order.

FixMethodOrder annotation

From version 4.11, you can change the test execution order simply annotate your test class using @FixMethodOrder and specify one of the available MethodSorters:

@FixMethodOrder(MethodSorters.JVM): Leaves the test methods in the order returned by the JVM. This order may vary from run to run.

@FixMethodOrder(MethodSorters.NAME_ASCENDING): Sorts the test methods by method name, in lexicographic order.

If you do no specify either @FixMethodOrder or @OrderWith, the default ordering is equivalent to @FixMethodOrder(MethodSorters.DEFAULT)

Example

import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TestMethodOrder {

    @Test
    public void testA() {
        System.out.println("first");
    }
    @Test
    public void testB() {
        System.out.println("second");
    }
    @Test
    public void testC() {
        System.out.println("third");
    }
}

Above code will execute the test methods in the order of their names, sorted in ascending order.