[FIXED] Mockito expecting argument mismatch

Issue

I'm writing a Unit test with JUnit 4 and Mockito 4.6.1. I'm mocking an interface method with a specific input value. When a different value is passed, I'm expecting an argument mismatch error, but it's not being thrown.

Consider the following example:

public class SomeTest {
    @Test
    public void test() {
        SomeInterface mock = Mockito.mock(SomeInterface.class, Mockito.withSettings().strictness(Strictness.STRICT_STUBS));

        // Only mock test(true) and not test(false).
        Mockito.when(mock.test(true)).thenReturn(1);

        Assert.assertEquals(mock.test(true), 1);
        
        Assert.assertEquals(
                // Expecting argument mismatch error
                mock.test(false), 
                2
        );
    }

    interface SomeInterface {
        int test(Boolean arg);
    }
}

I'm mocking SomeInterface.test(true) to return 1. This works as expected. Now when I call mock.test(false), I'm expecting an argument mismatch because it's not defined in the mock setup and strict mode is enabled. Instead, it returns 0 as if it was mocked.

Am I missing something that's causing this to happen?

Solution

Problem 1. We must enable STRICT_STUBS for test.

STRICT_STUBS - ensures clean tests, reduces test code duplication, improves debuggability. Best combination of flexibility and productivity. Highly recommended. Planned as default for Mockito v4. Enable it via MockitoRule, MockitoJUnitRunner or MockitoSession. See STRICT_STUBS for the details.

According to documentation it can be done via MockitoJUnitRunner.StrictStubs.class

@RunWith(MockitoJUnitRunner.StrictStubs.class)

Or strictness rule

@Rule public MockitoRule rule = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);

Problem 2. Argument strictness does not work when mock definition and mock invocation perform in one source class.
According to Mockito source code:

If stubbing and invocation are in the same source file we assume they are in the test code, and we don't flag it as mismatch:

Summarise. The working test will be:

import org.junit.Assert;
import org.junit.Test;

import org.junit.runner.RunWith;
import org.mockito.Mockito;

import org.mockito.junit.MockitoJUnitRunner;
import org.mockito.quality.Strictness;

import static org.mockito.Mockito.*;

@RunWith(MockitoJUnitRunner.StrictStubs.class)
public class SomeTest {
    @Test
    public void test() {
        SomeInterface mock = Mockito.mock(SomeInterface.class, withSettings().strictness(Strictness.STRICT_STUBS));
        Mockito.when(mock.test(true)).thenReturn(1);

        SomeInterfaceImpl someInterface = new SomeInterfaceImpl(mock);

        Assert.assertEquals(someInterface.test(), 2);
    }
}

public interface SomeInterface {
    int test(Boolean arg);
}

public class SomeInterfaceImpl {
    private SomeInterface someInterface;
    public SomeInterfaceImpl(SomeInterface someInterface) {
        this.someInterface = someInterface;
    }

    public int test() {
        return someInterface.test(false);
    }
}

On execution we get:

org.mockito.exceptions.misusing.PotentialStubbingProblem: 
Strict stubbing argument mismatch. Please check:
 - this invocation of 'test' method:
    someInterface.test(false);
    -> at com.example.SomeInterfaceImpl.test(SomeInterfaceImpl.java:10)
 - has following stubbing(s) with different arguments:
    1. someInterface.test(true);
      -> at com.example.SomeTest.test(SomeTest.java:26)
Typically, stubbing argument mismatch indicates user mistake when writing tests.
Mockito fails early so that you can debug potential problem easily.
However, there are legit scenarios when this exception generates false negative signal:
  - stubbing the same method multiple times using 'given().will()' or 'when().then()' API
    Please use 'will().given()' or 'doReturn().when()' API for stubbing.
  - stubbed method is intentionally invoked with different arguments by code under test
    Please use default or 'silent' JUnit Rule (equivalent of Strictness.LENIENT).
For more information see javadoc for PotentialStubbingProblem class.

More preferable solution
You can define the default answer with IllegalArgumentException for your mock in case if mock is executed with mismatch arguments. This solution will work always without source file restrictions.

import org.junit.Assert;
import org.junit.Test;

import org.junit.runner.RunWith;
import org.mockito.Mockito;

import org.mockito.junit.MockitoJUnitRunner;

import static org.mockito.Mockito.doReturn;

@RunWith(MockitoJUnitRunner.class)
public class SomeTest {
    @Test
    public void test() {
        SomeInterface mock = Mockito.mock(SomeInterface.class, Mockito.withSettings().defaultAnswer(
                invocation -> {
                    throw new IllegalArgumentException(String.format("You cannot invoke %s with %s", invocation.getMethod(), java.util.Arrays.toString(invocation.getArguments())));
                }
        ));

        // Only mock test(true) and not test(false).
        doReturn(1).when(mock).test(true);

        Assert.assertEquals(mock.test(true), 1);

        Assert.assertEquals(
                // Expecting argument mismatch error
                mock.test(false),
                2
        );
    }
}

On execution we get:

java.lang.IllegalArgumentException: You cannot invoke public abstract int com.example.SomeInterface.test(java.lang.Boolean) with [false]

Answered By - Eugene
Answer Checked By - Terry (JavaFixing Volunteer)