Mockito using thenAnswer versus thenReturn

Mockito is a great tool for creating Test Doubles. Some time back I wrote a tutorial demonstrating Mockito. At the time, I did not need to use a feature of Mockito where the value returned by a stubbed method was evaluated later instead of immediately.

The week of 2011/9/11 I was working with a few people and we came across a need based on the design of an underlying domain class. I cannot share with you the real code, but the only thing that was important was the return type of the method in question:

package shoe.ex;

import java.util.Enumeration;

public interface ProblemInterface {
  Enumeration<object> returnTypeHurts();
}

To understand this problem, let's look at a simple example:

package shoe.ex;

import java.util.ArrayList;
import java.util.Iterator;
import org.junit.Test;

public class WhenToUseThenReturn {  
  @Test
  public void thisWorksFine() {
    ArrayList<object> aList = new ArrayList<object>();
    aList.add("Brett");
    Iterator<object> iterator = aList.iterator();
    iterator.next();
  }
}

This first example works without fail. However, change the order of lines 12 and 13:

@Test
public void thisDoesNotWork() {
  ArrayList<object> aList = new ArrayList<object>();
  Iterator<object> iterator = aList.iterator();
  aList.add("Brett");
  iterator.next();
}

And you'll see the following failure:

java.util.ConcurrentModificationException at
java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at
java.util.AbstractList$Itr.next(AbstractList.java:343) at
shoe.ex.WhenToUseThenReturn.thisDoesNotWork(WhenToUseThenReturn.java:14)
<snip>

What is going on?
If you create the Iterator before you update the collection, then the Iterator becomes invalidated. This is also true of Enumeration (used in the interface above).

Why does the method signature of the interface cause problems?
In the underlying problem, we needed to create a stub class that returns hard-coded collections. Now by "hard-coded" I really mean hard-coded per test. That is, each @Test needs its own set of values, which is typical (or if not typical, why are you writing generic test setup?).

The initialization of the underlying stub object, using Mockito, was created in the @Setup. During the @Setup, the hard-coded return values are created by first creating empty collections, then associating those empty collections' Enumerations with the interface. Here's such an example:

package shoe.ex;

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Enumeration;
import java.util.Iterator;
import org.junit.Before;
import org.junit.Test;

public class WhenToUseThenReturn {
  private ProblemInterface mock;
  private ArrayList<object> arrayList;

  @Before
  public void createMockObject() {
    arrayList = new ArrayList<object>();
    mock = mock(ProblemInterface.class);
    when(mock.returnTypeHurts()).thenReturn(
      Collections.enumeration(arrayList)); 
    }

  @Test
  public void seeTheProblem() {
   arrayList.add("Brett");
   Enumeration<object> i = mock.returnTypeHurts();
   i.nextElement();
  }
}

To make this work, the following call:

Collections.enumeration(arrayList)

Must happen after this call:

arrayList.add("Brett");

So one option is to add to the collection then execute this line of code:

when(mock.returnTypeHurts()).thenReturn(Collections.enumeration(arrayList));

Using thenReturn forces moving the statement out of the @Before method. More importantly, assuming there are many @Tests (and in this case there are), you'll have to remember to do that in each individual test. That's a violation of the DRY principle.

Alternatively, you can use thenReturn to have Mockito defer evaluation until later. The method thenReturn takes an anonymous inner class, so it's a few more moving parts:

when(mock.returnTypeHurts()).thenAnswer(new Answer<object>() { 
  public Object answer(InvocationOnMock invocation) throws Throwable {
    return Collections.enumeration(arrayList);
  }});

Rather than creating the Enumeration immediately, this instead creates an instance of an anonymous inner class that will be executed later. When the code uses the returnTypeHurts() method, this anonymous inner class instance will create a new Enumeration. Also, since this performed each time the returnTypeHurts() method is called it does so again. In fact, using theAnswer behaves semantically more like the underlying code so it is a better fit.

Should I always use thenAnswer?
I don't think so. While it will always work, it's not always necessary. I'd recommend using the simpler thenReturn unless it causes problems. In practice, I've not had to use it very often.

Why now?
This example has an interface that returns a generated value. Furthermore, that generated value is only good so long as the underlying collection does not change. And finally, we need to perform this setup for every test (there's only one here, but there will be more in the real system), and each test requires unique collection contents. This combination makes it necessary to defer creation of the Enumeration until after the collection has been updated. The collection needs to be updated in the test, therefore we need to either create the Enumeration in the test or later. By using thenAnser we manage to do it later.