JMockit Evaluation

In this document we evaluate JMockit against a set of common unit testing and injection use cases.   To see how JMockit stacks up against Moles, another test double injection framework, see our JMockit vs. Moles Comparison.

To get started with JMockit, see our JMockit: The Basics.

For the source code illustrated here, check out https://github.com/duderino/injection.

Finally, refer to the JMockit Tutorial for official documentation.

1. Inject test doubles without changing the API

Yes, JMockit supports this.

Imagine a class with a single dependency.   Can we use JMockit to inject a test double in lieu of that dependency?

Class.java:

public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}
Dependency.java:
public class Dependency {
    public int generate() {
        return 999;
    }
}
ClassTest.java:
public class ClassTest {
    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, new Dependency() {
            @Mock
            @Override
            public int generate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

Here we subclass Dependency with an anonymous class.   The class uses JMockit's @Mock annotation to tell JMockit that the test double's method should be used in lieu of the real one.   Mockit.setUpMock() registers the test double with the JMockit runtime.

2. Replace final/non-virtual functions with test doubles

Yes, JMockit supports this.

Let's make the Dependency final so that we can no longer subclass it with a test double.

Dependency.java:
public final class Dependency {
    public final int generate() {
        return 999;
    }
}
We don't change the class under test for this.

Class.java:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}
Since Java won't let us subclass Dependency, we instead create a static class.    We associate the static class with the Dependency using JMockit's @MockClass annotation.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        public int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}
JMockit also supports an alternative mechanism for these cases via its MockUp generic class.   The JMockit authors recommend this as a replacement for the convenience of anonymous mock classes, but the code looks strange at first glance to a Java programmer.   Creating new objects assigned to nothing, while permitted by the language, isn't typically done.

ClassTestAlternative.java:
public class ClassTestAlternative {
    @Test
    public void testIt() {
        new MockUp<Dependency>() {
            @Mock
            public int generate() {
                return 123;
            }
        };


        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

3. Replace static functions with test doubles

Yes, JMockit supports this.

Here we change our dependency's method to be static.

Dependency.java:
public class Dependency {
    public static int generate() {
        return 999;
    }
}
The class under test no longer instantiates the dependency and instead calls its static method.

Class.java:
public class Class {
    public int generate() {
        return Dependency.generate() * 2;
    }
}

The static method can be replaced with a test double using the same mechanism used in the previous example.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        public static int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

4. Replace constructors with test doubles

Yes, JMockit supports this.

Here we change our dependency to take an argument in its constructor.

Dependency.java:
public class Dependency {
    private int value;

    public Dependency(int value) {
        this.value = value;
    }

    public int generate() {
        return value;
    }
}
The class under test is similarly modified to pass the argument through its constructor to the dependency's constructor.

Class.java:
public class Class {
    private Dependency dependency = new Dependency(999);

    public int generate() {
        return dependency.generate() * 2;
    }
}

JMockit will call any "$init" method with a void return type in lieu of the replaced class's constructor.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        void $init(int value) {
            assert 999 == value;
        }

        @Mock
        public static int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}
The replacement constructor may also modify the real object, but in such scenarios it's limited by the accessibility of the real object's members.    If, for instance, we made the Dependency.value field public, we could change its value in the $init constructor and avoid overriding the generate method:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        public Dependency it;

        @Mock
        void $init(int value) {
            assert 999 == value;

            it.value = 123;  // Compiles only if Dependency.value is accessible
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

Here we rely on a special 'it' member to access the real object from the test double.   This mechanism is treated in section 8 below.

5. Replace anonymous inner classes with test doubles

No, JMockit does not support this.

Sometimes the class under test creates an anonymous inner class that we'd like to replace with a test double.   This is a particularly hard problem because the inner class may call back on the class under test's internals.   It's also arguably not a valid use case.   The inner class is arguably part of the class under test's implementation and not a proper dependency.

To be sure, JMockit might support this, but we couldn't figure out how to do it.   This is what we tried to do...

First, we turned the Dependency into an interface.

Dependency.java:
public interface Dependency {
    int generate();
}
Next we modified the class under test to instantiate an anonymous implementation of the interface.

Class.java:
public class Class {
    public int generate() {
       Dependency dependency = new Dependency() {
           @Override
           public int generate() {
               return 999;
           }
       };

        return dependency.generate() * 2;
    }
}
Finally the test case was modified to inject a mock version of the Dependency.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        public int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}
Unfortunately JMockit threw a "Not a modifiable class" IllegalArgumentException when the test was run:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.42 sec <<< FAILURE!
testIt(org.duderino.injection.jmockit._5.ClassTest)  Time elapsed: 0.033 sec  <<< FAILURE!
java.lang.IllegalArgumentException: Not a modifiable class: org.duderino.injection.jmockit._5.Dependency
	at org.duderino.injection.jmockit._5.ClassTest.testIt(ClassTest.java:22)

Results :

Failed tests:   testIt(org.duderino.injection.jmockit._5.ClassTest): Not a modifiable class: org.duderino.injection.jmockit._5.Dependency

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

6. Replace inaccessible (e.g., private) inner classes with test doubles

No, JMockit does not support this.

Extending the previous example, it appears that JMockit must have access to the definition of every class it needs to mock out.   Private inner classes, static or not, are not visible to the JMockit annotations so they break compilation.

For this test we moved the Dependency class into the Class class.

Class.java:
public class Class {
    private static class Dependency {
        public int generate() {
            return 999;
        }
    }

    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}
Next we tried to inject a mock implementation in the test case.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Class.Dependency.class)
    public static class MockDependency {
        @Mock
        public int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Class.Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

The result was a predictable compilation error:
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /Users/blattj/dev/injection/src/test/java/org/duderino/injection/jmockit/_6/ClassTest.java:[12,32] org.duderino.injection.jmockit._6.Class.Dependency has private access in org.duderino.injection.jmockit._6.Class
[ERROR] /Users/blattj/dev/injection/src/test/java/org/duderino/injection/jmockit/_6/ClassTest.java:[22,30] org.duderino.injection.jmockit._6.Class.Dependency has private access in org.duderino.injection.jmockit._6.Class
[INFO] 2 errors 


The same result was observed by omitting the 'static' qualifier.

7. Replace subclass and superclass functions with test doubles

Yes, JMockit supports this.

To test this use case, we split our dependency into a superclass and a subclass.   The superclass and subclass share one method so we can test overriding and overshadowing.   The superclass and subclass also each have a distinct method that isn't overriden or doesn't overshadow methods in the other.

To tell them apart, the super class always returns '999' and the subclass always returns '333'.

SuperDependency.java:

public class SuperDependency {
    public int generate() {
        return 999;
    }

    public int superGenerate() {
        return 999;
    }
}
SubDependency.java:
public class SubDependency extends SuperDependency {
    @Override
    public int generate() {
        return 333;
    }

    public int subGenerate() {
        return 333;
    }
}
The class under test instantiates the subclass and exposes all three methods to the test case.

Class.java:
public class Class {
    private SubDependency dependency = new SubDependency();

    public int generate() {
        return 2 * dependency.generate();
    }

    public int superGenerate() {
        return 2 * dependency.superGenerate();
    }

    public int subGenerate() {
        return 2 * dependency.subGenerate();
    }
}
Our test class tests four conditions:
  1. Can we inject a test double for the subclass that overrides the superclass?   See testOverride()
  2. Can we inject a test double for the subclass that is overriden by the non-replaced subclass?   See testOvershadow()
  3. Can we inject a test double for the super class that is not overshadowed by the subclass?   See testSuper()
  4. Can we inject a test double for the subclass that does not override the superclass?  See testSub()
public class ClassTest {
    @Test
    public void testOverride() {
        Mockit.setUpMock(SubDependency.class, new SubDependency() {
            @Mock
            public int generate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 2 * 123 == clazz.generate();      // value should come from mock - mock should override
        assert 2 * 333 == clazz.subGenerate();   // value should come from subclass
        assert 2 * 999 == clazz.superGenerate(); // value should come from super class
    }

    @Test
    public void testOvershadow() {
        Mockit.setUpMock(SuperDependency.class, new SuperDependency() {
            @Mock
            public int generate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 2 * 333 == clazz.generate();      // value should come from subclass - mock should be overshadowed
        assert 2 * 333 == clazz.subGenerate();   // value should come from subclass
        assert 2 * 999 == clazz.superGenerate(); // value should come from super class
    }

    @Test
    public void testSuper() {
        Mockit.setUpMock(SuperDependency.class, new SuperDependency() {
            @Mock
            public int superGenerate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 2 * 333 == clazz.generate();      // value should come from subclass
        assert 2 * 333 == clazz.subGenerate();   // value should come from subclass
        assert 2 * 123 == clazz.superGenerate(); // value should come from mock
    }

    @Test
    public void testSub() {
        Mockit.setUpMock(SubDependency.class, new SubDependency() {
            @Mock
            public int subGenerate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 2 * 333 == clazz.generate();      // value should come from subclass
        assert 2 * 123 == clazz.subGenerate();   // value should come from mock
        assert 2 * 999 == clazz.superGenerate(); // value should come from super class
    }
}
The one important JMockit limitation of note is that  the limitation that methods may only be replaced on the class that declares them, not the class that inherits them.   We could not, for instance, try to replace the superGenerate() method on the SubDependency class because that method is only declared in the SuperClass.   This test...:
    @Test
    public void testSuper() {
        Mockit.setUpMock(SubDependency.class, new SubDependency() {
            @Mock
            public int superGenerate() {
                return 123;
            }
        });

        Class clazz = new Class();

        assert 2 * 333 == clazz.generate();
        assert 2 * 333 == clazz.subGenerate();
        assert 2 * 123 == clazz.superGenerate();
    }
... produces the following runtime error:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.578 sec <<< FAILURE!
testSuper(org.duderino.injection.jmockit._7.ClassTest)  Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.IllegalArgumentException: Matching real methods not found for the following mocks of org.duderino.injection.jmockit._7.ClassTest$3:
int superGenerate()
	at org.duderino.injection.jmockit._7.ClassTest.testSuper(ClassTest.java:43)

Results :

Failed tests:   testSuper(org.duderino.injection.jmockit._7.ClassTest): Matching real methods not found for the following mocks of org.duderino.injection.jmockit._7.ClassTest$3:

Tests run: 11, Failures: 1, Errors: 0, Skipped: 0

The Moles injection framework evaluated in our Moles Evaluation had a similar limitation.   Moles stubs were generated only for the methods declared, not inherited, in each class.

8. Inject a test double that delegates to a real instance

Yes, JMockit supports this.

Here we created a simple dependency and class that depends on it.

Dependency.java:
public class Dependency {
    public int generate() {
        return 999;
    }
}
Class.java:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}

In our test class we inject a test double that 'decorates' the real class.   Our test double delegates to the real class, but counts the number of times the real class is called.

ClassTest.java:
public class ClassTest {
    private static class Count {
        private int value = 0;

        public void increment() {
            ++value;
        }

        public int total() {
            return value;
        }
    }

    @Test
    public void testIt() {
        final Count count = new Count();

        Mockit.setUpMock(Dependency.class, new Dependency() {
            public Dependency it;

            @Mock(reentrant = true)
            @Override
            public int generate() {
                count.increment();

                return it.generate();
            }
        });

        Class clazz = new Class();

        for (int i = 0; i < 10; ++i) {
            assert 999 * 2 == clazz.generate();
        }

        assert count.total() == 10;
    }
}
Several special things have to happen when we setup this test double.

We have to declare a 'public Dependency it;' member.   All three tokens are important here.   If the member is declared private, JMockit will throw an exception that looks like this:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.529 sec <<< FAILURE!
testIt(org.duderino.injection.jmockit._7.ClassTest)  Time elapsed: 0.012 sec  <<< FAILURE!
java.lang.IllegalAccessError: tried to access field org.duderino.injection.jmockit._7.ClassTest$1.it from class org.duderino.injection.jmockit._7.Dependency
	at org.duderino.injection.jmockit._7.Dependency.generate(Dependency.java)
	at org.duderino.injection.jmockit._7.Class.generate(Class.java:10)
	at org.duderino.injection.jmockit._7.ClassTest.testIt(ClassTest.java:37)

Results :

Failed tests:   testIt(org.duderino.injection.jmockit._7.ClassTest): tried to access field org.duderino.injection.jmockit._7.ClassTest$1.it from class org.duderino.injection.jmockit._7.Dependency

Tests run: 7, Failures: 1, Errors: 0, Skipped: 0

Assuming there is a publicly accessible 'it' member with the same type as the real, non-mocked instance, JMockit will automatically copy the real, non-mocked instance reference into the 'it' member.   While not tested, getting the data type wrong would probably produced a ClassCastException.

The other special gotcha is the 'reentrant = true' qualifier on the @Mock annotation.   Without this qualifier JMockit will still redirect calls to the real instance to the mock.   That is, the mock will call itself recursively, generating a stack overflow that looks like this:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.668 sec <<< FAILURE!
testIt(org.duderino.injection.jmockit._7.ClassTest)  Time elapsed: 0.095 sec  <<< FAILURE!
java.lang.StackOverflowError
	at java.util.HashMap.get(HashMap.java:298)
	at java.util.Collections$SynchronizedMap.get(Collections.java:1975)
	at org.duderino.injection.jmockit._7.Dependency.generate(Dependency.java)
	at org.duderino.injection.jmockit._7.ClassTest$1.generate(ClassTest.java:26)
	at org.duderino.injection.jmockit._7.Dependency.generate(Dependency.java)
	at org.duderino.injection.jmockit._7.ClassTest$1.generate(ClassTest.java:26)
	at org.duderino.injection.jmockit._7.Dependency.generate(Dependency.java)

...
at org.duderino.injection.jmockit._7.ClassTest$1.generate(ClassTest.java:26) at org.duderino.injection.jmockit._7.Dependency.generate(Dependency.java) Results : Failed tests: testIt(org.duderino.injection.jmockit._7.ClassTest) Tests run: 7, Failures: 1, Errors: 0, Skipped: 0

9. Inject hand-coded test doubles

Yes, JMockit supports this. 

Every example in this document has injected a hand-coded test double.

10. Test double injection can be toggled

Yes, JMockit supports this.

Again we declare a simple dependency and a class that depends on it.

Dependency.java:
public class Dependency {
    public int generate() {
        return 999;
    }
}
Class.java:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}

In our test case we alternate between the test double and the real object by alternating between the Mockit.setUpMock() and Mockit.tearDownMocks() methods.

ClassTest.java:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        public int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();

        Mockit.tearDownMocks(Dependency.class);

        assert 999 * 2 == clazz.generate();

        Mockit.setUpMock(Dependency.class, MockDependency.class);

        assert 123 * 2 == clazz.generate();
    }
}

11. Test double injection can be narrowly targeted

Yes, JMockit supports this, but the resulting solution is brittle.

For this example the dependency is unchanged.

Dependency.java:
public class Dependency {
    public int generate() {
        return 999;
    }
}
But we modify the class under test to call the dependency twice.

Class.java:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() + (2 * dependency.generate());
    }
}
Can we replace the second generate() call but not the first?

To do this we created a slightly more complicated test double that counts the number of times it has been called.   On every call but the second call it delegates to the real instance.

ClassTest.java:
public class ClassTest {
    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, new Dependency() {
            public Dependency it;
            private int calls = 0;

            @Mock(reentrant = true)
            public int generate() {
                return ++calls == 2 ? 123 : it.generate();
            }
        });

        Class clazz = new Class();

        assert 999 + (2 * 123) == clazz.generate();
    }
}
See section 8 for a more details on delegating to the real instance from a test double.

The solution is of course brittle because slight changes to and non-determinism in the code under test will break the narrow targeting.

12. Replace any implementation of an interface with a test double

No, JMockit does not support this.

There are cases when the implementation of a class's dependency is not known until runtime and only its interface is known to the compiler.   Such cases are increasingly common with inversion of control frameworks like Spring and Guice.

It would be nice if we could inject a test double implementation of an interface without requiring access to the real implementation of that interface.

Here we turn our Dependency into an abstract interface.

Dependency.cs:

public interface Dependency {
    int generate();
}
And we implement it with an anonymous class.

Class.cs:
public class Class {
    public class DependencyImpl implements Dependency {
        public int generate() {
               return 999;
           }
    }

    private Dependency dependency = new DependencyImpl();

    public int generate() {
        return dependency.generate() * 2;
    }
}
Finally we try to replace the real dependency implementation with a test double using only the shared interface (for this test we have to pretend the real implementation is not available).

ClassTest.cs:
public class ClassTest {
    @MockClass(realClass = Dependency.class)
    public static class MockDependency {
        @Mock
        public int generate() {
            return 123;
        }
    }

    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, MockDependency.class);

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}
Unfortunately JMockit produces the following error report when we do this:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.595 sec <<< FAILURE!
testIt(org.duderino.injection.jmockit._12.ClassTest)  Time elapsed: 0.003 sec  <<< FAILURE!
java.lang.IllegalArgumentException: Not a modifiable class: org.duderino.injection.jmockit._12.Dependency
	at org.duderino.injection.jmockit._12.ClassTest.testIt(ClassTest.java:22)

Results :

Failed tests:   testIt(org.duderino.injection.jmockit._12.ClassTest): Not a modifiable class: org.duderino.injection.jmockit._12.Dependency

Tests run: 14, Failures: 1, Errors: 0, Skipped: 0

13. Safely replace a class library class with a test double

Yes, JMockit supports this, but the resulting solution is brittle.

It would be nice if we could safely mock out core classes from the class library without destabilizing the entire system.

In a simple test of this, we create a class that depends on java.io.FileInputStream to return a number read from a file.

Class.java:
public class Class {
    public int generate() throws Exception {
        FileInputStream stream = new FileInputStream("foo");

        byte[] bytes = new byte[3];

        stream.read(bytes);

        String string = new String(bytes, "ASCII");

        return 2 * Integer.parseInt(string);
    }
}
In our test case we replace the FileInputStream core class with a test double that returns a canned response:

ClassTest.java:
public class ClassTest {
    @Test
    public void test() throws Exception {
        Mockit.setUpMock(FileInputStream.class, new InputStream() {
            private byte[] bytes = new byte[]{0x31, 0x32, 0x33};
            private int index = 0;

            @Mock
            void $init(String fileName) {
                assert "foo".equals(fileName);
            }

            @Mock
            public int read() throws IOException {
                if (index >= bytes.length) {
                    return 0;
                }

                return bytes[index++];
            }

            @Mock
            public int read(byte[] out) throws IOException {
                for (int i = 0; i < out.length; ++i) {
                    int result = read();

                    if (0 == result) {
                        return i;
                    }

                    out[i] = bytes[i];
                }

                return out.length;
            }
        });

        Class clazz = new Class();

        assert 2 * 123 == clazz.generate();
    }
}

While this works, more complicated cases do not work.   Consider the following Configuration class that retrieves key value pairs from an external XML file.

Configuration.java:

public class Configuration {
    private Map<String, String> map = new HashMap<String, String>();
    private final String path;

    public Configuration(String path) throws Exception {
        this.path = path;

        reload();
    }

    public String get(String key) {
        return map.get(key);
    }

    public void reload() throws Exception {
        map.clear();

        DocumentBuilder docBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();

        Document document = docBuilder.parse(new FileInputStream(path));

        NodeList settings = document.getElementsByTagName("setting");

        for (int i = 0; i < settings.getLength(); ++i) {
            Node setting = settings.item(i);

            NodeList children = setting.getChildNodes();

            String key = null;
            String value = null;

            for (int j = 0; j < children.getLength(); ++j) {
                Node child = children.item(j);

                if ("key".equals(child.getNodeName())) {
                    key = child.getNodeValue();
                    continue;
                }

                if ("value".equals(child.getNodeName())) {
                    value = child.getNodeValue();
                    continue;
                }
            }

            if (null != key) {
                map.put(key, value);
            }
        }
    }
}

We can replace the FileInputStream with a similar test double that returns a canned XML document.

ConfigurationTest.java:
public class ConfigurationTest {
    @Test
    public void test() throws Exception {
        final String xml = "<settings><setting><key>foo</key><value>bar</value></setting></settings>";

        Mockit.setUpMock(FileInputStream.class, new InputStream() {
            private byte[] bytes = xml.getBytes();
            private int index = 0;

            @Mock
            void $init(String fileName) {
            }

            @Mock
            public int read() throws IOException {
                if (index >= bytes.length) {
                    return 0;
                }

                return bytes[index++];
            }

            @Mock
            public int read(byte[] out) throws IOException {
                for (int i = 0; i < out.length; ++i) {
                    int result = read();

                    if (0 == result) {
                        return i;
                    }

                    out[i] = bytes[i];
                }

                return out.length;
            }
        });

        Configuration configuration = new Configuration("README");

        assert "bar".equals(configuration.get("foo"));
    }
}

The problem we see here is that other code we depend on also uses FileInputStream. 

Output:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.718 sec <<< FAILURE!
test(org.duderino.injection.jmockit._13.ConfigurationTest)  Time elapsed: 0.022 sec  <<< FAILURE!
java.io.IOException: Bad file descriptor
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(FileInputStream.java:220)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2961)
	at com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:299)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1742)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.arrangeCapacity(XMLEntityScanner.java:1619)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipString(XMLEntityScanner.java:1657)
	at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:193)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:124)
	at org.duderino.injection.jmockit._13.Configuration.reload(Configuration.java:32)
	at org.duderino.injection.jmockit._13.Configuration.<init>(Configuration.java:20)
	at org.duderino.injection.jmockit._13.ConfigurationTest.test(ConfigurationTest.java:49)

Results :

Failed tests:   test(org.duderino.injection.jmockit._13.ConfigurationTest): Bad file descriptor

Tests run: 15, Failures: 1, Errors: 0, Skipped: 0

In general, the more dependencies the class under test has, the greater the chance of a conflict with a test double.

If necessary narrow targeting tricks can be borrowed from section 11.   Since these tricks are brittle, we say that the ability to replace core class libraries, while possible, is also brittle.

14. Inject async-friendly test doubles

Yes, JMockit supports this.

Sometimes dependencies call back on the classes that depend on them.   This pattern is especially common in asynchronous programming.

To test this we make our Dependency call back on the class under test.

Dependency.cs:

public class Dependency {
    public void generate(Class clazz) {
        clazz.callback(999);
    }
}
The class under test similarly returns a value collected by its callback.

Class.cs:
public class Class {
    private Dependency dependency = new Dependency();
    private int result = 0;

    public void callback(int result) {
        this.result = result;
    }

    public int generate() {
        dependency.generate(this);

        return 2 * result;
    }
}
For our test case we can simply inject a test double that calls back on the class yet passes a value of our choosing.

ClassTest.cs:
public class ClassTest {
    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, new Dependency() {
            @Mock
            @Override
            public void generate(Class clazz) {
                clazz.callback(123);
            }
        });

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}

15. Inject thread-friendly test doubles

No, JMockit does not support this.

When testing concurrent code, it can be useful to inject different test doubles for different threads.  

It would also be nice to inject test doubles from any thread in the program.   The JMockit runtime must be threadsafe itself to support this.

To test this we changed our Dependency to return a value derived from the thread ID.

Dependency.cs:

public class Dependency {
    public long generate() {
        return Thread.currentThread().getId() + 999;
    }
}
No change to our class under test though.

Class.cs:
public class Class {
    private Dependency dependency = new Dependency();

    public long generate() {
        return dependency.generate() * 2;
    }
}
In our test class we launch 10 concurrent threads.    Each thread injects a test double that returns another value tied to the thread ID, yet different from the real dependency.   This lets us distinguish between the test double and the real implementation as well as between test doubles created by different threads.

ClassTest.cs:
public class ClassTest implements Runnable {
    @Override
    public void run() {
        final long magicNumber = Thread.currentThread().getId() + 123;

        Mockit.setUpMock(Dependency.class, new Dependency() {
            @Mock
            @Override
            public long generate() {
                return magicNumber;
            }
        });

        Class clazz = new Class();

        assert 2 * magicNumber == clazz.generate();
    }

    @Test
    public void testIt() throws InterruptedException {
        Thread[] threads = new Thread[10];

        for (int i = 0; i < threads.length; ++i) {
            threads[i] = new Thread(this);
        }

        for (int i = 0; i < threads.length; ++i) {
            threads[i].start();
        }

        for (int i = 0; i < threads.length; ++i) {
            threads[i].join();
        }
    }
}
Unfortunately we saw that all threads shared the same test double / the test double was not thread-specific.  As a result 7/10 thread assertions failed due to the race condition between injecting the test double and invoking the test double in the class under test.

Output:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Exception in thread "Thread-10" Exception in thread "Thread-4" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-7" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-3" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-11" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-5" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-9" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-12" java.lang.AssertionError
	at org.duderino.injection.jmockit._15.ClassTest.run(ClassTest.java:25)
	at java.lang.Thread.run(Thread.java:680)

16. Associate failed assertions in test doubles with test cases in same thread

Yes, JMockit supports this.  To be sure, this is due to TestNG not JMockit.

When test doubles fail assertions, those failures should be associated with the test case.

Here we create a simple class with a dependency.

Dependency.cs:

public class Dependency {
    public int generate() {
        return 999;
    }
}
Class.cs:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}
In our test case we inject a test double that fails an assertion.

ClassTest.cs:
public class ClassTest {
    @Test
    public void testIt() {
        Mockit.setUpMock(Dependency.class, new Dependency() {
            @Mock
            @Override
            public int generate() {
                assert 1 == 2;

                return 123;
            }
        });

        Class clazz = new Class();

        assert 123 * 2 == clazz.generate();
    }
}
In the testng result file we can see that the assertion was indeed associated with the test case.

testng-results.xml:
      <class name="org.duderino.injection.jmockit._16.ClassTest">
        <test-method status="FAIL" signature="testIt()" name="testIt" duration-ms="8" started-at="2011-12-03T17:50:15Z" finished-at="2011-12-03T17:50:15Z">
          <exception class="java.lang.AssertionError">
            <full-stacktrace>
              <![CDATA[java.lang.AssertionError
        at org.duderino.injection.jmockit._16.ClassTest$1.generate(ClassTest.java:17)
        at org.duderino.injection.jmockit._16.Dependency.generate(Dependency.java)
        at org.duderino.injection.jmockit._16.Class.generate(Class.java:10)
        at org.duderino.injection.jmockit._16.ClassTest.testIt(ClassTest.java:25)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:122)
        at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:92)
        at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:101)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
        at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
        at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:172)
        at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:104)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:70)
]]>
            </full-stacktrace>
          </exception>
        </test-method>
      </class>


17. Associate failed assertions in test doubles with test cases in different threads

No, JMockit does not support this.  To be sure, this is due to TestNG not JMockit.

When test doubles fail assertions, those failures should be associated with the test case.   When the test double is executed in a different thread than the test case, it's considerably more difficult for the test case to associate the assertion on one thread with the test case on another.

Here we again create a simple class with a dependency.

Dependency.cs:

public class Dependency {
    public int generate() {
        return 999;
    }
}
Class.cs:
public class Class {
    private Dependency dependency = new Dependency();

    public int generate() {
        return dependency.generate() * 2;
    }
}
In our test case we simply fail an assertion in a different thread than the test case.   There's no JMockit here at all.

ClassTest.cs:
public class ClassTest implements Runnable {
    public void run() {
        assert 1 == 2;
    }

    @Test
    public void test() throws InterruptedException {
        Thread[] threads = new Thread[10];

        for (int i = 0; i < threads.length; ++i) {
            threads[i] = new Thread(this);
        }

        for (int i = 0; i < threads.length; ++i) {
            threads[i].start();
        }

        for (int i = 0; i < threads.length; ++i) {
            threads[i].join();
        }
    }
}
Even though assertions are thrown, since TestNG is not in the call stack the aren't associated with the test case.   TestNG thinks the test case passed.

testng-results.xml:
      <class name="org.duderino.injection.jmockit._17.ClassTest">
        <test-method status="PASS" signature="test()" name="test" duration-ms="11" started-at="2011-12-03T18:02:46Z" finished-at="2011-12-03T18:02:46Z">
        </test-method>
      </class>

console:
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
JMockit: loaded external tool mockit.coverage.CodeCoverage
Running TestSuite
Exception in thread "Thread-1004" Exception in thread "Thread-1003" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1005" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1012" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1011" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1010" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1009" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1008" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1007" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1006" java.lang.AssertionError
	at org.duderino.injection.jmockit._17.ClassTest.run(ClassTest.java:11)
	at java.lang.Thread.run(Thread.java:680)
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.811 sec

Results :

Tests run: 18, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.925s
[INFO] Finished at: Sat Dec 03 18:02:46 PST 2011
[INFO] Final Memory: 9M/81M
[INFO] ------------------------------------------------------------------------

It's possible though that if we had configured TestNG to run each test case in a different process TestNG might have been able to make the association.

Next Steps

To see a similar evaluation for Moles, another test double injection framework, see our Moles Evaluation.