JMockit: No holds barred testing with instrumentation over mocking

Introduction

The transition from junit to stubs to mocking occurred long back in the world of testing. Traditionally mocking has used jdk proxies for interface based classes and cglib for non-final concrete classes. This article draws attention to the yet ongoing transition from mocking to instrumentation for testing. Particularly it highlights how to achieve complete access to your API by being able to instrument and thereby mock areas of your code that were previously inaccessible.

In exploring this it becomes evident that testing as it stood prior to instrumentation required one to alter the design of production code fundamentally to accommodate testing. Prime examples of such practices have been the non-production related use of interfaces, widening the visibility of methods and not using the final and static keywords. Conversely the practices of overusing dependency injection, interfaces and instance based state, all for testing, have become all too common and established under the purported benefit that they make code more testable.

Examples

A clear delineation between production code and testing code finally becomes possible with JMockit which uses the java.lang.instrument package of JDK5 and ASM to modify bytecode at runtime and thereby has access to all regions that are accessible in the bytecode itself.

The following sections illustrate the most powerful features of JMockit that distinguish it from proxy based mocking tools with each code example being self contained and runnable – complete with imports. In each case the test subject API is contained within the test case itself for ease of reference and not having to switch back and forth. All source code is also available as a self contained eclipse maven project. Update [11/11/2009]: This download now has all fixes and enhancements to code and maven pom file suggested by, Rogério Liesenfeld, the author of JMockit.

The term ‘mocking’ is used here loosely to refer to mocking through instrumentation as it is common vocabulary. Strictly speaking, however, there is a distinction between mocking which creates a second instance of the test subject and instrumentation which mostly modifies or redefines the original test subject. However, here, these terms will be used interchangeably. Also please excuse the slight contrived and convoluted nature of the test subject API.

Mock classes that don’t implement interfaces

Classes that don’t implement interfaces cannot be mocked using jdk proxies and generally have to be deferred to cglib for subclassing. In this case neither is necessary. Here the @RunWith annotation loads the javaagent contained in jmockit.jar through the entry point of JMockit.class into the JVM prior to the testcase running. The @Mocked annotation designates which classes you’d like instrumented. The Expectations block encloses the specifications of how you’d like your classes instrumented. The end result is that the user name of ‘joe’ is replaced with ‘fred’.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockNoInterfaceClassTest {

    static class User {
        String name() {
            return "joe";
        }
    }

    @Mocked
    User user;

    @Test
    public void mockNoInterfaceFinalClass() {

        new Expectations() {
            {
                user.name();
                returns("fred");
            }
        };

        assertEquals("fred", user.name());

    }

}

Mock final classes and methods

As no proxying is done JMockit is able to modify the bytecode of the test subject without needing to subclass the subject.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockFinalClassTest {

    static final class User {
        final String name() {
            return "joe";
        }
    }

    @Mocked
    User user;

    @Test
    public void mockFinalClass() {

        new Expectations() {
            {
                user.name();
                returns("fred");
            }
        };

        assertEquals("fred", user.name());

    }

}

Mock internal (hidden) instantiations

The User might have a dependency on Address and as illustrated below neither the User nor the test class may have access what the Address is doing. This is referred to here as an internal or hidden invocation or instantiation. Since only the behaviour of Address is of any interest then only that class is marked for instrumentation. Note particularly that no dependency injection of any kind is necessary to make this code testable. JMockit can instrument internally managed dependencies.


package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockInternalInstantiationTest {

    static class User {
        Address address() {
            return new Address();
        }
    }

    static class Address {
        String postCode() {
            return "sw1";
        }
    }

    @Mocked
    Address address;

    @Test
    public void mockInternalInstantiation() {

        new Expectations() {
            {
                new Address().postCode();
                returns("w14");
            }
        };

        String postCode = new User().address().postCode();
        assertEquals("w14", postCode);

    }

}

Mock static methods

Static methods can be instrumented in jmockit in exactly the same way as instance methods.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockStaticMethodTest {

    static class User {
        static String name() {
            return "joe";
        }
    }

    @Mocked
    @SuppressWarnings("unused")
    private final User user = null;

    @Test
    public void mockStaticMethod() {

        new Expectations() {
            {
                User.name();
                returns("fred");
            }
        };

        assertEquals("fred", User.name());

    }

}

Mock native methods

Being able to alter the behaviour of native methods is one of the most powerful features of JMockit. Frequently native code isn’t mockable in the traditional way due to being final or static. However in the example below the JMockit API provides a means of substituting the original native method with one that you define. The end result here is that Runtime.getRuntime().availableProcessors() returns 999. That’s a far cheaper way to approximate Azul.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Mock;
import mockit.MockUp;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockNativeMethodTest {

    @Test
    public void mockNativeMethod() {
        new MockUp<Runtime>() {
            @Mock
            @SuppressWarnings("unused")
            int availableProcessors() {
                return 999;
            }
        };
        assertEquals(999, Runtime.getRuntime().availableProcessors());
    }

}

Mock private methods

Private methods are instrumentable through reflection based utilities provided by JMockit. Here the user name is a private field with no mutator methods to alter it. It is also a final field. Note that despite all that the Expectations block is able to modify the field, record an expected invocation of the accessor method and specify a return value for it. Although the API is slightly different than what it is for visible methods nevertheless it is concise and intuitive.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Deencapsulation;
import mockit.Expectations;
import mockit.Mocked;
import mockit.NonStrict;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockPrivateMethodTest {

    static class User {

        private final String name = "joe";

        @SuppressWarnings("unused")
        private final String name() {
            return name;
        }

    }

    @Mocked
    @NonStrict
    User user;

    @Test
    public void mockPrivateMethod() {
        new Expectations() {
            {
                invoke(user, "name");
                returns("fred");
            }
        };
        assertEquals("fred", Deencapsulation.invoke(user, "name"));
    }

}

Mock default constructors

Here the default constructor of an object is substituted by a constructor specified in the test case itself. The only additional API necessary here is the registration of the new constructor body with JMockit in the beforeClass() method prior to the test case running. The new constructor body is denoted by a special method name called $init which is named after its form in the class file.

package jmockit.examples;

import static org.junit.Assert.assertEquals;

@RunWith(JMockit.class)
public class MockDefaultConstructorTest {

    static class User {

        String name;

        public User() {
            this.name = "joe";
        }

        String name() {
            return name;
        }

    }

    private static final User user = new User();

    @Test
    public void mockDefaultConstructor() {
        new MockUp<User>() {
            @Mock
            @SuppressWarnings("unused")
            public void $init() {
                user.name = "fred";
            }
        };
        new User();
        assertEquals("fred", user.name());
    }

}

Mock static initialiser blocks

In the same way as constructors can be redefined so can static initialiser blocks. The only difference is that the new static initialiser block must be represented by a method called $clinit which again is named after its form in the class file after the jvm has compacted all static initialiser blocks into one block.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Mock;
import mockit.MockUp;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockStaticInitialiserBlockTest {

    static class UserInitialiser {
        static {
            User.name("joe");
        }
    }

    static class User {

        private static String name;

        static void name(String name) {
            User.name = name;
        }

    }

    @Test
    public void mockStaticInitialiserBlock() throws ClassNotFoundException {
        new MockUp<UserInitialiser>() {
            @Mock
            @SuppressWarnings("unused")
            public void $clinit() {
                User.name("fred");
            }
        };
        Class.forName(UserInitialiser.class.getName());
        assertEquals("fred", User.name);
    }

}

Verify internal (hidden) method invocations

Verifications can be an untuitive and confusing concept to grasp so here is a brief account of the jmockit test run lifecycle. The Expectations block puts in place the record phase. Once that block has ended it automatically switches to replay phase. The execution phase is the test subject API invocations themselves. So where do the verifications fit in? Well if Expectations can be thought of as the pre-expectations then verifications are post-expectations. However note that verifications can only complete those expectations that haven’t already been recorded in the Expectations block.

In the example below the creation and population of the User pojo is entirely hidden. So the testcase has no pre-expectations of return values or exceptions being thrown. The only thing the test case would like to verify is that the User was populated through specific method invocations in a given order with a specified set of arguments. The Verifications block verifies exactly that although one can also have optional verifications that are in any order with arguments matching supplied patterns. Note that in the verification of the setName() method the test case specifies that the method invocation must have taken place with an argument of exactly ‘fred’. However with setAge() it specifies only that the method should have been invoked with any integer. These are hamcrest matchers and are add incredible leeway in specifying invocation criteria.

package jmockit.examples;

import mockit.FullVerificationsInOrder;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class VerifyInternalMethodsTest {

    static class User {

        String name;
        int age;

        public void setName(String name) {
            this.name = name;
        }

        public void setAge(int age) {
            this.age = age;
        }

    }

    static class UserService {

        void populateUser() {
            User user = new User();
            user.setName("fred");
            user.setAge(31);
        }
    }

    @Mocked
    User user;

    @Test
    public void verifyInternalMethods() {
        new UserService().populateUser();
        new FullVerificationsInOrder() {
            {
                User user = new User();
                user.setName("fred");
                user.setAge(withAny(1));
            }
        };

    }

}

Mock method invokes real method

Here the mock method is invoking the real method in the test subject. JMockit allows a special ‘it’ instance variable in a mock class that is set to the real class instance for each indirect call to a mock method. Unusual and interesting although it’s not immediately obvious why one might use this feature. It is also disabled by default as it can potentially result in stack overflow due to each calling the other in an infinite recursive loop.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import mockit.Expectations;
import mockit.Mock;
import mockit.MockClass;
import mockit.Mockit;
import mockit.integration.junit4.JMockit;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockInvokesRealMethodTest {

    static class User {
        String name() {
            return "joe";
        }
    }

    @MockClass(realClass = User.class)
    static class MockUser {
        User it;

        @Mock(reentrant = true)
        String name() {
            return it.name();
        }
    }

    @BeforeClass
    public static void beforeClass() {
        Mockit.setUpMocks(MockUser.class);
    }

    @AfterClass
    public static void afterClass() {
        Mockit.tearDownMocks();
    }

    @Test
    public void mockInvokesRealMethod() {

        new Expectations() {
            {
                new User().name();
            }
        };

        assertEquals("joe", new User().name());
    }

}

Mock JDK randomness

Ultimately the power of JMockit is used to mutate certain JDK methods that otherwise are inaccessible and behave very differently! This is done to show that the reach of JMockit extends even to the most diverse behaviours in the JDK and also of course for amusement and intrigue.

package jmockit.examples;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertSame;

import java.security.SecureRandom;

import mockit.Expectations;
import mockit.Mock;
import mockit.MockUp;
import mockit.Mocked;
import mockit.integration.junit4.JMockit;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(JMockit.class)
public class MockJdkRandomnessTest {

    @Test
    public void mockSystemNanoTime() {
        new MockUp<System>() {
            @Mock
            @SuppressWarnings("unused")
            long nanoTime() {
                return 0L;
            }
        };
        assertSame(0L, System.nanoTime());
    }

    @Test
    public void mockMathRandom() {
        new MockUp<Math>() {
            @Mock
            @SuppressWarnings("unused")
            double random() {
                return 0D;
            }
        };
        assertEquals(0D, Math.random(), 0D);
    }

    @Mocked
    SecureRandom secureRandom;

    @Test
    public void mockSecureRandom() {

        new Expectations() {
            {
                new SecureRandom().nextInt();
                returns(0);
            }
        };
        assertSame(0, new SecureRandom().nextInt());
    }

}

Conclusion

JMockit, by way of instrumentation, creates a new genre of testing toolkit that exposes the fundamental limitations and weaknesses in the preceding genre of traditional mocking tools. Critically, the design of production code, should remain unaffected by testing of that code. Interfaces should only be used when felt necessary for production code and there should be no penalty for using the final or static keywords or other language features. By operating at a lower level of abstraction and at a different stage of the VM lifecycle JMockit is able to reinstate the use of the language in its entirety*.

*There’s one language feature which JMockit does not support due to the fact that it is not distinctly supported in the bytecode itself. This is left as a quiz for the reader. 10 points to the first winner. Sorry – points are all I have to give.

Download

All source code above is available as a self contained eclipse maven project. Update [11/11/2009]: This download now has all fixes and enhancements to code and maven pom file suggested by, Rogério Liesenfeld, the author of JMockit.

10 thoughts on “JMockit: No holds barred testing with instrumentation over mocking

  1. Looks like it would be handy for testing in multi-threaded situations too…perhaps. For instance, in the following code its difficult and messy, to test inside the catch for InterruptedException. Not sure about your question, but thinking about Type Erasure or perhaps some Annotation issue…need to think harder on that one.

        @Override
        public void run() {
            try {
                produceFilteredFiles(this.rootSearchFolder, this.fileProcessor, this.folderFilter);
            } catch (InterruptedException ex) {
                logFailure(LOG, ex);
                Thread.currentThread().interrupt();
            } finally {
                logExit(LOG);
                filesProducedCdl.countDown();
            }
        }
  2. Martin – indeed. In expectations you could throw interrupted exception for produceFilteredFiles and verify that appropriate handling code is called accordingly. So far the quiz guesses have been synchronisation, serialisation, type erasure and annotations. All fascinating guesses but not the feature I was thinking of. A hint would be that it is a little used syntactic feature of a class that helps it initialise its state. Thanks for stopping by.

  3. Must be the instance initializer?

    int i;
    { i=2;}

    Introduced in Java 1.3 for Anonymous classes. They get inserted into every constructor of the class at runtime. Its not possible to define a constructor for an Anonymous class but they are given a default constructor. The static initializer is inserted into that constructor giving you a way to initialize things in Anonymous classes. I don’t know how it gets from the bytecode, to the constructor at runtime, but I would guess its not part of the actual class as such. Go on…how does it work?

  4. Martin – 10 points. Quote from JMockit:

    “The Java compiler does not preserve instance initializers as separate elements in the class file, instead inserting any statements in such blocks into each and every constructor, right after the necessary call to the superclass constructor. Therefore, it is not possible to separately mock instance initialization blocks.

    Of course it’s not a limitation as such because JMockit can mock constructors but it is interesting.

  5. Thanks for the article, it’s very well written!

    I spotted a few small mistakes, though:

    In MockPrivateMethod, an invocation is recorded for User#name(), but later (in the replay phase) the test never calls the method. The test passes because the expected field value was set directly on it, and then read in the “assert”. Instead, the calls to “setField” and “getField” could be removed, with the last line becoming assertEquals("fred", Deencapsulation.invoke(user, "name"));.
    In MockDefaultConstructor, the Mockit.redefineMethods was used when it should be Mockit.setUpMock or Mockit.setUpMocks (a MockUp class would also do). “redefineMethods” is an older method (Core API) which knows nothing about the @Mock and @MockClass annotations (it could be used in tests written with JDK 1.4 code).
    The same occurs in MockStaticInitialiserBlock. BTW, a nicer way to apply the MockUserInitialiser mock class would be to annotate the test class with @UsingMocksAndStubs(MockUserInitialiser.class).
    In VerifyInternalMethods, the “@Mocked” field should be of type User, not UserService. With that, the userService.populateUser(); line inside the verification block should be removed.
    With regard to verification blocks, they are used to verify non-strict expectations only, which are those recorded inside a NonStrictExpectations block, or belonging to a @NonStrict mocked type, or like in this case when no invocations at all are recorded for the mocked type. All other cases are strict expectations, which are verified by JMockit implicitly and do not allow “unexpected” invocations.

    There are other insteresting features not mentioned in the article, as well:

    Local mock fields (declared inside expectation blocks) and mock parameters in test methods.
    Delegate objects passed to the “returns” method.
    Argument matching fields, which allow you to write user.setAge(anyInt); instead of user.setAge(withAny(1));.
    The @Capturing annotation, which tells JMockit to mock unspecified implementation classes on demand, as they are loaded by the JVM.

  6. I downloaded the Maven project and tried running the tests through the Maven “test” goal.
    At first, I got the message that “no tests were found”.

    I eventually figured out that Maven only finds test classes whose names end in “Test”, so I renamed them.

    With that, Maven finds all test classes and runs them, but fails because neither tools.jar nor “-javaagent:jmockit.jar” was specified. Assuming a JDK 1.6 environment, the best option here is to add the following dependency to the pom.xml:

            <dependency>
                <groupId>com.sun</groupId>
                <artifactId>tools</artifactId>
                <version>1.6.0</version>
                <scope>system</scope>
                <systemPath>${java.home}/../lib/tools.jar</systemPath>
            </dependency>
    

    With this, all tests ran succesfully in my environment. (Remember to remove/change the “skipTests”, though.)

  7. Rogério what an honour and a pleasure to receive feedback from no other than the JMockit author himself! I’m new to JMockit but having come across I’ve become fascinated by it. I’ve made the fixes and enhancements you suggested to the existing example test classes and I’ve also updated the maven pom file so that all tests can be run using maven. Originally I was running only in Eclipse one at a time and I presumed that others would do the same but maven ideally should work as well. The new project download has all these changes.

    Given the massive span of JMockit’s feature set this article certainly wasn’t intended to be exhaustive. My primary goal was to offer the most compelling and fundamental advantages of JMockit over other non-instrumentation based mocking tools to my colleagues both from the point of view of generating interest but also to provoke thought and also naturally to provide examples as a spring board. I’ll certainly take on board your suggestions for other aspects of JMockit that are interesting and perhaps do another article on them.

    Your comments are very gratefully received. Thanks for stopping by.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s