Pex & Moles: The Basics

This is a guide to quickly getting started with Pex and Moles.

To create this tutorial, we installed the following:

We first tried this on Microsoft XP Professional with Service Pack 2, but were only able to get Pex and not Moles working.

Parameterized Unit Testing With Pex

Pex explores different inputs to a test case in order to generate a minimal set of inputs that exercise as many code paths as possible.

This style of testing is called 'Parameterized Unit Testing'. It's 'Parameterized' in the sense that Pex will generate inputs for every parameter exposed by the test case. If the exposed parameters are able to exercise all code paths, Pex can generally find the input values necessary to reach them. If some code paths are controlled by parameters not exposed to Pex, Pex cannot generate inputs capable of reaching them without assistance.

Creating a simple class to test

First let's create a simple Y2KChecker class to illustrate Pex in action. If the caller passes in a DateTime equal to January 1st, 200, the Y2KChecker class will throw an ApplicationException. Otherwise it won't. So there are two paths we want Pex to find.

Let Pex generate a starter test suite

Next we ask Pex to generate a unit test suite for us.

Inspect Pex's generated top-level test suite

As you can see, Pex will generate a new unit test suite. It will generate a test case for every public method in every public class in the solution.

The generated top-level test classes are meant to be modified by the programmer. Pex uses C# partial classes for this. Later, Pex will generate additional partial classes that pass different inputs to the methods of the top-level classes. These must not be modified by the programmer since they should be regenerated whenever class interfaces or side-effects change. So programmer-modified code and periodically regenerated code will be stitched together at runtime to create complete test classes.

Each [PexMethod] in a top-level test class will expose all parameters in the code under test to the frequently regenerated partial classes.

Each [PexMethod] in a top-level test class can be modified by the programmer to do additional assertions and, as we will see later, inject test doubles using Moles.

Run Pex Explorations...

Now let's tell Pex to explore the code under test. Pex will generate different inputs and see how they exercise different code paths at runtime. Pex will ultimately reduce the set of inputs to a minimally sufficient set - one input per distinct code path.

Pex found the Y2K Bug!

When done with its exploration Pex will generate 1+ partial classes to pass the inputs it found to the methods in the top-level, programmer-modifiable test cases.

In this case Pex generates two different 'checks' - one that triggers the ApplicationException throw and one that does not.

Good code coverage

At this point we can inspect the generated code coverage report and confirm that for the method under test code coverage is 100%.

Tell Pex the exception is not a bug

Without instruction Pex will assume that any thrown exception is a test failure. You can tell Pex, however, that any thrown exception is in fact to be expected.

Rerun explorations

Next tell Pex to rerun its exploration.

Note the [PexRaisedException(typeof(ApplicationException))] attribute on the Pex check. This will change as a result of allowing the exception.

And everything is green

After rerunning explorations Pex will regenerate the checks. Note that the [PexRaisedException(typeof(ApplicationException))] attribute has been replaced with [PexAlloweddException(typeof(ApplicationException))].

And code coverage is still 100%

The Pex checks still achieve the same code coverage after we told Pex a thrown ApplicationException should not fail the test case.

Introducing Moles

Moles is test double generation and injection framework complimentary to Pex.

Create a new class to test that isn't parameterized

We can illustrate Moles' complimentary nature by removing the DateTime parameter from the class under test's Check() signature.

After we make the change, we ask Pex to generate a new unit test suite.

Run Pex Explorations...

As before, after generating the top-level test classes, we as Pex to explore different inputs and generate a set of parameter generating test cases.

Pex didn't generate much

This time, however, Pex wasn't able to do much. Since we removed the DateTime parameter from the Check() method, all Pex can do is generate a single check that calls the method without arguments.

And according to the code coverage report, it missed the Y2K bug

Not surprisingly, the code coverate report shows that the line which throws the ApplicationException was not exercised by Pex's generated test suite. Unless we can somehow expose the DateTime.Now property to Pex there's nothing Pex can do here.

Moles to the rescue! Generate Moles Types (Test Doubles)

We can address this issue by using Moles to expose the DateTime.Now property to Pex.

First we add a Moles Assembly for the assembly that contains DateTime, mscorlib. This will generate a 'mscorlib.moles' XML file and nothing else.

Compile the stubs

Next time the solution is built, however, Moles will see the mscorlib.moles file and generate 'Moles Types' (stubs) for every public class in mscorlib.

Inspect the generated Moles Types

Now inspect the generated stubs after the build completes using the Object Browser...

Inspect the DateTime Stub

Moles prepends an 'M' to the name of every class it stubs to avoid name collision. Since we're interested in the 'DateTime' stub, we search for 'MDateTime' in the object browser.

Looking at MDateTime's methods, we can see that a NowGet getter method was generated for the Now property. We will be able to assign a function delegate to this method to which Moles will reroute DateTime.Now calls.

Please refer to Appendix B in the Moles Reference Manual for complete description of the methods generated for each Moles Type.

Import the Moles Stubs into the Pex-Generated Top-Level Test Suite

Before we can use the generated Moles Types, we have to import them into our test suite. Here we import Microsoft.Moles.Framework for the Moles infrastructure itself and System.Moles for the generated Moles Types for mscorlib. In general, Moles Types are imported by appending a '.Moles'.

Turn the test into a parameterized test using Moles

Now we can expose the DateTime.Now parameter to Pex.

Here we...

Add a [HostType("Moles")] attribute to the top-level generated Pex test case
Expose a DateTime input to Pex
Assign a delegate to MDateTime.NowGet. This delegate will use C# closures to return the DateTime object generated by Pex

When the Y2KChecker.Check() method is called, internally Moles will reroute the DateTime.Now property getter to our delegate.

Delete the old generated tests

For some reason Pex complains and won't regenerate its tests because we manually modified one of its PexMethods. If we delete the old generated tests Pex will be happy.

Rerun Pex Explorations

Now that we've exposed the DateTime parameter to Pex, we can tell Pex to rerun its explorations and potentially find new inputs.

Pex rediscovered the Y2K bug with Moles' help

Like our original example, Pex was again able to find an input that throws the ApplicationException. The only difference is this time the input was injected with Moles rather than passed in through the class under test's public API.

Tell Pex to allow the ApplicationException

Once again we tell Pex to allow the ApplicationException.

Note the PexAllowedException Annotation

As before Pex will change the [PexRaisedException(typeof(ApplicationException))] attribute to [PexAllowedException(typeof(ApplicationException))].

Tell Pex to rerun its explorations

This will make Pex regenerate the test cases to allow the exception.

Everything is green!

And everything passes again!

And the code coverage report confirms the Y2K bug was triggered

And we're back to 100% code coverage for the Check() method even though DateTime is no longer passed in.

How does dynamic routing work with the debugger?

Digging a little deeper, we can see that Moles is well integrated into Visual Studio. We first set a break point in the code under test.

Run the unit tests under the debugger

Next we run the entire test suite under the debugger.

Confirm the Moles Stub was Injected

When our breakpoint is hit we can inspect the DateTime.Now object and confirm that the values of our MolesType were injected. To the programmer everything looks the way it should.

Next Steps

If this didn't work for you and you want to see a complete working example, our test code has been checked in at https://github.com/duderino/injection/tree/master/MolesTest.

If you want to keep going, see our Moles Evaluation for a more detailed overview of Moles.

See also the official Pex & Moles documentation.