Moles Evaluation

In this document we evaluate Moles against a set of common unit testing and injection use cases.   To see how JMockit compares to JMockit, another test double injection framework, see our JMockit vs. Moles Comparison.

To get started with Moles, see our Pex & Moles: The Basics.

For the source code illustrated here, check out https://github.com/duderino/injection and focus on the MolesTest subdirectory.

Finally, refer to the Pex & Moles page for official documentation.

1. Inject test doubles without changing the API

Yes, Moles supports this.

Imagine a class with a single dependency.   Can we use Moles to inject a test double in lieu of that dependency?

Class.cs:
    public class Class01
    {
        private Dependency01 dependency = new Dependency01();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

Dependency.cs:
    public class Dependency01
    {
        public virtual int generate()
        {
            return 999;
        }
    }
ClassTest.cs:
    [TestClass]
    public class ClassTest01
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency01.AllInstances.generate = _ => 123;
            
            Class01 clazz = new Class01();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }
Moles prepends an 'M' to the name of every class it stubs to avoid name collision.   Since we're interested in the 'Dependency01' stub, we use the 'MDependency01' stub.

Since we want to replace the generate() function, we simply assign a replacement delegate to MDependency01.AllInstances.generate.   The AllInstances nested class tells Moles that all instances of MDependency01 should reroute their generate function to our delegate.

Please refer to Appendix B in the Moles Reference Manual for complete description of the methods generated for each Moles Type.

Generated Moles Type Names Are Unstable

When classes with the same name in different namespaces are moled (e.g., MolesTest._1.Dependency vs. MolesTest._2.Dependency), Moles will inconsistently add a disambiguating suffix to the generated class name.  Initially when relied on namespaces to disambiguate we saw Moles generate type names like MolesTest._1.Moles.MDependency for MolesTest._1.Dependency and MolesTest._2.Moles.MDependency01 for MolesTest._2.Dependency.   As we added more test cases, we saw the suffix assignment occasionally swap.   Suddenly MolesTest._1.Moles.MDependency was replaced by MolesTest._1.Moles.MDependency01, etc.    Fortunately this broke our build outright rather than introduce subtler errors.   To compensate we started adding our own disambiguating suffixes to our classes (e.g., MolesTest._1.Dependency01) so Moles wouldn't do its own inconsistent disambiguation.

2. Replace out final/non-virtual functions with test doubles

Yes, Moles supports this.

Let's seal the dependency so test doubles cannot subclass it.

Dependency.cs:
    public sealed class Dependency02
    {
        public int generate()
        {
            return 999;
        }
    }
We don't change the class under test for this.

Class.cs:
    public class Class02
    {
        private Dependency02 dependency = new Dependency02();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

Moles has no issues rerouting calls to the sealed class to our delegate.

ClassTest.cs:
    [TestClass]
    public class ClassTest02
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency02.AllInstances.generate = _ => 123;
            
            Class02 clazz = new Class02();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

3. Replace static functions with test doubles

Yes, Moles supports this.

Here we change our dependency's method to be static.

Dependency.cs:
    public static class Dependency03
    {
        public static int generate()
        {
            return 999;
        }
    }
The class under test no longer instantiates the dependency and instead calls its static method.

Class.cs:
    public class Class03
    {
        public int generate()
        {
            return Dependency03.generate() * 2;
        }
    }

The static method can be mocked using the MDependency03.generate delegate.   Use of the AllInstances inner class is not required for mocking out static functions.

ClassTest.cs:
    [TestClass]
    public class ClassTest03
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency03.generate = () => 123;
            
            Class03 clazz = new Class03();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

4. Replace constructors with test doubles

Yes, Moles supports this.

Here we change our dependency to take an argument in its constructor.

Dependency.cs:
    public class Dependency04
    {
        private int value;

        public Dependency04(int value)
        {
            this.value = value;
        }

        public int generate()
        {
            return value;
        }
    }
The class under test is similarly modified to pass the argument through its constructor to the dependency's constructor.

Class.cs:
    public class Class04
    {
        private Dependency04 dependency = new Dependency04(999);

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

Moles generates a distinct function for every constructor.   Here we can assign a delegate to ConstructorInt32 (a constructor that takes one 32bit int). 

ClassTest.cs:
    [TestClass]
    public class ClassTest04
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency04.ConstructorInt32 = (@this, value) => { 
                Assert.AreEqual(999, value);

                var mole = new MDependency04(@this) { generate = () => 123 }; 
            };
            
            Class04 clazz = new Class04();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

This delegate can assert on the 32 bit int that is passed to it as well as dynamically modify the stub by assigining a delegate to the stubs generate function.   Unlike other examples, this one replaces generate only for the instance, not for all instances. 

5. Replace anonymous inner/nested classes with test doubles

This is not applicable to C#.   While C# has anonymous classes, they cannot implement an interface but are instead limited to a collection of simple, read-only properties.   This class to test, for instance, is so contrived it's not very useful.   The anonymous class is really a hard-coded constant and not a true dependency.
    public class Class05
    {
        public int generate()
        {
            var dependency = new { generate = 999 };

            return dependency.generate * 2;
        }
    }

6. Replace inaccessible (e.g., private) inner/nested classes with test doubles

No, Moles does not support this.

For this test we moved the Dependency class into the Class class.

Class.cs:
    public class Class06
    {
        private class Dependency06
        {
            public int generate()
            {
                return 999;
            }
        }

        private Dependency06 dependency = new Dependency06();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }
Next we tried to replace the dependency's generate function with a delegate.

ClassTest.cs:
    [TestClass]
    public class ClassTest04
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MClass06.MDependency06.AllInstances.generate = _ => 123;

            Class06 clazz = new Class06();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

Moles only generates stubs for publicly accessible symbols.  The result was a compilation error:
------ Build started: Project: MolesTest, Configuration: Debug Any CPU ------
  MolesTest -> C:\Users\foo\Documents\Visual Studio 2010\Projects\MolesTest\MolesTest\bin\Debug\MolesTest.dll
------ Build started: Project: MolesTest.Tests, Configuration: Debug Any CPU ------
C:\Users\foo\Documents\Visual Studio 2010\Projects\MolesTest\MolesTest.Tests05\_6\ClassTest06.cs(22,22): error CS0117: 'MolesTest._6.Moles.MClass06' does not contain a definition for 'MDependency06'

Compile complete -- 1 errors, 0 warnings
  Microsoft Moles v0.94.51006.1 - http://research.microsoft.com/moles - .NET v4.0.30319
  Copyright (c) Microsoft Corporation 2007-2010. All rights reserved.
  
  Generating 2 Moles assemblies with 2 concurrent processes
  3:start> MolesTest.moles
  4:start> mscorlib.moles
  4:end> mscorlib.moles Success (0 - 0x0)
  3:end> MolesTest.moles Success (0 - 0x0)
  
  	Moles compilation SUCCESS - 2.5885226s
========== Build: 1 succeeded or up-to-date, 1 failed, 0 skipped ==========

The same test works if the accessibility of the Dependency06 class is changed from private to public.   We also tried making the internals visible to Moles by adding these two lines to the assembly, but they did not help:
[assembly: InternalsVisibleTo("MolesTest._6.Moles")]
[assembly: InternalsVisibleTo("MolesTest.Moles")]

7. Replace subclass and superclass functions with test doubles

Yes, Moles supports this.

To test this use case, we split our dependency into a superclass and a subclass.   The superclass and subclass share one method so we can test overriding and overshadowing.   The superclass and subclass also each have a distinct method that isn't overriden or doesn't overshadow methods in the other.

To tell them apart, the super class always returns '999' and the subclass always returns '333'.

SuperDepenency.cs:

    public class SuperDependency07
    {
        public virtual int generate()
        {
            return 999;
        }

        public int superGenerate()
        {
            return 999;
        }
    }
SubDependency.cs:
    public class SubDependency07 : SuperDependency07
    {
        public override int generate()
        {
            return 333;
        }

        public int subGenerate()
        {
            return 333;
        }
    }
The class under test instantiates the subclass and exposes all three methods to the test case.

Class.cs:
    public class Class07
    {
        private SubDependency07 dependency = new SubDependency07();

        public int generate()
        {
            return 2 * dependency.generate();
        }

        public int superGenerate()
        {
            return 2 * dependency.superGenerate();
        }

        public int subGenerate()
        {
            return 2 * dependency.subGenerate();
        }
    }
Our test class tests four conditions:
  1. Can we inject a test double for the subclass that overrides the superclass?   See testOverride()
  2. Can we inject a test double for the subclass that is overriden by the non-replaced subclass?   See testOvershadow()
  3. Can we inject a test double for the super class that is not overshadowed by the subclass?   See testSuper()
  4. Can we inject a test double for the subclass that does not override the superclass?  See testSub()
ClassTest.cs:
    [TestClass]
    public class ClassTest07
    {
        [TestMethod]
        [HostType("Moles")]
        public void testOverride()
        {
            MSubDependency07.AllInstances.generate = _ => 123;

            Class07 clazz = new Class07();

            Assert.AreEqual(2 * 123, clazz.generate());      // value should come from mock - mock should override
            Assert.AreEqual(2 * 333, clazz.subGenerate());   // value should come from subclass
            Assert.AreEqual(2 * 999, clazz.superGenerate()); // value should come from super class
        }

        [TestMethod]
        [HostType("Moles")]
        public void testOvershadow()
        {
            MSuperDependency07.AllInstances.generate = _ => 123;

            Class07 clazz = new Class07();

            Assert.AreEqual(2 * 333, clazz.generate());      // value should come from subclass - mock should be overshadowed
            Assert.AreEqual(2 * 333, clazz.subGenerate());   // value should come from subclass
            Assert.AreEqual(2 * 999, clazz.superGenerate()); // value should come from super class
        }

        [TestMethod]
        [HostType("Moles")]
        public void testSuper()
        {
            // does not compile: MSubDependency07.AllInstances.superGenerate = _ => 123;

            MSuperDependency07.AllInstances.superGenerate = _ => 123;

            Class07 clazz = new Class07();

            Assert.AreEqual(2 * 333, clazz.generate());      // value should come from subclass
            Assert.AreEqual(2 * 333, clazz.subGenerate());   // value should come from subclass
            Assert.AreEqual(2 * 123, clazz.superGenerate()); // value should come from mock
        }

        [TestMethod]
        [HostType("Moles")]
        public void testSub()
        {
            MSubDependency07.AllInstances.subGenerate = _ => 123;

            Class07 clazz = new Class07();

            Assert.AreEqual(2 * 333, clazz.generate());      // value should come from subclass
            Assert.AreEqual(2 * 123, clazz.subGenerate());   // value should come from mock
            Assert.AreEqual(2 * 999, clazz.superGenerate()); // value should come from super class
        }
    }

While this test case passed, we did observe one limitation of note.   Moles only generates stubs for the declared methods in each class.   Inherited methods are not stubbed out.   This would make it difficult to stub out multiple methods in a class when some methods are declared in the class and others are inherited.

A similar limitation was seen in our JMockit Evaluation.

    

8. Inject a test double that delegates to a real instance

Yes, Moles supports this.

Here we created a simple dependency and class that depends on it.

Dependency.java:
    public class Dependency08
    {
        public int generate()
        {
            return 999;
        }
    }

Class.java:
    public class Class08
    {
        private Dependency08 dependency = new Dependency08();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

In our test class we assign a delegate that 'decorates' the real class to the Moles Type.   Our delegate in turn delegates to the real class, but counts the number of times the real class is called.

ClassTest.java:
    [TestClass]
    public class ClassTest01
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            int count = 0;

            MDependency08.AllInstances.generate = (Dependency08 dependency) =>
            {
                ++count;

                return MolesContext.ExecuteWithoutMoles(() => {
                    return dependency.generate();
                });
            };
            
            Class08 clazz = new Class08();

            for (int i = 0; i < 10; ++i)
            {
                Assert.AreEqual(2 * 999, clazz.generate());
            }

            Assert.AreEqual(10, count);
        }
    }
Here use of MolesContext.ExecuteWithoutMoles let's us call the real class from the test double.

9. Inject hand-coded test doubles

Yes, Moles supports this.

Again we declare a simple dependency and a class that depends on it.

Dependency.cs:
    public class Dependency09
    {
        public int generate()
        {
            return 999;
        }
    }
Class.cs:
    public class Class09
    {
        private Dependency09 dependency = new Dependency09();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }
In our test case we simply make the Moles Type delegate to our hand-coded test double.

ClassTest.cs:
    [TestClass]
    public class ClassTest09
    {
        private class MockDependency
        {
            private int value = 123;

            public int generate()
            {
                return value;
            }
        }

        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            // While this works, it doesn't appear to be threadsafe.

            MockDependency mockDependency = new MockDependency();

            MDependency09.AllInstances.generate = _ =>
            {
                return mockDependency.generate();
            };
            
            Class09 clazz = new Class09();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

If the same hand-coded test double cannot be shared by all Moles Type instances, we could use the technique illustrated in section 4 to create specific delegates for each instance in the Moles Type constructor.

10. Test double injection can be toggled

Yes, Moles supports this.

Again we declare a simple dependency and a class that depends on it.

Dependency.cs:
    public class Dependency10
    {
        public int generate()
        {
            return 999;
        }
    }
Class.cs:
    public class Class10
    {
        private Dependency10 dependency = new Dependency10();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

In our test case we alternate between the test double and the real object by alternating between the Mockit.setUpMock() and Mockit.tearDownMocks() methods.

ClassTest.cs:
    [TestClass]
    public class ClassTest01
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency10.AllInstances.generate = _ => 123;
            
            Class10 clazz = new Class10();

            Assert.AreEqual(2 * 123, clazz.generate());

            MDependency10.AllInstances.generate = null;

            Assert.AreEqual(2 * 999, clazz.generate());

            MDependency10.AllInstances.generate = _ => 123;

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }
}	

11. Test double injection can be narrowly targeted

Yes, Moles supports this, but the resulting solution is brittle.

For this example the dependency is unchanged.

Dependency.cs:
    public class Dependency11
    {
        public virtual int generate()
        {
            return 999;
        }
    }
But we modify the class under test to call the dependency twice.

Class.cs:
    public class Class11
    {
        private Dependency11 dependency = new Dependency11();

        public int generate()
        {
            return dependency.generate() + (2 * dependency.generate());
        }
    }
Can we replace the second generate() call but not the first?

To do this we created a slightly more complicated test double that counts the number of times it has been called.   On every call but the second call it delegates to the real instance.

ClassTest.cs:
    [TestClass]
    public class ClassTest11
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            int calls = 0;

            MDependency11.AllInstances.generate = (Dependency11 dependency) =>
            {
                if (++calls == 2) {
                    return 123;
                }

                return MolesContext.ExecuteWithoutMoles(() =>
                {
                    return dependency.generate();
                });
            };

            Class11 clazz = new Class11();

            Assert.AreEqual(999 + (2 * 123), clazz.generate());
        }
    }
The solution is of course brittle because slight changes to and non-determinism in the code under test will break the narrow targeting.

12. Replace any implementation of an interface with a test double

No, Moles does not support this.

There are cases when the implementation of a class's dependency is not known until runtime and only its interface is known to the compiler.   Such cases are increasingly common with inversion of control frameworks.

It would be nice if we could inject a test double implementation of an interface without requiring access to the real implementation of that interface.

Here we turn our Dependency into an abstract interface.

Dependency.cs:

    public interface Dependency12
    {
        int generate();
    }
Class.cs:
    public class Class12
    {
        // Make this public so Moles can generate a mole for it
        public class DependencyImpl : Dependency12
        {
            public int generate()
            {
                return 999;
            }
        }

        private Dependency12 dependency = new DependencyImpl();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }
Finally we try to replace the real dependency implementation with a test double using only the shared interface (for this test we have to pretend the real implementation is not available).

ClassTest.cs:
    [TestClass]
    public class ClassTest12
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency12.AllInstances.generate = _ => 123;  // Does not compile

            Class12 clazz = new Class12();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }
Unfortunately Moles does not generate Moles Types for interfaces.

13. Safely replace a class library class with a test double

Yes, Moles supports this, but the resulting solution is brittle.

It would be nice if we could safely mock out core classes from the class library without destabilizing the entire system.

In a simple test of this, we create a class that depends on File, FileStream, and StreamReader to return a number read from a file.

Class.cs:
    public class Class13
    {
        public int generate1() 
        {
            StreamReader reader = File.OpenText("foo.txt");

            string line = reader.ReadLine();

            return 2 * int.Parse(line);
        }

        public int generate2()
        {
            FileStream stream = File.Open("foo", FileMode.Open);

            StreamReader reader = new StreamReader(stream);

            string line = reader.ReadLine();

            return 2 * int.Parse(line);
        }
    }
Moles lets us replace core classes with Moles types that return canned values.   In some cases though we have to inject multiple test doubles to handle their interaction.

ClassTest.cs:
    [TestClass]
    public class ClassTest13
    {
        [TestMethod]
        [HostType("Moles")]
        public void test1()
        {
            MFile.OpenTextString = _ => new MStreamReader();

            MStreamReader.AllInstances.ReadLine = _ => "123";

            Class13 clazz = new Class13();

            Assert.AreEqual(2 * 123, clazz.generate1());
        }

 
        [TestMethod]
        [HostType("Moles")]
        public void test2()
        {
            MemoryStream memoryStream = new MemoryStream(new byte[] { 0x31, 0x32, 0x33 });   // "123" in ascii/unicode hex

            MFileStream  mFileStream = new MFileStream();

            mFileStream.CanReadGet = () => memoryStream.CanRead;

            mFileStream.ReadByteArrayInt32Int32 = (byte[] array, int offset, int length) => memoryStream.Read(array, offset, length);

            MFile.OpenStringFileMode = (string path, FileMode mode) => mFileStream;

            Class13 clazz = new Class13();

            Assert.AreEqual(2 * 123, clazz.generate2());
        }
    }

While this works, the resulting solution is nevertheless brittle.   The rerouting Moles does is global for all instances of the real classes.    Core classes are used by many classes.   As a class becomes more complex, the chances increase that one of its dependencies will also use the core class in a way that the test double disrupts.

If necessary narrow targeting tricks can be borrowed from section 11.   Since these tricks are brittle, we say that the ability to replace core class libraries, while possible, is also brittle.

14. Inject async-friendly test doubles

Yes, Moles supports this.

Sometimes dependencies call back on the classes that depend on them.   This pattern is especially common in asynchronous programming.

To test this we make our Dependency call back on the class under test.

Dependency.cs:

    public class Dependency14
    {
        public void generate(Class14 clazz)
        {
            clazz.callback(999);
        }
    }
The class under test similarly returns a value collected by its callback.

Class.cs:
    public class Class14
    {
        private Dependency14 dependency = new Dependency14();
        private int result = 0;

        public void callback(int result)
        {
            this.result = result;
        }

        public int generate()
        {
            dependency.generate(this);

            return 2 * result;
        }
    }
For our test case we can simply inject a test double that calls back on the class yet passes a value of our choosing.

ClassTest.cs:
    [TestClass]
    public class ClassTest14
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency14.AllInstances.generateClass14 = (Dependency14 dependency, Class14 clazz) =>
            {
                clazz.callback(123);
            };
            
            Class14 clazz2 = new Class14();

            Assert.AreEqual(2 * 123, clazz2.generate());
        }
    }

15. Inject thread-friendly test doubles

No, Moles does not support this.   The Moles runtime is not threadsafe and will crash Visual Studio.

When testing concurrent code, it can be useful to inject different test doubles for different threads.  

It would also be nice to inject test doubles from any thread in the program.   The Moles runtime must be threadsafe itself to support this.

To test this we changed our Dependency to return a value derived from the thread ID.

Dependency.cs:

    public class Dependency15
    {
        public int generate()
        {
            return Thread.GetDomainID() + 999;
        }
    }
No change to our class under test though.

Class.cs:
    public class Class15
    {
        private Dependency15 dependency = new Dependency15();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }

In our test class we launch 10 concurrent threads.    Each thread injects a test double that returns another value tied to the thread ID, yet different from the real dependency.   This lets us distinguish between the test double and the real implementation as well as between test doubles created by different threads.

ClassTest.cs:
    [TestClass]
    public class ClassTest14
    {
        public void run()
        {
            int magicNumber = Thread.GetDomainID() + 123;

            MDependency15.AllInstances.generate = _ => magicNumber;

            Class15 clazz = new Class15();

            Assert.AreEqual(2 * magicNumber, clazz.generate());
        }

        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            Thread[] threads = new Thread[10];

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i] = new Thread(this.run);
            }

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i].Start();
            }

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i].Join();
            }
        }
    }

Unfortunately this test case crashed Visual Studio:



While we are not sure why this test case crashed Visual Studio, the Microsoft Moles Reference Manual makes it clear that this is not supported:
Mole types apply to all threads and do not have thread affinity. This is an important fact if you plan to use a test runner that support concurrency: tests involving 
mole types cannot run concurrently.

16. Associate failed assertions in test doubles with test cases in the same thread

Yes, Moles supports this.  To be sure, this is due to Microsoft's unit testing framework and not really Moles.

When test doubles fail assertions, those failures should be associated with the test case.

Here we create a simple class with a dependency.

Dependency.cs:

    public class Dependency16
    {
        public int generate()
        {
            return 999;
        }
    }
Class.cs:
    public class Class16
    {
        private Dependency16 dependency = new Dependency16();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }
In our test case we inject a test double that fails an assertion.

ClassTest.cs:
    [TestClass]
    public class ClassTest16
    {
        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            MDependency16.AllInstances.generate = _ =>
            {
                Assert.AreEqual(1, 2);

                return 123;
            };

            Class16 clazz = new Class16();

            Assert.AreEqual(2 * 123, clazz.generate());
        }
    }

In Visual Studio we can see that the failed assertion was associated with the correct test case:

17. Associate failed assertions in test doubles with test cases in different threads

No, Moles does not support this.   The Moles runtime is not threadsafe and will crash Visual Studio.

When test doubles fail assertions, those failures should be associated with the test case.   When the test double is executed in a different thread than the test case, it's considerably more difficult for the test case to associate the assertion on one thread with the test case on another.

Here we again create a simple class with a dependency.

Dependency.cs:

    public class Dependency17
    {
        public int generate()
        {
            return 999;
        }
    }
Class.cs:
    public class Class17
    {
        private Dependency17 dependency = new Dependency17();

        public int generate()
        {
            return dependency.generate() * 2;
        }
    }
In our test case we simply fail an assertion in a different thread than the test case.   While we aren't using Moles directly here, we have added the [HostType("Moles")] attribute to the test method.

ClassTest.cs:
    [TestClass]
    public class ClassTest17
    {
        private void run()
        {
            Assert.AreEqual(1, 2);
        }

        [TestMethod]
        [HostType("Moles")]
        public void test()
        {
            Thread[] threads = new Thread[10];

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i] = new Thread(this.run);
            }

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i].Start();
            }

            for (int i = 0; i < threads.Length; ++i)
            {
                threads[i].Join();
            }
        }
    }

Like section 15, executing test case crashes Visual Studio.



If we remove the [HostType("Moles")] attribute from the test method, Visual Studio fares better.   Visual Studio does associate the assertions with the test case.   So apparently the Moles runtime is not threadsafe.

Visual Studio emits the following failures with the test case:
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR
Error	12/4/2011 6:38:22 PM	One of the background threads threw exception: 
Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.AreEqual failed. Expected:<1>. Actual:<2>. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual[T](T expected, T actual)
   at MolesTest.Tests._17.ClassTest17.run() in C:\Users\foo\documents\visual studio 2010\Projects\MolesTest\MolesTest.Tests05\_17\ClassTest17.cs:line 21
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.runTryCode(Object userData)
   at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()	BAR

Next Steps

To see a similar evaluation for JMockit, another test double injection framework, see our JMockit Evaluation.