Visual Studio 2012 Update 1

Update 1 for Visual Studio 2012 is now available. There are a number of bugs fixes and pending features added to this update. The details can be found form the blog post http://blogs.msdn.com/b/somasegar/archive/2012/11/26/visual-studio-2012-update-1-now-available.aspx and the update can be downloaded from here.

Unit Tests using Microsoft Fakes framework in Team Build.

My last post was about Microsoft Fakes Framework. One thing to know about using Microsoft Fakes is that MSTest.exe does not support it. MSTest.exe used to be the executable that Visual Studio 2010 and earlier versions used to run unit tests. It is not the case anymore Visual Studio 2012 uses a command line tool vstest.console.exe. It resides in the %ProgramFiles%\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\ directory of your visual studio installation.

VSTest.console.exe provides most of the features of MSTest.exe  (there are a few alternatives), however, the version that was shipped with the initial release of Visual Studio, did not support publishing test results to TFS. This will change with Visual Studio 2012 Update 1, which is due to be released soon, but if you are a daredevil, you can try the Visual Studio CTP which is available for download now.

Also, if you are still using Team Foundation Server 2010, the default build template using MSTest activity, which uses MSTest.exe behind the scenes. This means that some of the unit tests which are working for you in Visual Studio environment might not work on the Team Build.

I am working on providing an activity in the Community TFS Build extension project to fill up this gap. For now, you can use InvokeProcess activity passing in all the command line parameters.

Microsoft Fakes Framework

If you have done any Test Driven Development than chances are that you have used one or more mocking frameworks. Mocking frameworks are great in that they isolate your code module from the layers underneath thus allowing you to execute your unit tests without elaborate test set up code. Also, they allow replicating different test scenarios. You can set up different responses from same calls to external components for different test scenarios.

With Visual Studio 2012, Microsoft has introduced a new Mocking framework called Fakes. To date, it is only available with the Ultimate edition of visual studio. The framework is comparable to other open source mocking frameworks such as RhinoMocks and Moq but has the following two advantages over others

1. Great Integration with Visual Studio: The integration with Fakes with Visual Studio 2012 is brilliant. To create a "fake" of an assembly, a user needs to just right click on the references and click the option "Add Fake Assembly".

2. Shims: Most mocking framework works by allowing developers to create a "Stub" of an object. Stubs are objects created on the fly, which implements the same interface as the original object. The caller controls the behaviour of the stub by specifying responses for the methods in the interface. Shims are different in that they "inject" code in the original assembly i.e. although the original object will be created but the code executed will the one specified in the shim. This is quite powerful in that it allows developers to specify behaviour of system assemblies, sealed classes and non-virtual methods. E.g. if you are writing some code that need to do something on the 1st day of a year and you are using System.Now, you can "Shim out" this System.Now to return first of January.

When you add a fakes assembly in your test project by right clicking on the reference assembly and clicking the "Add Fakes Assembly" option, visual studio adds the following files

1. A new "Fakes" folder is created with a .fakes files created in it. For example, if you are creating fakes for "System.Configuration", a file called "System.Configuration.fakes" will be created. 

2. A new "FakesAssemblies" folder is created with three files created namely originalAssembly.assemblyversion.Fakes.dll, originalAssembly.assemblyversion.Fakes.xml and originalAssembly.assembly.Fakes.fakesconfig. So, for "System.Configuration" assembly the files generated would be System.Configuration.4.0.0.0.Fakes.dll, System.Configuration.4.0.0.0.Fakes.fakesconfig and System.Configuration.4.0.0.0.fakes.xml.

Visual Studio also adds a reference to the generated assembly in the "FakesAssemblies" folder.

The files in the "Fakes" folder are only generated at the time of adding the fakes assembly. They are added to the solution and should be checked into source control. 

The "FakesAssemblies" folder and all the files in it are generated whenever the project is compiled. This is important because if you are adding fakes for assembly for a changing component, the generation of FakesAssemblies will ensure that all changes are reflected in the generated assembly.

The purpose of this post is not to write details about Fakes and how to use it, but to mention about a "Gotcha" that you might stumble upon while working with it. I spent some time trying to fix that and wanted to write about it so that no one else gets burnt.

The issue was that one of the test projects using fakes was giving a compilation error when built by Team Build and failed with the following error

"Could not resolve this reference. Could not locate the assembly "System.Configuration.4.0.0.0.Fakes". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors."

The error would go away if I added the FakesAssemblies folder to TFS. This is not how it is meant to work. On closer examination, the reason this error happened was because the “Build action” for the .fakes file added to the project was not set correctly. These files should have the build action set to "Fakes". 

It was set to “None” 

and this is the reason why the fakesassemblies were not getting generated in the team build.