Thursday, July 29, 2010

UnitTests and Code Coverage in TFS Build 2008

The last couple of days I've been working on getting code coverage results from unittests in automated TFS builds. Getting the code coverage results in Visual Studio 2008 is very straightforward:
  • Double click on the LocalTestRun.testrunconfig
  • Go to "Code Coverage"
  • Select the artifacts to instrument (dll's)
  • Enter the path to the resigning key file
  • Click "Apply" and "Close"
Everything is now set up to run the UnitTests with the instrumentation for code coverage.After you run your UnitTests, left click on a TestResult and open the Code Coverage Results.

Getting the Code Coverage Results in the Automated TFS Build is a bit tricky.
Open the TFSBuild file and add the following line in the PropertyGroup node:  "<RunTest>true</RunTest>"

In the ItemGroup for UnitTesting add the following lines (if you have multiple unittest projects in your Build, add one for each unittest project)

<MetaDataFile Include="$(FolderPath)/TestListEditor.vsmdi">
  <TestList>NameOfTestListToRun</TestList>
  <TestRunConfig>$(FolderPath)/LocalTestRun.testrunconfig</TestRunConfig>
</MetaDataFile>

After you run your build you should see that there are now UnitTest Results and Code Coverage Results in your Build Summary.

Unfortunately this wasn't the case in my project. I got a partially succeeded build, the building itself  succeeded and all my unittest passed, but still I got a partially succeeded build. I noticed that the buildlog contained a lot of the following warnings: "Warning VSP2013: Instrumenting this image requires it to run as a 32-bit process. The CLR header flags have been updated to reflect this." After some google searches I found this.
The article basically says that the buildserver makes 64 bit dll's and it can't place the instrumentation code for Code Coverage in these dll's because the Code Coverage runs as a 32 bit proces. Microsoft doesn't plan to fix this in the current release, but keep an eye out for new versions. After some more searching I found some other fixes for this problem, you can change the BuildConfiguration to always build Debug|x86 or you can do some Registry hacks on the buildserver. I didn't like any of these solutions so I started to look for different options.

The best option I could find was the following:
In the LocalTestRun.testrunconfig under Code Coverage there's an option to "Instrument assemblies in place" if you unselect this option the Code Coverage instrumention will be placed in a copy of the dll and not in the same dll. Unselect this option and restart the build.
You will now see that the build will complete succesfully!