A while ago I wrote a post, describing how to setup a continues build system using CruiseControl.Net. In that post I showed how to get your project ready be continuously being build.

It’s all about improvement

That was a big step towards continues improvement in the software developing lifecycle. The next big thing is to be always up-to-date with the quality of your software and to make sure that all those bugs get fixed real soon.

To accomplish this, I created a unit-test project for my application. This can be used to do test-driven development or just to test the code you just wrote. However – you basically want to run this code whenever you change something and keep an close eye on the red-green status-icons. Whenever I make some modifications this should not affect existing features.

This means, we have to run the rest on a regular basis. This is especially important when we want to deliver some bits. We definitely want to make sure, that all test pass before we ship something.

This is where our continues integration server comes back into the game. We already have this setup to make a new build of the project whenever something changes – so now all we need to do is to make sure all our unit-tests are being run as well.

Updating the project

First of, we need to extend our NAnt build-file. We add a new target, which will run the unit-tests. I prefer to use Gallio to run my unit-tests, because this allows me to use a big variety of unit-testing-frameworks without changing the execution runtime.

<target name="run-tests" description="Run Unit tests">
    <loadtasks assembly="C:\Program Files\Gallio\bin\Gallio.NAntTasks.dll"/>
    <trycatch>
      <try>
        <gallio report-types="Xml-Inline;Html"
                report-directory="testresults"
                report-name-format="Pizza.Testing.dll-results"
                runner-type="NCover"
                failonerror="true">
          <runner-property value="NCoverCoverageFile='testresults\coverage.xml'" />
          <runner-property value="NCoverArguments='//a Pizza.Testing'"/>
          <files refid="fileset_test_assemblies">
            <include name="Pizza.Testing.dll" />
          </files>
        </gallio>
      </try>
      <catch property="failure">
        <echo message="At least one test failed: ${failure}"/>
      </catch>
    </trycatch>
    <loadtasks assembly="C:\Program Files\NCoverExplorer\NCoverExplorer.NAntTasks.dll"/>
    <echo message="Running NCoverExplorer report generation ..."/>
    <ncoverexplorer program="C:\Program Files\NCoverExplorer\NCoverExplorer.console.exe"
                    reportType="3"
                    xmlReportName="testresults\coveragereport.xml"
                    satisfactoryCoverage="80">
      <fileset>
        <include name="testresults\coverage.xml"/>
      </fileset>
    </ncoverexplorer>
</target>

This target will run all tests in the Pizza.Testing assembly of the project. Furthermore all test-results are being written to a file called Pizza.Testing.dll-results.xml.

Finally I also run the NCoverExplorer to get a nice summary of my code coverage. You might notice the runner-type=NCover, which causes Gallio to run all unit-tests within the NCover context. This way we get also some nice numbers of which code is actually covered by unit-tests.

Doing it on the server

After getting the build-file up and running we can make some adjustments to the project-configuration within CCNet (ccnet.config). Obviously we need to make sure, that the newly created target is being called during the build-process. So that’s fairly simple.

<nant description="main build">
    <executable>d:\nant\bin\NAnt.exe</executable>
    <buildFile>d:\ccnet\repositories\Pizza\Source\default.build</buildFile>
    <targetList>
      <target>Compile run-tests</target>
    </targetList>
 </nant>

But we also need to make sure, we collect all data, that is being recorded during the tests (like the test results). Since this is being recorded in a separate xml file, we need to merge this file into the main build-log, which is being maintained by CCNet.

<publishers>
    <merge>
        <files>
            <file>.\source\build\release\test-results\coverage*.xml</file>
            <file>.\source\build\release\test-results\*-results.xml</file>
        </files>
    </merge>
  <xmllogger />
  <statistics />
  <modificationHistory />
</publishers>

Showing what was done

So now we have the tests automated, and setup to run with each build. The final step is to show the test-results in the project dashboard.

For the results to show up, we have to modify the dashboard configuration in that respect, that the results are being rendered into HTML. So far they have only been merged into the build-file log.

This is being done by adding new XSL-transformations to the configuration (dashboard.config). These are part of the Gallio-package.

<xslReportBuildPlugin description="Gallio Test Report"
  actionName="GallioTestReport"
  xslFileName="gallio\xsl\Gallio-Report.ccnet-details.xsl" />
<xslReportBuildPlugin description="Gallio Test Report (Condensed)"
  actionName="GallioTestReportCondensed"
  xslFileName="gallio\xsl\Gallio-Report.ccnet-details-condensed.xsl" />
<xslReportBuildPlugin description="NCoverExplorer Report"
  actionName="NCoverExplorerBuildReport"
  xslFileName="xsl\NCoverExplorer.xsl" />
<gallioAttachmentBuildPlugin />

After that we can have a look on how many tests succeed and how many failed. The display looks just like the Gallio Runner.