Automated Build – Getting better

By Henning Eiben in Programming 0 Comment

A while ago I wrote a post, describing how to setup a continues build system using CruiseControl.Net. In that post I showed how to get your project ready be continuously being build.

It’s all about improvement

That was a big step towards continues improvement in the software developing lifecycle. The next big thing is to be always up-to-date with the quality of your software and to make sure that all those bugs get fixed real soon.

To accomplish this, I created a unit-test project for my application. This can be used to do test-driven development or just to test the code you just wrote. However – you basically want to run this code whenever you change something and keep an close eye on the red-green status-icons. Whenever I make some modifications this should not affect existing features.

This means, we have to run the rest on a regular basis. This is especially important when we want to deliver some bits. We definitely want to make sure, that all test pass before we ship something.

This is where our continues integration server comes back into the game. We already have this setup to make a new build of the project whenever something changes – so now all we need to do is to make sure all our unit-tests are being run as well.

Updating the project

First of, we need to extend our NAnt build-file. We add a new target, which will run the unit-tests. I prefer to use Gallio to run my unit-tests, because this allows me to use a big variety of unit-testing-frameworks without changing the execution runtime.

<target name="run-tests" description="Run Unit tests">
    <loadtasks assembly="C:\Program Files\Gallio\bin\Gallio.NAntTasks.dll"/>
    <trycatch>
      <try>
        <gallio report-types="Xml-Inline;Html"
                report-directory="testresults"
                report-name-format="Pizza.Testing.dll-results"
                runner-type="NCover"
                failonerror="true">
          <runner-property value="NCoverCoverageFile='testresults\coverage.xml'" />
          <runner-property value="NCoverArguments='//a Pizza.Testing'"/>
          <files refid="fileset_test_assemblies">
            <include name="Pizza.Testing.dll" />
          </files>
        </gallio>
      </try>
      <catch property="failure">
        <echo message="At least one test failed: ${failure}"/>
      </catch>
    </trycatch>
    <loadtasks assembly="C:\Program Files\NCoverExplorer\NCoverExplorer.NAntTasks.dll"/>
    <echo message="Running NCoverExplorer report generation ..."/>
    <ncoverexplorer program="C:\Program Files\NCoverExplorer\NCoverExplorer.console.exe"
                    reportType="3"
                    xmlReportName="testresults\coveragereport.xml"
                    satisfactoryCoverage="80">
      <fileset>
        <include name="testresults\coverage.xml"/>
      </fileset>
    </ncoverexplorer>
</target>

This target will run all tests in the Pizza.Testing assembly of the project. Furthermore all test-results are being written to a file called Pizza.Testing.dll-results.xml.

Finally I also run the NCoverExplorer to get a nice summary of my code coverage. You might notice the runner-type=NCover, which causes Gallio to run all unit-tests within the NCover context. This way we get also some nice numbers of which code is actually covered by unit-tests.

Doing it on the server

After getting the build-file up and running we can make some adjustments to the project-configuration within CCNet (ccnet.config). Obviously we need to make sure, that the newly created target is being called during the build-process. So that’s fairly simple.

<nant description="main build">
    <executable>d:\nant\bin\NAnt.exe</executable>
    <buildFile>d:\ccnet\repositories\Pizza\Source\default.build</buildFile>
    <targetList>
      <target>Compile run-tests</target>
    </targetList>
 </nant>

But we also need to make sure, we collect all data, that is being recorded during the tests (like the test results). Since this is being recorded in a separate xml file, we need to merge this file into the main build-log, which is being maintained by CCNet.

<publishers>
    <merge>
        <files>
            <file>.\source\build\release\test-results\coverage*.xml</file>
            <file>.\source\build\release\test-results\*-results.xml</file>
        </files>
    </merge>
  <xmllogger />
  <statistics />
  <modificationHistory />
</publishers>

Showing what was done

So now we have the tests automated, and setup to run with each build. The final step is to show the test-results in the project dashboard.

For the results to show up, we have to modify the dashboard configuration in that respect, that the results are being rendered into HTML. So far they have only been merged into the build-file log.

This is being done by adding new XSL-transformations to the configuration (dashboard.config). These are part of the Gallio-package.

<xslReportBuildPlugin description="Gallio Test Report"
  actionName="GallioTestReport"
  xslFileName="gallio\xsl\Gallio-Report.ccnet-details.xsl" />
<xslReportBuildPlugin description="Gallio Test Report (Condensed)"
  actionName="GallioTestReportCondensed"
  xslFileName="gallio\xsl\Gallio-Report.ccnet-details-condensed.xsl" />
<xslReportBuildPlugin description="NCoverExplorer Report"
  actionName="NCoverExplorerBuildReport"
  xslFileName="xsl\NCoverExplorer.xsl" />
<gallioAttachmentBuildPlugin />

After that we can have a look on how many tests succeed and how many failed. The display looks just like the Gallio Runner.

ccnet_gallio

ccnet_ncoverexplorer

Who shall be tested?

By Henning Eiben in .Net, Programming 0 Comment

In a time, where test-driven-development (TDD) is becoming an established way to write a software you naturally write tons of (unit-)testing-code (that’s why it’s called test-driven).

OK, so you define some functionality, which should be exposed by a certain component. So you will most likely have a couple of public methods supporting this functionality. To make sure everything works as expected you write your usual testing-code.

So far so good. So imagine you have a couple of public methods exposing your predefined functionality, and a whole lot of other (private) methods doing some helper-stuff.

During the testing phase you will suddenly find out, some tests are failing. The good developer you are, you have messages associated with your asserts, so will have a clue on why something went wrong. But still, you will have to digg into your code to find, that some helper-methode is all over sudden screwing your whole component.

At this point in time you might want to write tests for those helper methods as well, just to make sure these helpers work expected – just like you do for your public methods. But wait, there is a problem! Usually your unit-tests will reside in a separate assembly, so you will not be able to call those private helper methods at all. But fortunatly there is help on the way … using reflection you can circumvent this problem.

Fortunatly MbUnit has already built in support, for testing such kind of methods using reflection.

Let’s assume you have a class called BusinessClass with a private helper methode DoSomething, which takes a string as a parameter, does some processing and returns some other string. So an appropriate test would be:

using System;
using MbUnit.Framework;
using MbUnit.Framework.Reflection;
[TestFixture]
public class SampleTests
{
[Test]
public void DoSomethingTest()
{
BusinessClass foo = new BusinessClass();
string output = (string)Reflector.InvokeMethod(AccessModifier.NonPublic, foo, "DoSomething", "my input string");
Assert.AreEqual("my output string", output);
}
}

Unit-Testing with HttpContext

By Henning Eiben in .Net, Programming 0 Comment

One big bummer of unit-testing (especially with nunit) is, that you cannot test web-based code.

OK, so I kinda gave up on that one. But I recently developed quite some code, targeted to ease development of web-based apps. So this code is making use of the HttpContext in order to get to know something about who’s actually make the request etc.

So with regards to Phil Haacked I wrote a little method to inject a fake HttpContext and thus could at least write some unittest for my project.

But unfortunatly HttpContext.Current.User is returning null – so I might have to look a little bit around. I already figured, that Phil did an update on his original post – I’ll check that out!

MBunit on Steroids?

By Henning Eiben in .Net, Programming 0 Comment

Well, according to Scott Hanselmann MbUnit is supposed to be Unit Testing on Crack – I would prefer it to be on steroids (as I consider that to be more political correct, because I wouldn’t want to have a drug addict on my team :)) ). But anyway!

So I checked out MbUnit, and so far I think it has quite some neat features. After working for some years with NUnit I long for something new, which satisfies more complex requirements for unit-testing. Event though there are a couple of extensions to NUnit to enabled row-based testing, I consider MbUnit as an alternativ.

I will most definitely give it a try in one of my next projects.