Since I started using Git a few months ago, and particularly GitHub since last September, my life have been good in DVCS land.

I used to be a Subversion user and I was happy*, maybe because I didn’t know anything better. Of course I heard all the cool kids (specially in the Ruby world) talking about Git and bragging about it, but I was being loyal to my old pal SVN.

But this was slowing me down and I wasn’t aware of it. I’m not sure what make me install msysgit for the first time. After struggling a little bit at the beginning and wrapping my head around the concepts and the commands I noticed that my flow started to change.

This change was good. Was more Agile. Doing small commits for little changes of code seems natural since the speed of this commits was almost instantaneous (remember the commits happens in your own machine). I’m talking of commits been done almost at the line of code level.

Change this line, run the test, commit. Write another test. Make it fail. Write some code. Make it pass. Commit. Refactor. Run test. Commit. Keep going.

Once It was time to push those changes to the master a single command push every change + every comment at blasting speed.

Since them I started using Mercurial as well and the experience was very similar. In the case of Mercurial I just use TortoiseHg but I stick with the command line for Git for most of my operations and I’m glad of that decision.

I wasn’t planning on writing this post today but I just happen to grab the code for edgecase Ruby Koans from GitHub and it took me all of 1 minute to do everything. Here is the flow:

Open Console2 and launch the Git bash. Type the following sequence of commands.

Welcome to Git (version 1.6.4-preview20090730)
Run 'git help git' to display the help index.
Run 'git help <command>' to display help for specific commands.
Hernan@HERNAN-STUDIO /c/Development
$ cd Code/Ruby/Koans_unresolved/
Hernan@HERNAN-STUDIO /c/Development/Code/Ruby/Koans_unresolved
$ git pull git://github.com/edgecase/ruby_koans.git
fatal: Not a git repository (or any of the parent directories): .gi
t
Hernan@HERNAN-STUDIO /c/Development/Code/Ruby/Koans_unresolved
$ git init
Initialized empty Git repository in c:/Development/Code/Ruby/Koans_
unresolved/.git/
Hernan@HERNAN-STUDIO /c/Development/Code/Ruby/Koans_unresolved (mas
ter)
$ git help
Hernan@HERNAN-STUDIO /c/Development/Code/Ruby/Koans_unresolved (mas
ter)
$ git pull git://github.com/edgecase/ruby_koans.git
remote: Counting objects: 294, done.
remote: Compressing objects: 100% (286/286), done.
remote: Total 294 (delta 184), reused 0 (delta 0)Receiving objects:  44% (130/29Receiving objects:  45% (133/294)
Receiving objects: 100% (294/294), 51.84 KiB, done.
Resolving deltas: 100% (184/184), done.
From git://github.com/edgecase/ruby_koans
 * branch            HEAD       -> FETCH_HEAD
Hernan@HERNAN-STUDIO /c/Development/Code/Ruby/Koans_unresolved (master)
$

 

Notice that I had to type help because I didn’t remember if I had to add a remote or I was able to do a pull right away (I removed the help output from the console output fro brevity).

Of course the Ruby Koans are just a bunch of very simple text files but the whole process took me less than 2 minutes. I think that is not just because Git is faster but also using the command line for this type of task tends to be a lot faster than using a Gui tool.

If you haven’t try it yet go ahead and take the plunge. Go distributed, even if you work alone, you want regret it.

*I’m still using Subversion at work, looking forward to have the time to switch but we are dealing with some time sensitive projects so I don’t want to mess around with that.

I try to be pragmatic in my approach to development and one of the things that change the most are the tools I use to deliver software. There are several reasons why I change tools so often.

1. The evolution of the software I write call for new or different tools.
2. Search for efficiency and eliminate pain points.
3. I change my approach to software development and new methodologies need new tools.
4. I just love to try new stuff

In this post I want to concentrate in the tools used to manage projects and builds. Until recently my favoured tools in this context consisted of NAnt, CC.Net/Hudson and Subversion.

For bug tracking I used the bug tracker in Google code for my OSS projects integrated to TortoiseSVN or BugTracker.Net. (Note: At work we use FogBugz and we are very happy with it.)

Subversion was the first to go.

Subversion is a great VCS but I usually have two problems with it. Speed and problems with complex merges.

When I work alone, merges are not an issue of course, but speed still is. At the same time more and more people seems to be switching to DVCS (Distributed or Decentralized) systems like Git or Mercurial. After taking a quick look at GitHub and follow their tutorials I was able to move some of my projects there in a few hours. (Most of the time spent on reading and learning from other people mistakes).

Git is fast, Fast, FAST! The whole idea of local commits and clone repositories is fantastic and they really help me with my workflow in the sense that I can do one or two changes, commit, one or two more, commit, find a bug, revert change, finish a story, push to GitHub.

To manage Git I’m using a combination of command line and graphic interface using Git gui. So far I haven’t feel the need to use/install TortoiseGit.

I still have some projects hosted in google code and I’m using Subversion with them, I’m thinking on switch them over to GitHub once I start working on them again.

Nant followed it.

I don’t have a problem with XML so the fact that Nant is XML based does not bothers me, really. But I also like the power of a good scripting language to manage repetitive tasks. I decided to move away from Nant and try to adopt some of the newest build systems.

I took a look at psake and them I read “How a net developer learned ruby and rake to build net apps in windows” by Derrick Bailey and that convinced me to take the plunge.

My knowledge of Ruby is extremely rudimentary but I even with that handicap I was able to have a Rake build script up and running in less than 90 minutes. This script cleans the build target, create the Package folder, builds the VS solution, run Nunit test on two dll's, generate the NUnit report as an xml file, and run the migrations from Migrator.Net (that last part is not working yet, I will need to take a closer look at Migrator.Net command line args).

The most useful resource on Derik's post was Laribee's OMG Rake!. Laribee’s sample code and a few queries to Google was all I need. (No, I did not copy and paste ALL Laribee’s code, just use it as a reference)

Agile Zen or project management made easy.

I got a free account on Agile Zen the first week after they went live. I remember creating the account, playing around on the Board and leaving the site. My thoughts at the moment were something like. This is actually cool. Very nice design. Good usability. I should give it a proper test drive with some real project.

Of course, I never came back until last Friday. While working in a few features for Pronghorn I decided that it was time I stop spiking and started to write proper user stories to drive the framework. I remembered Agile Zen and went back.

The site looks great and there are some improvements since that last time I saw it.

What really amazed me is how intuitive the whole process is. There is no waste generated by using this tool. Creating stories is 1 click. Editing stories, another one. Color coding two clicks. Double click in a field to edit them, etc. Very easy, very intuitive. There is even a bar at the bottom with hints that I decided to hide and I didn’t miss it.

The site is very responsive and the only thing I miss is integration with a VCS system. They have a very simple API for paying accounts, but in the docs it looks like is read only, so no hooks in there (If anybody knows otherwise, please leave a comment).

I will keep using it for another week or so before making a final decision but so far everything indicates that I will be upgrading the account to a paying one pretty soon. (The free account only allows for one project and one developers, what is more than fair.)

Tying it up all together with CI.

As I mentioned before my main experience have been using CruiseControl.Net and Hudson as the CI servers. I installed TeamCity a long time ago to give it a try but I never use it. My first choice was to look for some hosted solution. After searching I only found a few and non of them worked with .Net (at least not at the moment).

I was going to go back once more with Hudson but I wasn’t able to connect to the Hudson site to download the code. (Their site is up now). So I decided to take a new look at JetBrain’s TeamCity. At the moment of this writing they are running the EAP for version 5.

I wanted to install it and use my newly crafted Rake build script with the .Net project and have my test report integrated in the dashboard.

First I got the msi from the daily builds for version 5.00. Double click on it to install the server and the agent and after a few minutes and two (three?) questions, the home page of the serve shows up in the browser.

Creating the first project and associated build profile was a breeze. I just followed along the instructions in the screen. A few times when I wasn’t sure what a given label meant, I made use of the contextual help.

I did have an issue but it was my fault when setting some relative path to the build file and the build target, for running test and creating the Package folder.

From downloading the installer to having the project building successfully took me between 10 to 15 minutes.

Conclusion.

This new setup makes me more productive since I have better tools that save me time in different areas like. Setting things up, working around their idiosyncrasies, configuration heavy or too verbose a language.

While this is a combination that works for me, yours may be completely different. The important part is to have tools that feel this roles and make sure that they are really working with and for you and not the other way around.

I have been playing at home for the last few days with different migrations strategies for db development. The one that I like the most so far is Migrator.Net. Here are the things I really like.

The migration project/code is independent from the technology you use to persist your data, so changing persistence strategies does not affect your migrations.
You have a good migration history, since each new migration is it’s own class. It provides an Up and a down method to run the migration or undo it. You mostly don’t have to write SQL, but you can if you need/want.

Let me clarify that point. I’m not one to mind writing SQL, I rather enjoy it and take pride to been able to write some clever statements. At the same time I recognize that most of the time you write very boring and repetitive task, but what bother’s me the most is the constant context switching from VS to Enterprise Manager. That has to affect your productivity.

The fluent interface is very easy to use, and I only had to consult the documentation to set up the MSBuild file to run the migrations from inside VS, mostly because my experience with MSBuild is very limited. I was tempted to use Nant that I’m very familiar with and they provide a custom task for it, but I use this opportunity as a way to get my hands dirty with MSBuild.

Writing migrations involves creating a new class that derives from Migrations and implement two methods Up and Down, you also need to add a [Migration()] attribute and specify the migration number in it. I decided to use a date+time migration number of the form “yyyyMMddhhmmss” that translates to something like: 20090726120152.

You can also use numbers like 1,2,3,etc… I prefer the date time format to work in a team, less chances of two people creating migrations at the same time.

But the title of the post is about a template for Resharper to create most of the work for you. So that was what I did, and you can download this Zip file with the template in it.

I just saw this project in Codeplex; CruiseControl.Net build station sounds interesting and very useful if you happen to work in a machine without IIS.

UPDATE:

I had a comment for this post, promoting another build system (not open source, not free) because it does not required IIS.

I think that my quick and dirty post should be more clear. You actually do not need IIS to run cc.net in your local computer, just to run the dashboard. There are other systems like TeamCity from JetBrains, that have a free version for developers that it doesn’t need IIS either and is very powerful.

You can also use Hudson, another one that doesn’t requires IIS, free, open source and very extensible.

But sometimes you need to use cc.net and I thought that this was a nice alternative on those situations.

 

If you use Cruise Control.Net you may want to take a look at this project: CC.Net Community PlugIns.

Designed to be a repository for many different plugins for CruiseControl.NET. Hope to have many users contributing plugins to the repository.

 

A post by Igor Brejc with a set of valid points when implementing Continuous Integration.

Ok, the patch can be downloaded from the Issue tracker of the project. This support the following notation in the app.config.

<setting name="IncludedProjects" serializeAs="Xml">
  <value>
    <ArrayOfString xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <string>CCNet</string>
    <string>Ivy.NET</string>
    </ArrayOfString>
  </value>
</setting>

Where each <string> is the name of a project. The project names are case sensitive. If you set a RegEx to filter the projects, the expression will be evaluated first.

This patch have been done over the Revision 125 of the Trunk.

Enjoy!

 

Lava lamps, Orbs, led displays, they all have been used and are still being used to show the build status of projects under Continuous Integration.

What about using a big flat panel TV? Cruise Control provides information via xml as a service and leveraging that information is fairly easy, so why not to present not just the status but also the name of the project in the screen? Well, that is exactly what Big Visible Cruise does.

This is a WCF application, you just put it in a computer connected to a TV. Double click in the exe file, provide the URL for your dashboard and there you go.

It has a property in the configuration file to provide a Regular Expression to indicate witch projects to include in the screen. I prefer to just specify the projects explicitly, so I made a small change in the code to load a list of projects from a txt file and filter the projects using this list. I’m not super happy with the quality of the code, so I will work a little bit more on it before sending a path in case somebody find it useful. I want to use the configuration file to set the projects instead of a text file and probably filter the projects in a different way.

We put the TV near the development team but visible by all the company. We haven’t make a public announcement yet, but I saw some of the guys on the marketing team to looking at it when they walked by and one of them asked me about it. This is exactly what you want great visibility, hopefully will help us to improve communication with the other divisions.

And besides all that, It looks really great.

In my previous post I show how to integrate PartCover reports into CC.Net and promise to look into make the reports look a bit nicer and try to integrate the Coverage metrics into the statistics reports. So I dust off my xslt, pick up two of my favorite books on XML (XML Hacks and XML for the world wide web) open PSPad and started to play around until I got something that I like.Click here to download the stylesheets.

The summary report

The for the summary I want to show a coverage percentage per each assembly in the project so I based my xsl in the assembly.report.xslt provided with PartCover.First I changed the table tag and the first row from this:
<table style="border-collapse: collapse;">···<tr style="font-weight:bold; background:whitesmoke;">
<td colspan="2">Coverage by assembly</td>
</tr>To:<table width="98%" cellspacing="0" cellpadding="2" border="0" class="section-table">
<tr class="sectionheader">
<td colspan="2">Coverage by assembly</td>
</tr>

Then I modified the colours used to identify the different percentage of coverages and made the fonts bold.

Note: I made move all the styles into a stylesheet later on, for now some stuff is hardcoded in the xsl file, what I don't particulary like. When possible I used pre-defined classes already available in the existing cc.net css files


<xsl:variable name="cov0style" select="'background:#ff0000;text-align:right;font-weight:bold;'"/>
<xsl:variable name="cov20style" select="'background:#ff6600;text-align:right;font-weight:bold;'"/>
<xsl:variable name="cov40style" select="'background:#ffcc00;text-align:right;font-weight:bold;'"/>
<xsl:variable name="cov60style" select="'background:#cc9933;text-align:right;font-weight:bold;'"/>
<xsl:variable name="cov80style" select="'background:#6699ff;text-align:right;font-weight:bold;'"/>
<xsl:variable name="cov100style" select="'background:#00cc00;text-align:right;font-weight:bold;'"/>

Then I added this lines of code to alternate the style for each row in the summary.

<xsl:if test="position() mod 2 = 0"><xsl:attribute name="class">section-oddrow</xsl:attribute></xsl:if>

This code needs to be added after opening the table row <tr> inside the for-each loop and before creating the td element or you will have an error on runtime when the server tries to generated the report.I also decided to modify the colour used for 0% coverage, in the original stylesheet it used the same colour for coverage between 0 and 20 percent.I wanted to use a bright red for assemblies with no coverage, so I added the following line after we calculate the coverage percentage.

<xsl:if test="$coverage = 0"> <xsl:attribute name="style"> <xsl:value-of select="$cov0style"/> </xsl:attribute> </xsl:if>

And I modified this line:
<xsl:if test="$coverage >= 0 and $coverage < 20"> <xsl:attribute name="style"> <xsl:value-of select="$cov20style"/> </xsl:attribute> </xsl:if>

to look like this:

<xsl:if test="$coverage > 0 and $coverage < 20"> <xsl:attribute name="style"> <xsl:value-of select="$cov20style"/> </xsl:attribute> </xsl:if>

notice that now I'm checking for $coverage greater than 0 and I deal with $coverage = to 0 in my new line.



Here is the end result:

Adding more details to the details report.

Again I start using the supplied stylesheet but and I made the same changes (or very similar changes as in the previous file).I wanted to provide not just measures by class, but also have detailed measures by method and I wanted this to work the same way as the Nunit report does.You have an arrow besides the class name than when clicked displays the details for the class.To do so I copy some of the code from the unittest.xsl style from cc.netFirst you need to declare the applicationPath parameter<xsl:param name="applicationPath"/>Them you include the javascript to show and hide the details. Them you include the code to display the arrow and finally you add a for each to select all the methods inside the class and calculate the coverage.I decided to mark anything less than 100% as red.It's not because I think that we should achieve 100% of test coverage and if not the project is not complete, it's just that I wanted and easy way to find methods that may need to be tested on more details.Here is an image of the result with no details showing.



And here with details open

Adding the statistics

This actually went very smooth, open your ccnet.config file and add the following under the statistics tag of your project.<statisticList><statistic name="Coverage" xpath="ceiling(100 * (sum(//PartCoverReport/type/method/code/pt[@visit>0]/@len)+0) div (sum(//PartCoverReport/type/method/code/pt/@len)+1))" /></statisticList>The only part that I have to give it any thought at all was in the xpath expression to calculate the project coverage percentage, but I think that I got it, notice the +1 in the second part of the expression, this is to prevent division by zero errors, this shouldn't affect the coverage percentage calculation in any meaningful way.

A few months ago I mentioned that I was using PartCover.Net to measure test coverage in my projects. I have been really busy but I decided that this week I was going to integrate the reports with CC.Net. Cruise Control ships with NCover support out of the box but it's also very easy to integrate other reports as far as they are in xml format and you know (or have) an xml stylesheet to generate html from the report.Partcover comes with two xslt files one to report results by assembly and a second one to report results by class.To use this files in CC.Net you need to make a minor change on them or they don't work.Let start with the class.report.xslt file.Open the file using your favorite text editor (PSPad anybody?) and look for this line:<xsl:for-each select="/PartCoverReport/type">Change it to:<xsl:for-each select="//PartCoverReport/type">Notice the double back slash, this change is needed because the way of CC.Net works is merging all the reports for all your tool together in a big xml file, so at that moment PartCoverReport is not the root element but a child element of the merged xml file.Open the Assembly.report.xslt file and add the "extra" back slash in any mention to /PartCoverReport.

Configuring CC.Net

Copy the modified xslt files into the stylesheets folder of your webdashboard install, usually CruiseControl.Net\webdashboard\xsl\I like to change the name of the files to indicate what report they affect, in this case I named PartCover.assembly.report.xsl and PartCover.class.report.xsl (notice that I drop the "t" from the extension, I don't thing this is important, but all the files in this folder have an xsl extension, this way I keep things consistent).Now we need to configure the reports plug ins, open dashboard.config usually at CruiseControl.Net\webdashborad and under <buildPlugins> <buildReportBuildPlugin> <xslFileNames>

You enter:<xslFile>xsl\PartCover.assembly.report.xsl</xslFile>
This will add a summary on the build report page.
Then under <buildPlugins> <buildReportBuildPlugin>······<buildLogBuildPlugin />
Enter:<xslReportBuildPlugin description="PartCover Details" actionName="PartCoverDetailsBuildReport" xslFileName="xsl\PartCover.class.report.xsl" />

This will add a link on the right navigation bar to see the details of the report for this build.You need to restart IIS for CC.Net to pick up the changes.

You will see a menu item to the PartCover report on the side.



Clicking on the link you will see a simple report like this:


(I had to hide the name of the company) but you will see the qualified name for the class.

Running PartCover.Net from Nant

I decided to Run PartCover from Nant using and exec task.This is an example:

<property name="partCoverReportPath" value="D:\BuildReports\PartCover\${projectName}\" />
<property name="nunitExePath" value="D:/NUnit-2.4.7/bin/nunit-console.exe" />
<target name="testCoverage" depends="UnitTest" description="Measures how much code have been covered by the test">
<mkdir dir="${partCoverReportPath}" unless="${directory::exists(partCoverReportPath)}"/>
<exec program="D:/PartCover/PartCover.exe" failonerror="false">
<arg value="--target=${nunitExePath}" />
<arg value="--target-work-dir=${buildPath}" />
<arg value="--target-args=${projectTests}.dll" />
<arg value="--include=[LaTrompa]*" />
<arg value="--output=${partCoverReportPath}\${projectTests}-Results.xml" />
</exec>
</target>

${projectName} is a property where I store the name of my project.
${projectTest} is a property where I store the name of the project that has all the unit test.
${buildPath} is a property where I store the build folder.

Let me explain each argument:
--target is the Path to your nunit-console.exe
--target-work-dir is the Path to the folder where the dll(s) that contains the test resides.
--target-args is the name of the dll
--include is used to indicate witch namespaces and classes to include in the report, in this cases I'm saying any class under the LaTrompa namespace
--output is where I want to save my report, this is the complete path with the name of the fileThere is another argument that is very useful
--exclude is used to ignore some classes or namespaces from the report.

Adding the report to the build

Now you need to configure your project in CC.Net to merge the report generated by PartCover.To do this, inside the project tag in your ccnet.config file lock for the publishers tag and add a new file under <publishers> <merge> <files> <file>D:\BuildReports\PartCover\LaTrompaLibraries\UnitTesting-Results.xml</file>

......After doing this you can run a build for the project and your PartCover results will be added.

Next steps

I will try in a future post to modify the xsl to have more data, make it look nicer and integrate a bit more with the general look and feel of the CC.Net dashboard.I think that I will be able to provide detailed information per class and method and I will also see how can I modify the Dojo based statistics plugin to include this data.

We were having a problem with our build in the CC.Net server.

The problem was that when doing the clean of the build folder sometimes it was failing with the following error. "Can't delete folder is not empty". The first thing was to check that not other process was accessing the folder. The second step was to make a search on google to figure out if somebody else had the same problem. We only found one case but was too all and they were mentioning to use Nant 0.84 to solve the problem, that is not an option for us since we need support for .Net 3.5.

We have several projects running in the same server with almost identical build scripts, but only three projects have this problem. They have in common that all of them have a large and deep folder structure. Another characteristic of the problem was that the error wasn't always in the same folder, it seemed totally random.

Forcing the build usually solve the problem, but sometimes we have to force the build more than one time, since it keep failing in different folders.
A build that fails randomly is not a good thing to have in a Continuous integration environment. You want your build to fail for good reasons, like a failing test or a threshold on simian or FxCop, in a few words, you want that when a build fails people pay attention to it. (Actually the ideal situation is that your build never fails because developers are running the build on their machines first, but you may have integration errors).

So after hitting my head into the wall for a few hours I decided to write a custom nant task to solve the problem.

The idea is that we have a task (forceDelete) that take two parameters, a target directory and a number of attempts. Internally the task will try to delete recursivelly the given folder and any folder below it, if there is an error will try again for as many times as indicated in the attempts parameter.

I'm including a zip file with the compile dll and another one with the source code. To being able to compile the source code you will need to add a reference to Nant.Core.dll, located in the bin folder of your Nant location.

To use the task you just need to copy LaTrompa.NantTasks.dll inside the bin folder of your Nant install (or whatever is that Nant.Core.dll is in your machine).

LaTrompa.NantTasks.zip
LaTrompa.NantTasks.source.zip

Notes

This task is super simple and at this moment it's not raising and error, something that you may want to do to make your script fails if you still can't delete the given folder after all the attempts.