To setup or not to setup

Posted by Chris on October 20, 2006

Recently I have been trying the “BDD”:http://behaviour-driven.org/
(Behavior-Driven Development) approach to developing software. Normally, when I am doing TDD there are a couple of “house-rules” that I like to follow. These have developed over time, often following advice from either a colleague or other resource.

One such “rule” that has developed over time is that I tend not to use the SetUp and TearDown methods that the xUnit tools have. These are used to execute some piece of setup and/or teardown code before/after every test method is executed. The reason to use these is of course that you might have some code that is needed to setup the system under test, and this setup code tends to be the same for all test methods in a fixture. To keep things “DRY”:http://en.wikipedia.org/wiki/Don’t_repeat_yourself you will naturally want to put this code in a single place and have it executed along with each test. The xUnit tools have different ways of accomplishing this. NUnit for instance uses reflection to find methods marked with the SetUpAttribute in a class and executes them before each test method is executed.

As I started out doing writing unit tests these methods seemed like a great idea. As soon as I learned about them all test fixtures would have them, in fact the first thing I did when creating a new fixture was to add these methods with a “copy-paste template” (or using snippets or similar when they exist). However, after a while a feeling of something being wrong started showing. Discussing the matter with “others”:http://www.taylor.se/blog and doing some “online reading”:http://www.agileprogrammer.com/dotnetguy/articles/SetupTeardown.aspx led me to the conclusion that SetUp/TearDown was *evil incarnated*.

One of the best things about using TDD is that you almost never need to do any debugging. One of the reasons for this is that when you do some modification to your code and then run the tests, if there is anything wrong you will instantly get feedback about it from a red test. You read the name and location (fixture) of the test and take a look at it, and if you have done things right the test will tell you exactly what it does. With this you will hopefully be able to more or less immediately figure out what you did wrong, and fix it. However, if the test is not well written and does not quickly and easily tell you what it does, then you lose this feedback, or at least a part of it. So what has this got to do with SetUp/TearDown? Well, when the test called Foo in fixture Bar blows up on you and you take a look at it, you want to quickly see what it does. If you are using a SetUp method then you will not get the full picture by simply looking at the test. You also need to take a look at the SetUp, and possibly the TearDown (and then TestFixtureSetUp/TestFixtureTearDown if it is really bad). And, of course we must also know that our xUnit tool works this way, since the test method code shows no evidence of a setup method being called. So, instead of using these tools that xUnit gives us, we should instead be refactoring the common setup code into a separate method and then call that method “explicitly” from each test method. That way it is clear when we look at the test method what it does.

Here is an example of a test fixture written this way (in C# using NUnit):

using System;
using NUnit.Framework;
using SystemUnderTest;

namespace SystemUnderTest.Tests
{
	[TestFixture]
	public class AccountWithBalance100_WithoutSetup {
		private Account account;

		private void Init() {
			account = new Account(100);
		}

		[Test]
		public void Depositing50LeavesBalanceOf150() {
			Init();
			account.Deposit(50);
			Assert.AreEqual(150, account.Balance);
		}

		[Test]
		public void Withdrawing50LeavesBalanceOf50() {
			Init();
			account.Withdraw(50);
			Assert.AreEqual(50, account.Balance);
		}

		[Test]
		public void Withdrawing100LeavesBalanceOf0() {
			Init();
			account.Withdraw(100);
			Assert.AreEqual(0, account.Balance);
		}

		[Test]
		[ExpectedException(typeof(ArgumentException))]
		public void Withdrawing101ThrowsException() {
			Init();
			account.Withdraw(101);
		}
	}
}

So, that is the end of that story the, right? Well, I started out this blog entry writing about BDD, not about SetUp/TearDown. So I guess I need to tie this together now. As I said I have been trying BDD instead for a while now. Apart from calling tests specifications and fixtures contexts, there is not a whole lot different between TDD and BDD. At least on the surface, that is. The whole reason to change the terminology is to “force” people in doing TDD the right way. This means using the tests (specifications) to specify behavior, not testing bugs. This means that you will think a bit differently, depending on how you are used to thinking with TDD. It might not be a huge step for all, but for me it has made me reflect a bit.

I did not notice it until after a while, but one interesting reflection I make now is that I do not follow some of my old house-rules when specifying in BDD. Take this example, in “Boo”:http://boo.codehaus.org/ using “Specter”:http://specter.sourceforge.net/:

import System
import Specter
import SystemUnderTest

context "An account with a balance of 100":
	account as Account

	setup:
		account = Account(100)

	specify "Depositing 50 should leave a balance of 150":
		account.Deposit(50)
		account.Balance.Must.Equal(150)

	specify "Withdrawing 50 should leave a balance of 50":
		account.Withdraw(50)
		account.Balance.Must.Equal(50)

	specify "Withdrawing 100 should leave a balance of 0":
		account.Withdraw(100)
		account.Balance.Must.Equal(0)

	specify "Withdrawing 101 should throw an exception":
		{ account.Withdraw(101) }.Must.Throw(typeof(ArgumentException))

This code example shows a typical BDD context and specifications the way I have been writing them. Note the setup part. Specter, the “xUnit tool” I am using here, sees this and executes the setup code before each specification is executed. I used it without even thinking about it. The way the specs are qritten, following the *Given* _an account with a balance of 100_, *when* _a withdrawal of 50 is made_ *then* _there should be 50 left_ style, it seems so natural to setup the context in this way. I suppose it is also largely due to the way specifications is so often written using only a single row, or at least very short.

So, when I made this reflection, I thought that if you write unit tests following the one-assert-per-test-method rule, and of course write short test methods, and (maybe most important of all) create a test fixture per “situation” (or context…), then why should it not feel as natural to use setup here? Here is an example of the same tests as above but this time with the init code moved into a SetUp method.

using System;
using NUnit.Framework;
using SystemUnderTest;

namespace SystemUnderTest.Tests
{
	[TestFixture]
	public class AccountWithBalance100_WithSetup {
		private Account account;

		[SetUp]
		public void Init() {
			account = new Account(100);
		}

		[Test]
		public void Depositing50LeavesBalanceOf150() {
			account.Deposit(50);
			Assert.AreEqual(150, account.Balance);
		}

		[Test]
		public void Withdrawing50LeavesBalanceOf50() {
			account.Withdraw(50);
			Assert.AreEqual(50, account.Balance);
		}

		[Test]
		public void Withdrawing100LeavesBalanceOf0() {
			account.Withdraw(100);
			Assert.AreEqual(0, account.Balance);
		}

		[Test]
		[ExpectedException(typeof(ArgumentException))]
		public void Withdrawing101ThrowsException() {
			account.Withdraw(101);
		}
	}
}

I am not quite finished with my thinking about this, so I am not sure if I think this is better. But I do not think that one of these tests, when blowing up in the test runner, would give me less information than the ones in the example without using the SetUp method. Since I know that the failing test is in the “AccountWithBalance100″ fixture I can easily guess what the variable account holds. But I guess if the setup is more complex then it might not be as easy to name the fixture and/or understand the code.

Comments? Anyone else using BDD that find themselves using setup differently from when doing TDD?

Trackbacks

Trackbacks are closed.

blog comments powered by Disqus