Tests gone bad (part 3) – setup overkill

2 May 2013
| |
Reading time: 5 minutes

Automated testing is mandatory for successful software development in an agile process. However, with some man years poured into a project, teams often find themselves slowed down by a tangle of tests that no-one understands any more, cause builds to take a lot of time, and that randomly fail without any conceivable reason. In this series of blog posts, I’m going to dig up some anti-patterns, and discuss possible remedies.
Part 1: Make sure it does what it does
Part 2: Code coverage cargo cult
Part 3: Setup Overkill

This post examines the dreadful effects of repeating yourself in test setup code.

A unit test is easy to define: a single unit is being tested. A unit is just one object or even a single method. Everything that is outside of this unit must not be touched by a unit test.

A good unit tests executes fast because slow operations like network and I/O are simulated. It is stable because only changes within the unit under test require changing the test code.

Set up a test…

So we have got a web application with a view, a controller, a database access repository, and a database. The controller provides a method for filtering names. We want to test if it works correctly.

public void Sort_by_first_letter()
    // Arrange
    var dataSource = new Mock<IDataSource>();
    dataSource.Setup(x => x.GetNames())
              .Returns(new[] {"Alice", "Bob", "Alfred"});
    var subject = new Controller(dataSource.Object);

    // Act

    // Assert
              .BeEquivalentTo(new[] {"Alice", "Alfred"});

What we have here is a test in C#1. For the data source, a stub2 is created that returns three names. It is fed into the Controller via constructor injection. Then, the Filter method is called, and we check that only the names starting with “a” are in the result.

In line 5-7, an isolation framework is used to create a test double that plays the role of the data source. Using this stub, we do not have to include the real repository or database. This is great, but even in this simple example, the code looks kind of complicated already. In real-world projects, we get a lot more of that.

… and then do it again

Now that we know we can filter by a single letter, let’s try two. How do we do that?

Have you just thought “copy, paste, rename, use different names, apply different filter string”?

Congratulations. You’re a very honest person with a positive, pragmatic, can-do attitude. And you’re in good company.

On the other hand, you’re just creating a maintenance nightmare.

  • The Controller class uses constructor injection. Which dependencies are injected is an implementation detail that is likely to change over time. But every time it does, all tests need to provide a setup for all the dependencies.
  • When the DataSource class is extended and the Controller uses a new method, the stub does not know what to return. Again, all tests need to be modified.

As a result, the project becomes obstinate to change. Any change in an implementation detail drags an unforeseeable amount of work. This works directly against agile engineering practices, and can even put a process like Scrum off kilter.

The promised dreadful effects

In a Scrum project, one assumes that teams get more efficient over time. They learn more about their domain and technology, cooperate better, and optimize their processes. For each requirement of a given size (measured in story points), the effort spent should decrease, resulting in a higher development velocity.

Diagram: time to effort, in an ideal situation

Now the drag of redundant test code adds effort to implementing new features. If new features are added as new, isolated units of code, that is not so bad – existing code does not need to change that much. However, most of the time that is not how the specification process in an agile project works. You start with a minimum viable solution, and add new features and improvements incrementally. This approach implies that existing code does gets modified over and over again. The effort to groom a highly redundant code base eats at the team’s improvements.

Diagram: Time to effort, impeded by duplication drag

The team may even become worse, instead of better, at reacting to change. Being able to react to change is a corner stone of any agile approach. Coding in an agile environment must aim at producing code that is adaptive to change, even if that means a somewhat higher initial effort. Technical debt does not only pile up with oversimplified architecture, poor coding and sloppy testing – it takes home inside the test suite too.

To cut the long and winding story short: Setup code for complex tests should really, really exist just once.

Builders to the rescue

As a technical means of improving things, implementing the Builder pattern is a good way to avoid test setup code duplication. The idea is to have a class that can be configured with all the property values and behaviour that the test needs, and then create the test subject class:

public void Sort_by_two_letters()
    // Arrange
    var subject = new ControllerBuilder()
        .WithNames("Alice", "Amy", "Alfred")

    // Act

    // Assert
                 .BeEquivalentTo(new[] { "Alice", "Alfred" });

With the help of the Builder, the test has stopped referencing any implementation detail of the Controller class. When implementation details of the Controller class change, the Builder needs to be adapted, but that is about it. Here is the code of the Builder used in this test.

There are a lot of variations to this theme:


Duplication of setup code makes small changes in implementation details cause large-scale changes in test code. Here, adhering to the principle of DRY (Don’t Repeat Yourself) pays off generously.

The Builder Pattern and its variations can help keeping setup code duplication in check.

1 Tools used: the unit test framework xUnit.net, the mocking framework Moq, and FluentAssertions for asserting the elements in the array in an unspecific order.
2Stubs vs. Mocks and other test doubles: Mocks Aren’t Stubs
3This is all .NET tooling. That is simply because I’m mostly a .NET guy. If you have something that provides these things in your world, dropping a comment would be extremely warmly welcome.

Comments (3)


Sign up for our Updates

Sign up now for our updates.

This field is required
This field is required
This field is required

I'm interested in:

Select at least one category
You were signed up successfully.

Receive regular updates from our blog


Or would you like to discuss a potential project with us? Contact us »