The Problem with Tests

Ok so that got your attention, from someone who has spent the last few years jumping up and down about unit testing, now he has problems with it!

Actually no, its not the tests, it’s the what.   Going back over some old code the other day, and by old I mean about three months old, I had a problem, after some fairly heavy refactoring which involved combining one repository class with another, so that the repository link-to-SQL matched the database, where as the repository API matched what the business layer wanted, I noticed some push back from the tests.

The refactoring should have broken a single group of tests, those that relied on the original repository methods, but instead may tests across the application were failing, what made it a bit worse was the fact that the mass of tests made it difficult to actually predict which tests should fail because of the changes made, an important distinction, when refactoring you need to know what you expect to happen, against those cases where you have actually broken something without realising.

So what was the problem.  I have lots of tests, (a good thing), virtually all the code is covered, if I were to run a code coverage tool across the code, then the result would be in the high 80%-90% range, (you can very rarely reach 100%), but, some of these tests are fragile, which means that the test is tightly bound to the structure of the code, also some of my tests actually cover the same code many time, and these tests maybe in different test fixtures, which also added to the confusion.

So what has happened? Well all the code written in this application was test driven using the Behaviour Driven Development style of testing, and I made the error of treating my tests as inferior to the production code, I refactored the production code,  but didn’t treat the tests in the same way.  To be totally honest I am not sure that I refactored the production code as well as it could have been, which could also lead to the problems I was having.

So what have I learnt, when doing test driven development, one of the most important steps, which is actually very easy to miss is refactoring, you write a failing test, then make it pass, the temptation is to then write the next test, I find, by this point, the next test is in my head, I know where to go next! Its so important to stop at this point, write that test down on paper if you have to just so you don’t forget it, don’t write it in the code, because you will be tempted to then make it pass, as you should  never refactor on a red line!, which will start the whole process off again, just stop. Look at your production code, and refactor it, run the tests again, then go and look at your test code, all your test code, and refactor that, make sure that each test is still valid, that you are not covering the same area twice, don’t be afraid to delete, combine, or change tests, as your code develops, and changes, so should the tests, they are not immutable, they are the wrapping around your production code, and like a wrapping paper around a parcel, the shape should reflect the shape of the code they are covering.

This is another thing those silly katas can help with, just the simple muscle memory of refactoring after each test, both in the production code and in the test code can be practiced, and the action of red, green refactor will become second nature, which will help the tests to stay fresh and meaningful.


About Duncan Butler

Trying to be a very agile software developer, working in C# with Specflow, Nunit and Machine Specifications, and in the evening having fun with Ruby and Rails
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s