Testing as a process of discovery

The other day a coworker said,

Some times you get situations where the specification for the unit or module you are writing just are not available. The code writing is a discovery process as much as anything else. Moreover, some of the packages and methods being called don’t have predictable or documented behavior. That’s ugly and horrible, and I don’t know how that’s allowed, but nonetheless, from the perspective of someone who wants to do unit testing in such an environment, can you give any tips? I mean, do you mock up approximations to what you think these external things should be doing if you really don’t know what they are doing? Do you do your best, updating mocks and tests, “in the face of adversity”?

Really we’ve got two questions here: How do you address the desire to write tests when you don’t know what exactly you need / want yet?

How do you handle external libraries where you’re just not confident in what the expected behavior is?

How do you write tests for something when you don’t know what

exactly that something is yet?

Let’s look at coding a simple User Story and Acceptance Test and tackle them in a way that allows for maximum flexibility and exploration while coding.

The User Story (for a cash register): Keep a running receipt with a short description of each item and its price.

Its Acceptance Test: Setup: The cashier has a new customer. Operation: The cashier scans a toothbrush for $2.25. Verify: The receipt has a description of the toothbrush and its price.

Let’s assume we’ve written absolutely no code. But we want to take a test driven approach and we don’t want to write tests for the bits we haven’t really figured out yet, because, well… we haven’t figured them out and we want to keep the possibilities open. So let’s write the bare minimum required to test this. From the user story and acceptance test above we can tell exactly what classes and methods we’ll need to write and test. So we’ll need:

1. ICashier (cashier interface)
2. ICustomer (customer interface)
3. IRegister (register interface)
4. IToothbrush (toothbrush interface)
5. IReceipt (receipt interface)

As for methods we’re going to need:

IToothbrush

-   getPrice()
-   getShortDescription()

IRegister

-   scanItem()

IReceipt

-   addItem()
-   print()

Now, there are three approaches we could take here:

1. Write a full interface and tests for all of these with all the other methods we expect them to need (like a getUPC() method on IToothbrush)
2. Write an interface and tests for all the methods we know we’ll need (the ones listed above)
3. Write an interface and tests just for the items we are using RIGHT NOW.

I’d been doing the latter for a little while and was pleased as punch to find out, when listening to a podcast interview with Martin Fowler, that this is exactly what he’s been doing lately too. I think that this is the best possible approach because it keeps you focused on what you actually need, it gives you much smaller and more quickly obtainable goals, and it gets things working faster. If you took this approach you might proceed something like this (again, assuming no code has been written at this point):

Write the IToothbrush interface.

Add the method you want to address first (get price)

Write a unit test for that method.

• check that a toothbrush never has a null price
• assert that the price is going to be a decimal number
• check that the price is never less than zero.

Now that you’ve got a properly failing test you write a Toothbrush implementation with just a getPrice() method.

In your Toothbrush implementation you’d probably have a no parameter constructor that set up a default price of $0.00, or your getPrice() method would be smart enough to guarantee a non-null price was returned, either way, your test would pass.

I’d say just repeat for each item in the list but it’s not quite that simple. When you get to the getShortDescription() you’ll probably have a test that it’s not null, but we’ve got no method in our spec for actually setting a description so we can’t make an implementation where that test passes unless we create a method to set the description or allow it to be set in the constructor. XP people advocate doing the simplest possible thing that will work, which in this case makes sense and would be to set it in a constructor. So, now we’ve got to add a unit test for the constructor, and it’s reasonable to think that we should set the price in the constructor too. So, in order to really test the getShortDescription() method we also need to write a test for the Toothbrush constructor. So, testing getShortDescription() leads us on to the next bits we need RIGHT NOW.

Add the getShortDescription() method to the IToothbrush interface

Add the testGetShortDescription() unit test

check that the description isn’t null

check that the description isn’t empty

check that it’s actually “short”

• this could, conceivably lead you to want create a generic Item interface that has a validateShortDescription(...) and / or truncateDescription(...) method, but I’d recommend holding off until you finish with the items required for this acceptance test. While a good idea, and something to plan on, it’ll be nicer to actually get through this initial acceptance test before diving down the rabbit hole of nice-to-haves.

Add testConstructor() method to your ToothbrushTest test case

• test that we can instantiate an object with a valid price and short description
• test that we can’t instantiate an object with an invalid price and valid short description
• test that we can’t instantiate an object with a valid price and invalid short description
• test that we can’t instantiate an object with an invalid price and short description
• test that the price and short description come out through the two getters we have in our interface.

Add a constructor that takes a price and a short description to Toothbrush

Add getShortDescription()`` to toothbrush``.

By taking such a focused approach we haven’t constrained what could happen with any of the other functionality in toothbrush. Let the emerging specification define if you need a getColor() method on your toothbrush. If it’s not there now and none of the methods you’ve worked on so far have implied the need don’t go there at all… otherwise when you finally do get there you’re likely to find that “blue” wasn’t an acceptable thing to for it to return at all what it really needed to return was an array of RGB values: [0,0,255]. My advice would be to add a comment to the interface noting that you think a getColor() method might be useful and what you think it should return, that way as you keep coming back and updating IToothbrush you’ll keep that in mind and won’t forget or accidentally code yourself into a situation where it’s difficult to derive the color.

How do you handle external libraries where you’re just not confident

in what the expected behavior is?

The same approach we just discussed for writing the unit test should guide what we mock up of untrusted external libraries AND how we proceed in working with them. Normally I try and work under the assumption that the libraries I depend on are dependable and never write unit tests for them, but sometimes you need to find out what you can depend on or what exactly some methods do because they’ve got no docs. So first you look at the library and ask yourself what methods, based on your limited understanding, do you absolutely need to use? Then you write unit tests that test that confirm that when using it the way you plan on using it it does what you think it would do. You don’t have time to test their entire library or all the possible edge cases for their code so you just test that the bits you will be using work the way you think they do when called in the way you’ll be calling them. For example if your code can’t possibly pass in a negative number then you don’t need to worry if their classes correctly handle negative numbers or not. You just need to make sure that the class handles the full range of numbers you could be passing it correctly and nothing else. Now you know that it does (or doesn’t do) what you expected and you know some things that you can safely pass it.

Depending on how untrusted the library is you may also want to make sure that everyone in your group knew to not use anything in that library that they didn’t have a minimal test case for.

Mike Clark wrote an article wherein he describes a similar approach but for a language instead of a library. His idea was that when you start learning a new programming language you just don’t know how exactly things work. You assume things may work one way but that’s not always the case. He set out to write test cases for all the things he learned how to do in the language. This had a few good side effects: it got him familiar with the testing framework of the language, it got him to actually use all the language constructs (doing is far better for memory retention than just reading) and he now has a test suite of his mental beliefs of how the language works. If a new version of the language comes out he’ll know right away if his mental model of how things works needs to change because his tests will fail.