You can tell weather or not someone really “gets” unit testing by asking them one simple question, “Do you use mock objects?” Almost invariably, they will say “no”. Even people who have totally gotten the testing religion. It’s like watching someone pray to a statue of Jesus; totally oblivious to the fact that Jesus himself is standing four feet away reading a book.

This is partially due to the fact that most geeks don’t actually know what a unit test is. They think that testing the methods of a specific class constitutes a unit test, but that’s only part of the story. A unit test test is when you test the methods of a specific class in isolation, and the difference is critical. You know how some people call us “computer scientists”. Yeah, well this is the science part.

Imagine you were creating a new medicine, and wanted to test it out on some cells in a petri dish. The weather was so nice you decided to take your dish outside and enjoy the sunlight while you ran your test. It’s a resounding success! Or… is it? How do you know it’s wasn’t just the UV rays from the sunlight? Or maybe there was something floating in from the nearby blossoms? You don’t, and you can’t. You can’t even say, with any confidence, that the medicine worked. Something worked. Maybe the medicine had a part in it, maybe it didn’t. You can’t tell.

The same thing applies to unit testing. If you’re not testing in isolation there is no way to know that the thing your are testing was actually effective or if it just happened to work in the current environment, and when the test fails there’s no way to know (without digging) if your code was responsible, or if something it relies on has broken and passed on its problems. Mock objects are the best tool we have for addressing this problem.

A Mock What?

You probably already know, but just in case, mock objects…

“…are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.” - Wikipedia

In practice it is a functional guarantee that a foreign object/method/API, will behave in a specific way regardless of its implementation. You figure out exactly what a call should return and then you encode that in your mock.

Why?

Let’s look at an example. In a banking application a User would have one or more Accounts, and it’s not uncommon for the User and Account objects to each have methods that reference the other. Maybe, the User object has an available_funds() method that adds up all the available funds in each of the Accounts. User is depending on each Account object to correctly calculate the current balance, minus any outstanding charges. Problems arise when there’s a bug in how Account is doing its calculation. Now, the test of your User’s available_funds() method is failing, but you can’t tell if it’s because it was performing a bad calculation or Account was. Even worse, it could be that there’s bad data in the database, or that someone changed your test data but didn’t update the tests. All you know is that the number expected isn’t the one received.

The problem with this setup is that you’re no longer testing your User object when you do this. You’re testing that the code in your User class works if the code in the Account class also works. The typical solution of carefully chosen test data in a database does not solve this problem in any way. The only thing it does is add in another source of potential failures and misdirections, and provides yet another thing to maintain.

Using mock objects allows you to test that the User class works regardless of how broken the Account class may or may not be, with no database installed anywhere, and no separate fake data to maintain. You create a real User object and mock up the calls to Account to return mocked account objects, mocked data, whatever else is appropriate.

If you’re not doing this, you are not unit testing. That’s not meant to deride what you’re doing. I mean that as a literal statement of fact. Anyone, for example, who is using the built in Rails “unit” testing framework with fixtures (or FactoryGirl fixtures) is guilty of this. No test that relies on a separate class, or API, or interfaces with a database in any way is a unit test. It is an integration test (some people call them functional test), and when something goes wrong it is just a matter of time before it misleads you.

Now, integration tests are great. They serve the very useful function of guaranteeing that your code works within the context of the expected ecosystem, but they’re not very good about telling you exactly what has broken.

People will disagree with me on this. They’ll argue that testing with the db and other classes isn’t bad, that they can generally tell what’s broken very quickly. And it’s true that a developer with a good knowledge of the system should be able to figure out what’s causing a test to fail pretty quickly, but until they start digging there’s no way to be 100% confident that the bug is where the unit test claims it is because of all the other things that could affect it.

But that’s not all

The process of creating mocks to support your tests will very frequently reveal unexpectedly tight coupling’s between classes and systems. Because you need to mock up every interaction with a foreign anything, you end up creating a bunch of additional lines in your test. Each one is an expenditure of effort and each one represents a coupling. If the object you were testing wasn’t coupled to anything else you wouldn’t need to mock anything. Adding a few lines of mocking to your test is no big deal, but when you have 20+ lines of mocking in a single test it’s a strong indication that something is wrong in your system. If you find you can’t figure out how to mock something up it either means that you don’t really understand what the thing you need to mock does, or that it suffers from a notably poor design.

Mocking should always be easy, and quick. The only times it isn’t are when you’ve got a poor design, or an insufficient grasp of what you’re attempting to test. These are both very good things to have revealed.

Sometimes, the mocks will reveal bad designs, or excessive coupling’s in an library you didn’t even write. Part of the reason I started writing this article was because of just that type of situation. I was trying to unit test a Rails controller. Typically people just don’t unit test rails controllers. They do functional tests that tie in the controller methods, the database, and the view generation, which again, is useful, but doesn’t do a great job of telling you exactly what is broken when the test fails until you start digging. Anyway, the methods I was testing required an the user viewing them to be logged in, but try as I might I could not find a way to successfully mock it so that it never touched the database.

I assumed that I simply didn’t understand the library, and began looking and source code and googling around. After hours of trying, and failing, to make it work I eventually started Googling from a different angle and discovered that the problem was that it simply wasn’t possible (by any reasonably definition of the word) to test this library with mock objects, even leveraging its test helper classes. The thing was so tightly coupled to the underlying database that it demanded the presence of a test database with appropriately structured fake data to even create a test login.

I had to make the choice of either giving up on any hope of decoupling my tests from the database or throwing out a working authentication system. I care about my users, and I care about being able to guarantee that my sites work for them, so I spent two evenings throwing out that system, learning and implementing a new one, figuring out how to test it without touching the database, and then making all my controller tests work again.

It was worth every minute.

Reasons not to use mocks

The downside to mocking everything is the inverse of not mocking anything. If you don’t do pure unit testing, and don’t mock anything you can never be 100% confident that your objects work in isolation. If you don’t do integration testing and do mock everything you can’t be 100% confident that your objects work in concert with each other.

A combination of unit tests and integration tests is always best. There’s no reason not to build both. It just makes your system more robust and catches more bugs before they are released. But, if I had to choose one only one method, I would choose pure unit tests with mock objects for two reasons:

1. The objects you test in unit tests constitute the foundation, the building blocks, of your system. If they have problems then everything has problems. I want the maximum possible confidence that each of those building blocks is reliable and well crafted.
2. The process of mocking up the interactions with other objects gives me a far better understanding of how my system works and frequently reveals design flaws in my code, and the libraries I’ve chosen to use. Exposing and correcting those flaws helps improve the maintainability of my code.

I realize this is not the popular choice, and many developers, would argue that if you’re going to write only one test it should be an integration test. There are good arguments for that, but there are no good arguments for doing either without the other.

A moment for FactoryGirl

This one is specifically for the Ruby geeks, but I’m sure there are similar tools for other languages. FactoryGirl is a system that allows you to easily create named fixtures with the data you need.

“When you invoke a factory, factory_girl uses your definitions to compile a list of attributes that should be assigned to that instance, as well as any associated factories… To create an instance, it calls new without any arguments, assigns each attribute (including associations), and then calls save!”

In other words, it’s a nice wrapper around what is an inherently bad idea for unit tests. In fact, using it guarantees that you are not writing unit tests.

From what I’ve seen FactoryGirl is almost always used as a bad stand-in for mock objects. It is a good solution for your integration tests, where you actually want to test that items can be sent to and retrieved from your database from the other parts of your system, but it should never be called from a unit test because that is precisely what you don’t want happening in that situation.

Chris Parsons has a good writeup with plenty of code examples about FactoryGirl vs mocking that you should check out.

The Takeaway

There is immense value to testing in isolation and the process of creating mock objects to support that testing. If you’re not testing in isolation, you’re not writing unit tests.

If you haven’t been using mocks, take the tiny amount of time it takes to learn how one of the popular mock object frameworks for your language, and start updating your unit tests to be true unit tests. Make sure that every new one is a true unit test, and every time you have to touch an old one take a moment to replace the fixtures with mocks. It will be worth it in the long run.

Resources for Ruby Geeks

In the Ruby world we have a good number of mock object frameworks.

• Mocha is probably the best known, and from what I’ve seen most widely used.
• Flexmock is another option, but I have essentially zero knowledge about it.
• RSpec has a built in mocking library called RSpec-Mocks. I have spoken with a number of RSpec users who prefer to use other mock libraries like Mocha instead of RSpec-Mocks. I’ve never gotten the RSpec religion so I can’t comment one way or the other.
• RR (Double Ruby) is a lesser-known option. The syntax is very nice (not nearly as verbose) and it offers some nice features like Proxies and Spies that other frameworks don’t.

Personally I’m a fan of Mocha and RR.