Fakes and Mocks and Stubs, Oh My!!!


So we covered how to use EasyMock to write tests in one of the previous posts, but Mocking is not the only option when you want to test something that depends on a slow and expensive service. Mocking allows you to expect calls and return values when a call is made, and make sure the number of calls were correct and the order, but sometimes, you want a bit more, or not even that much. So in those cases, you have the options of using Stubs or Fakes. So without further delay, your options are :

  1. Mocks :
    Well, Mocks… What can I say. I have already ranted and raved about the awesomeness that is Mocks. Mocks and Mocking frameworks allow you to replace heavy and expensive services with Mocks, with which you can set expectations on what calls are made to the service, and return values. This gives you the advantage of knowing exactly what is called and returned, and makes your test deterministic and super fast. You can even set expectations on the number of calls and the order, though this is something that is not extremely recommended, since that makes the test dependent on the implementation, which is never a good thing.

    Advantages :

    • Fast, Reliable
    • Deterministic
    • Lightweight
    • Control over expectations and return values.

    Disadavantages :

    • Can become dependent on implementation of method being tested.
    • Can become a mockery if not careful, that is, can be testing interaction between mocks and nothing of the actual code. Especially when all that the method under test is doing is delegating to a bunch of service calls.
    • Can involve complicated setup for expectations, especially with unwieldy objects
  2. Stubs :
    Now, whereas with Mocks, you can specify what calls will be received and what will be returned making them somewhat intelligent, Stubs are at the other end of the spectrum. The dumbest of the dumb, the easiest of the easiest, these just stub out the methods of your expensive and heavy class. So a null Stub could return null for each method, or just return some constant value each time it is called. That means that regardless of what the method is called with, it returns the exact same thing, day in and day out. It doesn’t adjust, it doesn’t react, just returns. Of course, this means that you might not be able to test all the cases you want to inspect / test. But its simple, its easy, and of course, its fast.

    Advantages :

    • Fast, reliable
    • Simple
    • Consistent

    Disadvantages :

    • Dumb
    • Can’t exercise different cases without different stubs
    • Returns only one value consistently
  3. Fakes :
    And finally, we come to the smartest (figuratively) of the lot. The Fakes. For once, being a fake can be good. While Stubs return the same thing again and again, and mocks return what you tell it to, Fakes are smarter. The easiest way to explain a Fake is with an example. Say your code depends on a heavy service like a Database. A FakeDatabase would be an in-memory implementation of the DB, thus making it faster while at the same time, providing the same logic as the normal DB would. There are different types of Fakes, like a listening / recording Fake which records all the values passed to it.

    Advantages :

    • Can test while preserving behavior of dependencies
    • Faster than using actual services
    • Can test complicated logic
    • Most comprehensive testing approach

    Disadvantages :

    • Can be complicated to set up
    • Not as fast as mocks / stubs
    • Not as easy to define expectations
    • When / Where do you test the fake (especially the complicated ones) ?

Thats my 2 cents on the different approaches to testing with dependencies. Now that we know what we can do with those darn heavy dependencies, I’ll talk more on how to make sure you can use these different approaches to test your code soon.

, , , , ,

  1. #1 by closingbraces on August 21st, 2008

    Thanks for the good article – a very clear summary of the differences.

    My own take on “fakes” is that it can indeed be a lot of work to create really good fakes (including testing them, as you point out). But for widely used API or services I think it can make sense for someone to do this centrally, and produce a library of “ideal” fakes for that API. Everyone else can then just use the pre-written, pre-tested fakes.

    Ideally I’d like to see this as a responsibility of API design and implementation. If an API actually needs test-doubles or a fake service in order to be easily testable, the API ought to supply this. And this ought to be taken into account in the API design (e.g. avoid needing it if possible, or make it as straightforward as possible). Well, we can all dream…

    In terms of speed, if a fake is just doing normal in-memory stuff, it might involve slightly more than a mock but this shouldn’t normally be so much as to impact performance. You can churn through an awful lot of “plain” code before noticing any impact, and the code that you’re testing probably does as much or more than the fake. At least that’s been my experience.

    I’d say defining expectations is different for a fake, rather than harder, with a lot depending on the precise nature of the fake. If the fake lets you configure and inspect everything relevant to the object’s behaviour, and can record the API calls too, then you’ve got eveything you need. I don’t think there has to be anything making it intrinsically harder than strictly necessary.

    So on the whole I tend to regard “fakes” as the ideal – if you can get someone to develop them for you!

    At any rate, that’s my personal view on fakes, based on writing a set for the Servlet API.

    Mike

  2. #2 by shyamseshadri on August 22nd, 2008

    I agree with you on the fact that Fakes are the most ideal of the lot. It gives you the closest test to an integration test while still being an unit test. And yeah, Fakes are generally fast, but compared to Mocks and stubs, not so much. But I’ll play devil’s advocate here.

    Fakes can (can, and not necessarily must) be hard to setup to work similar to an existing service. Also, to test the interesting cases with a Fake, you might need to get the, say FakeDB, into a particular state before you can do your actual test. These might be like adding specific rows, and getting the DB to a particular state. Whereas with a mock, you can just tell it to expect a call and return whatever you need it to. (Stubs of course, can never do this…). Also might be tough to use a Fake Http service and stuff (not impossible, but harder than it needs to be).

    In the end, I feel it comes down to using what is most ideal for the situation. Use Fakes whenever you can, and whenever time and complexity permits, but use Mocks to get things going quickly and if you want to test edge cases easily and use stubs if you need to just get through the offending material.

  3. #3 by closingbraces on August 22nd, 2008

    Shyam,

    I see what you mean about the possible work involved in configuring a fake DB, as opposed to mocking it. Guess a lot depends on the nature of what’s being faked, as well as what facilities the fake provides and how well it’s been done.

    Absolutely agree about using whatever’s best for each situation. I see far too much clutching at names and silver bullets these days, rather than thinking about what precisely is needed in each specific situation. Probably guilty of that myself at times, but do try to avoid it.

  4. #4 by Gabriel on August 17th, 2009

    Hi,

    I’ve been getting in to unit testing these past months and have been browsing around for any tips, best practices and so on I can read. I found this post today and in the disadvantages about Mocks you mentioned something that has been bothering me these past few days. I feel that some of the tests I wrote with mocks are mostly testing the interaction between the mocks and depend too much on the implementation! Do you have any advice on how can this be avoided? Some methods are just delegating to other collaborators and I’m not sure how to test them properly.

    Thanks in advance. :)

  5. #5 by Shyam on August 18th, 2009

    @Gabriel

    So this is something I run into quite often, and in my experience, the classes which just delegate to other services aren’t really worth testing that well. At best, test that it passes down the right parameters, and assume that the service you are delegating to is well tested and does what it should. If that service is one of your own, test that thoroughly instead. And most of all, have some integration tests which test the entire thing.

    Like if you are delegating to something that talks to a database, maybe have a test which has an actual (or fake db) at the back, and tests that this delegates properly and returns the right thing. But on the unit test level, have maybe one test per delegation, and leave it at that.

    Hope that helps.

(will not be published)

  1. No trackbacks yet.