Posts Tagged testing

Super fast JS Testing

Before I jump into how exactly you can perform super fast and easy JS testing, let me give you some background on the problem.

Javascript is a finicky language (Some people even hesitate to call it a language). And it can easily grow and become a horrible and complicated beast, incapable of being tamed once let loose. And testing it is a nightmare. Once you have decided on a framework (of which there are a dime a dozen), you then have to set it up to run just right. You need to set it up to actually run your tests. Then you have to figure out how to run it in a continuous integration environment. Maybe even run it in headless mode. And everyone solves it in their own ways.

But the biggest problem I have with most of these frameworks is that executing the tests usually requires a context switch. By that, I mean to run a JSUnit test, you end up usually having to open the browser, browse to a particular url or html page which then runs the test. Then you have to look at the results there, and then come back to your editor to either proceed further or fix your tests. Works, but really slows down development.

In java, all it takes is to click the run button in your IDE to run your tests. You get instant feedback, a green / red bar and details on which tests passed and failed and at what line. No context switch, you can get it to run at every save, and proceed on your merry way. Till now, this was not possible with Javascript.

But now, we have JS Test Driver. My colleagues Jeremie and Misko ended up running into some of the issues I outlined above, and decided that going along with the flow was simply unacceptable. So they created a JS Testing framework which solves these very things. You can capture any browser on any machine, and when you tell it to run tests, it will go ahead and execute them on all these browsers and return you the results in your client. And its blazing fast. I am talking milliseconds to run 100 odd tests. And you can tell it to rerun your tests at each save. All within the comforts of your IDE. And over the last three weeks, I have been working on the eclipse plugin for JS Test Driver, and its now at the point where its in a decent working condition.

The plugin in action

The plugin in action

The plugin allows you to, from within Eclipse, start the JS Test Driver server, capture some browsers, and then run your tests. You get pretty icons telling you what browsers were captured, the state of the server, the state of the tests. It allows you to filter and show only failures, rerun your last launch configuration, even setup the paths to your browsers so you can launch it all from the safety of eclipse. And as you can see, its super fast. Some 100 odd tests in less than 10 ms. If thats not fast, I don’t know what is.

For more details on JS Test Driver, visit its Google Code website and see how you can use it in your next project and even integrate it into a continuous integration. Misko talks a little bit more about the motivations behind writing it on his Yet Another JS Testing Framework post. To try out the plugin for yourselves, go add the following update site to eclipse : http://js-test-driver.googlecode.com/svn/update/. For all you IntelliJ fanatics, there is something similar in the works.

, , , , ,

7 Comments

The ROI of Testing

Nowadays, when I talk with (read: rant at) anyone about why they should do test driven development or write unit tests, my spiel has gotten extremely similar and redundant to the point that I don’t have to think about it anymore. But even when I do pairing with skeptics, even as I cajole and coax testable code or some specific refactorings out of them, I wonder, why is it that I have to convince you of the worth of testing ? Shouldn’t it be obvious ?

And sadly, it isn’t. Not to many people. To many people, I come advocating the rise of the devil itself. To others, it is this redundant, totally useless thing that is covered by the manual testers anyway. The general opinion seems to be, “I’m a software engineer. It is my job to write software. Nowhere in the job description does it say that I have to write these unit tests.” Well, to be fair, I haven’t heard that too many times, but they might as well be thinking it, given their investment in writing unit tests. And last time I checked, an engineer’s role is to deliver a working software. How do you even prove that your software works without having some unit tests to back you up ? Do you pull it up and go through it step by step, and start cursing when it breaks ? Because without unit tests, the odds are that it will.

But writing unit tests as you develop isn’t just to prove that your code works (though that is a great portion of it). There are so many more benefits to writing unit tests. Lets talk in depth about a few of these below.

Instantaneous Gratification

The biggest and most obvious reason for writing unit tests (either as you go along, or before you even write code) is instantaneous gratification. When I write code (write, not spike. That is a whole different ball game that I won’t get into now), I love to know that it works and does what it should do. If you are writing a smaller component of a bigger app (especially one that isn’t complete yet), how are you even supposed to know if what you just painstakingly wrote even works or not ? Even the best engineers make mistakes.

Whereas with unit tests, I can write my code. Then just hit my shortcut keys to run my tests, and voila, within a second or two, I have the results, telling me that everything passed (in the ideal case) or what failed and at which line, so I know exactly what I need to work on. It just gives you a safety net to fall back on, so you don’t have to remember all the ways it is supposed to work in. Something tells you if it is or not.

Also, doing Test Driven Development when developing is one of the best ways to keep track of what you are working on. I have times when I am churning out code and tests, one after the other, before I need to take a break. The concept of TDD is that I write a failing test, and then I write just enough code to pass that test. So when I take a break, I make it a point to leave at a failing test, so that when I come back, I can jump right back into writing the code to get it to pass. I don’t have to spend 15 – 20 minutes reading through the code to figure out where I left off. My asserts usually tell me exactly what I need to do.

Imposing Modularity / Reusability

The very first rule of reusable code is that you have to be able to instantiate an instance of the class before you can use it. And guess what ? With unit tests, you almost always have to instantiate an instance of the class under test. Therefore, writing a unit test is always a first great step in making code reusable. And the minute you start writing unit tests, most likely, you will start running into the common pain points of not having injectable dependencies (Unless of course, you are one of the converts, in which case, good for you!).

Which brings me to the next point. Once you start having to jump through fiery hoops to set up your class just right to test it, you will start to realize when a class is getting bloated, or when a certain component belongs in its own class. For instance, why test the House when what you really want to test is the Kitchen it contains. So if the Kitchen class was initially part of the House, when you start writing unit tests, it becomes obvious enough that it belongs separately. Before long, you have modular classes which are small and self contained and can be tested independently without effort. And it definitely helps keep the code base cleaner and more comprehensible.

Refactoring Safety Net

Any project, no matter what you do, usually ends up at a juncture where the requirements change on you. And you are left with the option of refactoring your codebase to add / change it, or rewrite from scratch. One, never rewrite from scratch, always refactor. Its always faster when you refactor, no matter what you may think. Two, what do you do when you have to refactor and you don’t have unit tests ? How do you know you haven’t horribly broken something in that refactor ? Granted, IDE’s such as Eclipse and IntelliJ have made refactoring much more convenient, but adding new functionality or editing existing features is never simple.

More often than not, we end up changing some undocumented way the existing code behaved, and blow up 10 different things (it takes skill to blow up more, believe me, I have tried). And its often something as simple as changing the way a variable is set or unset. In those cases, having unittests (remember those things you were supposed to have written?) to confirm that your refactoring broke nothing is godsend. I can’t tell you the amount of times I have had to refactor a legacy code base without this safety net. The only way to ensure I did it correct was to write these large integration tests (because again, no unit tests usually tends to increase the coupling and reduce modularity, even in the most well designed code bases) which verified things at a higher level and pray fervently that I broke nothing. Then I would spend a few minutes bringing up the app everytime, and clicking on random things to make sure nothing blew up. A complete waste of my time when I could have known the same thing by just running my unit tests.

Documentation

Finally, one of my favorite advantages to doing TDD or writing unit tests as I code. I have a short memory for code I have written. I could look back at the code I wrote two days ago, and have no clue what I was thinking. In those cases, all I have to do is go look at the test for a particular method, and that almost always will tell me what that method takes in as parameters, and what all it should be doing. A well constructed set of tests tell you about valid and invalid inputs, state that it should modify and output that it may return.

Now this is useful for people like me with short memory spans. But it is also useful, say, when you have a new person joining the team. We had this cushion the last time someone joined our team for a short period of time, and when we asked him to add a particular check to a method, we just pointed him to the tests for that method, which basically told him what the method does. He was able to understand the requirements, and go ahead and add the check with minimal handholding. And the tests give a safety net so he doesn’t break anything else while he was at it.

Also useful is the fact that later, when someone comes marching through your door, demanding you fix this bug, you can always make sure whether it was a a bug (in which case, you are obviously missing a test case) or if it was a feature that they have now changed the requirements on (in which case you already have a test which proves it was your intent to do it, and thus not a bug).

, , , , ,

4 Comments

Separation anxiety ?

We all have separation anxiety. Its a human tendency. We love inertia, and we don’t like change. But why should your code have separation anxiety ? Its not human (even though it might try and grow a brain of its own at times)!

I bring this up because I got the chance to work with someone who had some questions on how to test UI code. Fairly innocuous question, and in most cases, my response would have been, keep the UI code simple and free of any logic, and don’t worry too much about it. Or you could write some nice end to end / integration tests / browser based tests. So with that response set in mind, I set off into the unknown. Little was I to know was that was the least of my concerns. As I sat down to look at the code, I saw that there were already tests for the code. I was optimistic now, I mean, how bad could things be if there are already tests for it ?

Well, I should remember next time to actually look at the tests first. But anyways, there were tests, so I was curious what kinds of tests they wanted to write and what kind of help I could provide, if any. So it turns out, the class was some sort of GUI Component, which basically had some fields, and depending on whether they were set of not, displayed them as widgets inside of it. So there were, say, 5 fields, and differing combinations of what was set would produce different output. The nice thing was that the rendered data was returned as a nice Java object, so you could easily assert on what was set, and get a handle on the widgets inside of it, etc. So the method was something along the following lines :

public RenderedWidgetGroup render() {
    RenderedWidgetGroup group =
        createWidgetGroup(this.componentId,
                          this.parent);
    if (this.name = null) {
        return group;
    } 
    group.addWidget(new TextWidget(this.name));
    group.addWidget(
        new DateWidget(
            this.updatedTimestamp == null ?
                 this.createdTimestamp : this.updatedTimestamp));
    return group;
}

So far, so good, right ? Nice, clean, should be easy to test if we can set up this component with the right fields. After that, it should just be a few tests based on the different code paths defined by the conditionals. Yeah, thats what I thought too. So, me, being the naive guy that I was, said, yeah, that  looks nice, should be easy to test. I don’t see the problem.

Well, then I was taken to the tests. The first thing I see is a huge test. Or atleast thats what I think it is. Then I read it more closely, and see its a private method. It seems to take in a bunch of fields (Fields with names that I distinctly remembered from just a while ago) and churn out a POJO (Plain Old Java Object). Except this POJO was not the GUI Component object I expected. So obviously, I was curious (and my testing sensors were starting to tingle).  So I followed through to where it was called (which wasn’t very hard, it was a private method) and lo and behold, my worst fears confirmed.

The POJO object generated by the private method was passed in to the constructor of the GUI component, which (on following it further down the rabbit hole) in its constructor did something like the following :

public MyGUIComponent(ComponentId id,
                      Component parent,
                      MyDataContainingPOJO pojo) {
    super(id, parent);
    readData(pojo); 
} 

And of course, readData, as you would expect, is a :

  • Private method
  • Looks through the POJO
  • If it finds a field which is not null then it sets it in the Gui Component

And of course, without writing the exact opposite of this method in the unit test, it just wasn’t possible to write unit tests. And sudddenly, it became clear why they were having trouble unit testing their Gui Components. Because if they wanted to test render (Which was really their aim), they would have to set up this POJO with the correct fields (which of course, to make things more interesting / miserable, had sub POJOs with sub POJOs of their own. Yay!) to be exercised in their test.

The problem with this approach is two fold :

  1. I need tests to ensure that the parsing and reading from the POJO logic is sound, and that it correctly sets up the GUI Component.
  2. Every time I want to test the render logic, I end up testing (unintentionally, and definitely unwantedly) the parsing logic.

This also adds, as I saw, obviously complicated pre test setup logic which should not be required to test something completely different. This is a HUGE code smell. Unit testing a class should not be an arduous, painful task. It should be easy as setting up a POJO and testing a method. The minute you have to perform complicated setup to Unit test a class (Note, the keyword is unit test. You can have integration tests which need some non trivial setup.), stop! There is something wrong.

The problem here is that there is a mixing of concerns in the MyGuiComponent class. As it turns out, MyGuiComponent breaks a few fundamental rules of testability. One, it does work in the constructor. Two, it violates the law of demeter, which states that the class should ask for what it needs, not what it doesn’t need to get what it needs (Does that even make sense ?). Thirdly, it is mixing concerns. That is, it does too much. It knows both how to create itself as well as do the rendering logic. Let me break this down further :

Work in the constructor

If you scroll back up and look at the constructor for MyGuiComponent, you will see it calling readData(pojo). Now, the basic concept to test any class is that you have to create an instance of the class under test (unless it has static methods. Don’t get me started on that…). So now, every time you create an instance of MyGuiComponent, guess what ? readData(pojo) is going to get called. Every. Single. Time ! And it cannot be mocked out. Its a private method. And god forbid if you really didn’t care about the pojo and passed in a null. Guess what ? It most probably will blow up with a NullPointerException. So suddenly, that innocuous method in the constructor comes back to haunt you when you write yours tests (You are, aren’t you ?).

Law of Demeter

Furthermore, if you look at what readData(pojo) does, you would be even more concerned. At its base, MyGuiComponent only cares about 6 fields which it needs to render. Depending on the state of each of these fields, it adds widget. So why does the constructor ask for something totally unrelated ? This is a fundamental violation of the Law of Demeter. The Law of Demeter can be summed up as “Ask for what you need. If you need to go through one object to get what you need, you are breaking it.”. A much more fancier definition can be found on the web if you care, but the minute you see something like A.B().C() or something along those lines, there is a potential violation.

In this case, MyGuiComponent doesn’t really care about the POJO. It only cares about some fields in the POJO, which it then assigns to its own fields. But the constructor still asks for the POJO instead of directly asking for the fields. What this means is that instead of just creating an instance of MyGuiComponent with the required fields in my test, I now have to create the POJO with the required fields and pass that in instead of just setting it directly. Convoluted, anyone ?

Mixing Concerns

Finally, what could be considered an extension of the previous one, but deserves a rant of its own. What the fundamental problem with the above class is that it is mixing concerns. It knows both how to create itself as well as how to render itself once it has been created. And the way it has been coded enforces that the creation codepath is executed every single time. To provide an analogy for how ridiculous this is, it is like a a Car asking for an Engine Number and then using that number to create its own engine. Why the heck should a car have to know how to create its engine ? Or even a car itself ? Similarly, why should MyGuiComponent know how to create itself ? Which is exactly what is happening here.

Solution

Now that we had arrived at the root of the problem, we immediately went about trying to fix it. My immediate instinct was to pull out MyGuiComponent into the two classes that I was seeing. So we pulled out a MyGuiComponentFactory, which took up the responsibility of taking in the POJO and creating a GuiComponent out of it. Now this was independently testable. We also added a builder pattern to the MyGuiComponent, which the factory leveraged.

class MyGuiComponentFactory {
    MyGuiComponent createFromPojo(ComponentId id,
                                  Component parent,
                                  MyDataContainingPOJO pojo) {
      // Actual logic of converting from pojo to
      // MyGuiComponent using the builder pattern
     }
}
class MyGuiComponent {
    public MyGuiComponent(ComponentId id,
                          Component parent) {
        super(id, parent);
    }
    public MyGuiComponent setName(String name) {
        this.name = name;
        return this;
    }

And now, suddenly (and expectedly, I would like to add), the constructor for MyGuiComponent becomse simple. There is no work in the constructor. The fields are set up through setters and the builder pattern. So now, we started writing the unit tests for the render methods. It took about a single line of setup to instantiate MyGuiComponent, and we could test the render method in isolation now. Hallelujah!!

Disclaimer :
No code was harmed / abused in the course of the above blog post. There were no separation issues whatsoever in the end, it was a clean mutual break!

, , , , , , ,

1 Comment

Fakes and Mocks and Stubs, Oh My!!!

So we covered how to use EasyMock to write tests in one of the previous posts, but Mocking is not the only option when you want to test something that depends on a slow and expensive service. Mocking allows you to expect calls and return values when a call is made, and make sure the number of calls were correct and the order, but sometimes, you want a bit more, or not even that much. So in those cases, you have the options of using Stubs or Fakes. So without further delay, your options are :

  1. Mocks :
    Well, Mocks… What can I say. I have already ranted and raved about the awesomeness that is Mocks. Mocks and Mocking frameworks allow you to replace heavy and expensive services with Mocks, with which you can set expectations on what calls are made to the service, and return values. This gives you the advantage of knowing exactly what is called and returned, and makes your test deterministic and super fast. You can even set expectations on the number of calls and the order, though this is something that is not extremely recommended, since that makes the test dependent on the implementation, which is never a good thing.

    Advantages :

    • Fast, Reliable
    • Deterministic
    • Lightweight
    • Control over expectations and return values.

    Disadavantages :

    • Can become dependent on implementation of method being tested.
    • Can become a mockery if not careful, that is, can be testing interaction between mocks and nothing of the actual code. Especially when all that the method under test is doing is delegating to a bunch of service calls.
    • Can involve complicated setup for expectations, especially with unwieldy objects
  2. Stubs :
    Now, whereas with Mocks, you can specify what calls will be received and what will be returned making them somewhat intelligent, Stubs are at the other end of the spectrum. The dumbest of the dumb, the easiest of the easiest, these just stub out the methods of your expensive and heavy class. So a null Stub could return null for each method, or just return some constant value each time it is called. That means that regardless of what the method is called with, it returns the exact same thing, day in and day out. It doesn’t adjust, it doesn’t react, just returns. Of course, this means that you might not be able to test all the cases you want to inspect / test. But its simple, its easy, and of course, its fast.

    Advantages :

    • Fast, reliable
    • Simple
    • Consistent

    Disadvantages :

    • Dumb
    • Can’t exercise different cases without different stubs
    • Returns only one value consistently
  3. Fakes :
    And finally, we come to the smartest (figuratively) of the lot. The Fakes. For once, being a fake can be good. While Stubs return the same thing again and again, and mocks return what you tell it to, Fakes are smarter. The easiest way to explain a Fake is with an example. Say your code depends on a heavy service like a Database. A FakeDatabase would be an in-memory implementation of the DB, thus making it faster while at the same time, providing the same logic as the normal DB would. There are different types of Fakes, like a listening / recording Fake which records all the values passed to it.

    Advantages :

    • Can test while preserving behavior of dependencies
    • Faster than using actual services
    • Can test complicated logic
    • Most comprehensive testing approach

    Disadvantages :

    • Can be complicated to set up
    • Not as fast as mocks / stubs
    • Not as easy to define expectations
    • When / Where do you test the fake (especially the complicated ones) ?

Thats my 2 cents on the different approaches to testing with dependencies. Now that we know what we can do with those darn heavy dependencies, I’ll talk more on how to make sure you can use these different approaches to test your code soon.

, , , , ,

5 Comments

Mock, Mock! Who's there ?

The concept of mocking is not a new idea, but its one that has been gaining traction recently. But still, whenever I tell people to mock out their dependencies when they are trying to test, they look at me as if wondering what the heck I’m smoking. Well, if I was smoking something, I would share it. But mocking is a great thing for writing small unit tests.

I get usually one of the two statements / questions when I tell someone to just mock something out.

  1. Well, if I am mocking things out, then I am not really testing it, am I ? OR I don’t want to mock things out. I need to test the method’s interaction with other classes.
  2. What about the mocked class ? We have to make sure it works too.

Well, to number 1, I say, if by mocking things out, you have nothing but a series of expected calls and returns, and there’s no actual class specific behavior there, then does that Class really deserve to be a class ?

And the second part of number 1, well…. Is that really a unit test then ? Ideally, you should have a lot of unittests to make sure the class specific logic is sound, and then a few integration tests to make sure that everything is hooked up correctly.

And number 2… This is where you go and write unit tests for the class you have just mocked. Don’t depend on large scale integration tests to test all your classes. Write nice small unit tests which are fast and precise. The larger a test gets, the harder it is to find why a particular test broke and how.

So hopefully by now, I have you convinced that mocking may not be all that bad. Fine, you say, so how do I go about this mocking thing ? Glad you asked. While there are many mocking frameworks out there, I am going to just talk about EasyMock for Java. JMock is very similar and could be pretty interchangable.

The first requirement before you can use any mocking framework is the ability to inject mocks into your class. This is Dependency Injection at its core, without which you will have to find workarounds like protected setter methods and the like. But basically, if a class uses some Service or Database, make sure that you can override it with a Mock Database by passing it into a constructor or setting it via a method.

Once thats done, the first step in using a mock is creating the mock object. In EasyMock, its as simple as :

MyClass myObject = EasyMock.createMock(MyClass.class);

Thats it, nothing fancier than that. It helps if MyClass is an interface, but I believe EasyMock supports mocking non interface classes as well. Once you have done that, you can inject this mock object into the classes which you are testing.

The next step is setting expectations. For void methods, it as simple as just calling the method with the expected parameter. For example :

myObject.myVoidMethod(“ThisStringWillBePassed”);
EasyMock.expect(myObject.methodWithReturnValue(“ExpectedArg”)).andReturn(“ReturnValue”);
EasyMock.replay(myObject);

The EasyMock.replay(myObject) tells EasyMock that you are done setting expectations and that the next time a method on the object is called, treat it as an actual call. So then in your test, you proceed as normal invoking the methods you care about, and then finally, you call :

EasyMock.verify(myObject);

This ensures that all the expectations set on myObject were met. Now EasyMock also supports additional features like setting expectations on number of method calls, throwing exceptions, flexible argument matchers and so much more. For more information, check out the EasyMock home page.

Now a few caveats with regards to mocking. It is very easy to degenerate some tests into what we call a mockery, where we end up testing mocks and their interaction instead of the actual class we want to test. So Don’t overuse mocks. Use them when you have to test something which depends on Database or expensive service calls. Also if your test ends up exercising a bunch of mock calls and nothing else, that might be a hint that your class really does not belong. And of course, it goes without saying that don’t mock the class you are testing.

Also, don’t set up Mock layers where a class which indirectly depends on some service object uses the mock layer. You should always mock the classes which your Class Under Test directly depends on, and not classes which it indirectly depends on. And sometimes, a mock might not be what you are looking for. Instead, a simple Stub or a Fake might be more useful. I might talk more on this or Dependency Injection in my next post.

, , , , , ,

No Comments