Archive for category Tool
Using Tracker for Agile projects
I first got exposed to the Agile methodology when I took a “Thinking in Agile” course. It was a two day course, and they walked you through what it meant to be agile, the processes involved, etc. But thinking back, I fell into the same trap I did back in college. It was mostly talk, and not enough action. I learn by doing, and ipso facto, nothing really stuck. One and half years later, still at google, I became part of an internal project, which needed a kick start. And lo and behold, we decided to subscribe to the agile philosophies and not half ass it as many do. And we decided to use Tracker to manage the project.
The basic ideologies of Agile (and Tracker) are as follows. You work in short iterations (preferably a week or two). Every iteration, the entire team gets together to talk about what got done, and what is up next. You interact directly with the customer or his proxy, who gives you things to do in the form of stories. For a company search app, a story might look something along the lines of “As a product owner, I want to be able to search for a particular employee, so that I know what he works on.” Notice that this story is very well defined, especially for you as a developer. It tells you the target audience, what functionality is needed and why. The why is important because sometimes (not always), you might be able to think of a better way to do something, in which case you can pipe up with said suggestion. The example project below ignores that suggestion and I didn’t spend time to change the demo to meet my suggestions.
Tracker screen
Now tracker allows you to add and move stories around. You can move a story from the Icebox (which is where stories end up by default) into the Backlog, which are the things you need to work on. You can arrange them to order them by priority (and it is all immediate, so someone else with tracker open can see those changes as they happen). Now finally, during your iteration meeting, you sit as a team, and estimate how much each story is worth.
The most important point here is that clients are the only ones allowed to request stories (though you can probably figure out exceptions if you really think that some chores should get points). Customers get to decide what stories need to happen, and what order they need to happen in. Nothing more, nothing else. Now as developers, your role is to estimate how long it will take for the items in order. You can start off with some estimates like, “A point is one days work.” initially, when you are new. You can play planning poker to estimate, which is a fun little activity in itself to make sure no one is influenced by anyone else’s estimates. Now what tracker will do is it will measure how many stories you end up actually delivering per week, and calculate what we call your Average Velocity (pointed out in the screenshot above). This basically denotes how much tracker thinks you can do in the next iteration, assuming that your estimate base remains constant.
Another important thing to note, that only things that the customer cares about, or is customer facing, should be stories. These are thing the customer can see when delivered and brings value to him. So that refactoring you need to do to come out of that hole you dug for yourself? Guess what, its a chore and does not contribute to your velocity. Want to move database schemas to make it easier for yourself? Fixing that bug you introduced in trying to rush through all the stories? The customer doesn’t care (well he probably cares about the bugs, but you are not delivering value, you are cleaning up after your own mistakes), so no cookies nor any points for you.
So a basic flow for an iteration (after estimations and planning poker) is as follows ;
- Customer / Product owner prioritizes stories / chores / bugs in the backlog
- Tracker looks at average velocity and figures out how many it can squeeze into the next iteration
- Developers click start on a story / task when they start to work on it.
- They click finish when they are done implementing on it, but do not click on Deliver
- Pushmaster / Release Engineer clicks Deliver when those changes are pushed to a customer visible place
- Customer gets to try out each story, and then can decide whether it meets specifications, and decides if he wants to accept / reject.
Rinse and repeat, and you have a great way of managing requirements, release plans and so much more. You can figure out who’s working on what, you also get a great host of charting, including burndown charts, charts which allow you to figure out where you are spending the majority of time (whether stories, bugs, chores, etc). Example chart below :
An example burndown chart from Tracker
Did I mention Tracker is currently free to sign up and start using? Regardless of whether you are a product manager who wants to keep his project in line, a developer interested in using agile practices, or just plain curious about what this thing is all about, Tracker has something for everyone. So, what are you waiting for, a personal invitation? You don’t need to be an Agile team to try this out for yourself.
Super fast JS Testing
Before I jump into how exactly you can perform super fast and easy JS testing, let me give you some background on the problem.
Javascript is a finicky language (Some people even hesitate to call it a language). And it can easily grow and become a horrible and complicated beast, incapable of being tamed once let loose. And testing it is a nightmare. Once you have decided on a framework (of which there are a dime a dozen), you then have to set it up to run just right. You need to set it up to actually run your tests. Then you have to figure out how to run it in a continuous integration environment. Maybe even run it in headless mode. And everyone solves it in their own ways.
But the biggest problem I have with most of these frameworks is that executing the tests usually requires a context switch. By that, I mean to run a JSUnit test, you end up usually having to open the browser, browse to a particular url or html page which then runs the test. Then you have to look at the results there, and then come back to your editor to either proceed further or fix your tests. Works, but really slows down development.
In java, all it takes is to click the run button in your IDE to run your tests. You get instant feedback, a green / red bar and details on which tests passed and failed and at what line. No context switch, you can get it to run at every save, and proceed on your merry way. Till now, this was not possible with Javascript.
But now, we have JS Test Driver. My colleagues Jeremie and Misko ended up running into some of the issues I outlined above, and decided that going along with the flow was simply unacceptable. So they created a JS Testing framework which solves these very things. You can capture any browser on any machine, and when you tell it to run tests, it will go ahead and execute them on all these browsers and return you the results in your client. And its blazing fast. I am talking milliseconds to run 100 odd tests. And you can tell it to rerun your tests at each save. All within the comforts of your IDE. And over the last three weeks, I have been working on the eclipse plugin for JS Test Driver, and its now at the point where its in a decent working condition.
The plugin in action
The plugin allows you to, from within Eclipse, start the JS Test Driver server, capture some browsers, and then run your tests. You get pretty icons telling you what browsers were captured, the state of the server, the state of the tests. It allows you to filter and show only failures, rerun your last launch configuration, even setup the paths to your browsers so you can launch it all from the safety of eclipse. And as you can see, its super fast. Some 100 odd tests in less than 10 ms. If thats not fast, I don’t know what is.
For more details on JS Test Driver, visit its Google Code website and see how you can use it in your next project and even integrate it into a continuous integration. Misko talks a little bit more about the motivations behind writing it on his Yet Another JS Testing Framework post. To try out the plugin for yourselves, go add the following update site to eclipse : http://js-test-driver.googlecode.com/svn/update/. For all you IntelliJ fanatics, there is something similar in the works.
Gathering Rapid Feedback with TDD, better known as Infinitest!
So I have had this question often, especially from TDDers, on what the best way is to get rapid feedback as you type. One of the biggest things when you do Test Driven Development is the fact that you can see a Red / Green bar or some sort of indicator of your tests. Because that indicator is what tells you what you should be doing. Green bar? Time to write a failing test so you can add that next feature in. Red bar? Well, you know what is broken, so time to go ahead and write that feature. So initially, I used to just hit Alt + Shift + X, T to run the test I was currently editing. And then I learnt the joys of Ctrl + F11, which re runs your last configuration.
But still, I was left wanting more. I mean, I don’t want to have to hit something to tell eclipse to go run my tests. Eclipse already knows when I save, as it can automatically shoot off a build. Why couldn’t I just have another step afterwards which then runs my test, so I don’t have to do anything? It was along this line of thought that I stumbled upon Misko’s setup of using build steps to run all unit tests at each save. Hallelujah! But the more I used this, the more I started realizing its pain points. You actually had to setup each project to do this? Uh, no. Not happening. Too much effort. I like being lazy.
So then I happened upon this Infinitest thing. The description sounds promising, “A continuous test runner for your JUnit tests.” And what, it “integrates with Eclipse and IntelliJ“. And its “Intelligent and runs only tests that are needed.” I’m sold, where do I sign up? So I went ahead and installed it, and tried it out. And it actually seems to live up to its claim, so far. Its main configuration has a single checkbox, which basically says run Infinitest or don’t run it. Nice. And it integrates seamlessly with the problems view. Like so :
Infinitest results in the problem view
As can be seen above, I made a change which broke two tests. Half a second after I saved those changes, I have problem markers popping up all over my project telling me that the last thing I just did blew up some tests. I can double click on the markers, go to the exact line where the failure is, and see if it was an issue of the test being wrong or me being stupid. And in some cases, that isn’t even needed, because you know the tests shouldn’t have broken.
Infinitest seems to be smart enough to recognize all JUnit tests without you having to point it out, and runs only the tests that matter, and not all of it. But if you have other kinds of tests, then you might run into trouble. I am still playing around with it, and will probably update this post or add a new post later with more detailed info if I deem it necessary. But in the meantime, for those of you who want to run tests at every save, check out Infinitest. It is awesome!
Eclipse Productivity Shortcuts
Back in college, I used to be a notepad nazi, so as to say. I used to code all my giant programs solely in notepad (The most I upgraded to was to Textpad). And once I joined Google, I was apathetic to IDEs, so I just picked IntelliJ and went with it. Somewhere down the line, I attended a Testing and Refactoring workshop, and the only IDE available was Eclipse. And it was for a great reason.
While the focus of the workshop was testing and refactoring, what they did do, which I applaud them for, was they showed us some awesome shortcuts. And suddenly, I was doing these insane refactorings at the blink of the eye. It was then that I started searching for and learning every eclipse shortcut that would help me be more productive. And before I knew it, I was typing 6 words a minute and coding up 100 words a minute 
. And I loved it.
And so, today, I just wanted to share the best shortcuts that make life easier. And without further ado :
Ctrl + Space : One of the two most important keyboard shortcuts that eclipse offers. This one is probably commonly known for autocomplete in eclipse, but not many people know that it is also context sensitive. For example, hitting Ctrl + Space when you are in the middle of typing will show you all members and methods that begin with your text. But hitting Ctrl + Space when you have nothing typed shows you all members and properties available. But the real eclipse masters know that, hitting Ctrl + Space when you type in for or foreach will show you autocomplete options for generating a for loop or for each loop. And if you do it right after you assign something to a collection or a list, it will fill in the loop variables for the for each loop. Autocomplete after typing in test, will allow you to generate the skeleton of a JUnit test case method. Autocomplete after typing in new, generates you a skeleton for a new call, which you can tab through and fill in. So many more uses and use cases for Ctrl + Space that can be found. You can generate / override method signatures in child classes. Just use and abuse it, and you will learn so much.
Ctrl + 1 : If there is just one more shortcut you remember from this post, let it be this one. The other super awesome, context sensitive shortcut in Eclipse, which is basically Quick Fix. If you have an error in a line, Ctrl + 1 will show you potential options to fix it, like importing a class, or adding an argument to a method or fixing the method signature. If you just do a method call which returns something, then you can hit Ctrl + 1 and ask it to assign it to a new local or field variable. You can hit Ctrl + 1 on a parameter to a method and assign it to a field. Ctrl + 1 on a variable can allow you to inline it, and on an assignment, can allow you to split up the declaration and assignment, or convert it to a field, parameter, etc. It is, by far, The Most Awesome Keyboard shortcut that you can know and use. Especially on errors!
Ctrl + F11 : Reruns the last run configuration that was executed. If you do TDD, then Alt + Shift + X, T followed by Ctrl + F11 is the most standard approach.
Ctrl + Shift + R : Shows the Open Resource dialog. Type to filter, and jump directly between classes. I love this shortcut, and use and abuse it!
Ctrl + Shift + O : Organizes Imports, and gets rid of unused imports.
Ctrl + O : Shows the methods and properties of a class. You can start typing to filter and hit enter to jump to a particular signature / type. Hitting Ctrl + O again toggles showing inherited members. Very useful for jumping between sections in a class, or finding that one method you want to get to.
Ctrl + T : Opens the Type Heirarchy. Shows all super classes as well as sub classes / implementing types in your class path. Very useful for jumping to an implementation class. Can be called from the class type, or even a method signature. Can toggle between Supertype and Subtype heirarchy if you hit Ctrl + T again. Again, you can type and filter once you are in this menu.
Ctrl + / : Comment / Uncomment code. Single or multiple lines, depending on what you have selected. Enuff said.
Alt + Shift + R : One of my most used shortcuts, Rename. It renames anything from variables to methods to even classes, renaming the class files if necessary. Also fixes all references to refer to it by the new name. Can sometimes break if there are compile errors, so watch out when you use it. You can also ask it to fix all textual references as well.
Alt + Shift + M : Extract Method. Super useful method to break up a larger method into smaller chunks. If the code block you have selected does not need to return too many types, and looks reasonable as a separate method, then pulls up a prompt where you can basically edit the method signature, including return type, method name and order and type of parameters. Very useful
Alt + Shift + C : Only useful when the cursor is on a method signature, but this one allows you to refactor and change the method signature. This includes changing the return type, method name, and the parameters to the method, including order, and default values if you are introducing a new one. Automagically fixes all references to said method.
Alt + Shift + L : Once you have a expression selected (a method call, or whatever), then Alt + Shift + L extracts that to a local variable. It prompts you for the name of the variable, and automatically infers the type as best as it can. Extremely useful shortcut!
Alt + Shift + Up / Down : This one is a useful one. If you hit up, it selects the next biggest code block, down selects the next smallest. Useful in conjunction with refactoring shortcuts like extract local variable, extract method, etc. Useful to know.
Alt + Shift + T : Brings up the Refactor menu. Depending on the context, this will show options like Rename, Move, Extract Interfaces and classes, Change Method Signature, etc. Nice to know, but not one I use very often. The ones I do use have already been listed above.
Alt + Shift + S : Shows the Source menu. This includes menu options like Comment related, and the ever useful Override / Implement Methods, Generate Getters and Setters, and much more. Some of the menu options have direct shortcuts, but a lot of the generate commands don’t, so useful to know.
Alt + Shift + X : Pulls up the Run menu, and shows what key you have to press to run a particular type. Now I generally use this as Alt + Shift + X, followed by T, which basically executes a JUnit Test. Fastest way to run unit tests without leaving the comfort of your keyboard.
Alt + Up / Down : Moves a block of lines up or down. Rather than say, selecting, hitting Ctrl + X and then going to the place and pasting, why not just select all the lines, and use Alt + Up or Down to move them. Automatically handles indentation depending on the block. Very convenient
Ctrl + D : Nice and Simple, deletes the current line the cursor is on. If you have selected multiple lines, then they are all blown away. Much faster than selecting a line and hitting delete.
UPDATE: Adding in some of the shortcuts that I forgot or were mentioned in the comments for easy finding
Ctrl + L : Jump to a Line number
Ctrl + Shift + T : Display available types. A better version of Ctrl + Shift + R if you are only looking for Java classes
Alt + Shift + Up / Down : Duplicate selected lines above or below. Easier than hitting Ctrl + C followed by Ctrl + V
Ctrl + Alt + H : This one, I didn’t know about. but pulls up the Call heirarchy, showing you all callers and users of the method under the cursor. Super useful, especially if you are refactoring.
Ctrl + Shift + L : Show the list of shortcuts. You can hit it again to go in and edit your shortcuts.
Mock, Mock! Who's there ?
The concept of mocking is not a new idea, but its one that has been gaining traction recently. But still, whenever I tell people to mock out their dependencies when they are trying to test, they look at me as if wondering what the heck I’m smoking. Well, if I was smoking something, I would share it. But mocking is a great thing for writing small unit tests.
I get usually one of the two statements / questions when I tell someone to just mock something out.
- Well, if I am mocking things out, then I am not really testing it, am I ? OR I don’t want to mock things out. I need to test the method’s interaction with other classes.
- What about the mocked class ? We have to make sure it works too.
Well, to number 1, I say, if by mocking things out, you have nothing but a series of expected calls and returns, and there’s no actual class specific behavior there, then does that Class really deserve to be a class ?
And the second part of number 1, well…. Is that really a unit test then ? Ideally, you should have a lot of unittests to make sure the class specific logic is sound, and then a few integration tests to make sure that everything is hooked up correctly.
And number 2… This is where you go and write unit tests for the class you have just mocked. Don’t depend on large scale integration tests to test all your classes. Write nice small unit tests which are fast and precise. The larger a test gets, the harder it is to find why a particular test broke and how.
So hopefully by now, I have you convinced that mocking may not be all that bad. Fine, you say, so how do I go about this mocking thing ? Glad you asked. While there are many mocking frameworks out there, I am going to just talk about EasyMock for Java. JMock is very similar and could be pretty interchangable.
The first requirement before you can use any mocking framework is the ability to inject mocks into your class. This is Dependency Injection at its core, without which you will have to find workarounds like protected setter methods and the like. But basically, if a class uses some Service or Database, make sure that you can override it with a Mock Database by passing it into a constructor or setting it via a method.
Once thats done, the first step in using a mock is creating the mock object. In EasyMock, its as simple as :
MyClass myObject = EasyMock.createMock(MyClass.class);
Thats it, nothing fancier than that. It helps if MyClass is an interface, but I believe EasyMock supports mocking non interface classes as well. Once you have done that, you can inject this mock object into the classes which you are testing.
The next step is setting expectations. For void methods, it as simple as just calling the method with the expected parameter. For example :
myObject.myVoidMethod(“ThisStringWillBePassed”);
EasyMock.expect(myObject.methodWithReturnValue(“ExpectedArg”)).andReturn(“ReturnValue”);
EasyMock.replay(myObject);
The EasyMock.replay(myObject) tells EasyMock that you are done setting expectations and that the next time a method on the object is called, treat it as an actual call. So then in your test, you proceed as normal invoking the methods you care about, and then finally, you call :
EasyMock.verify(myObject);
This ensures that all the expectations set on myObject were met. Now EasyMock also supports additional features like setting expectations on number of method calls, throwing exceptions, flexible argument matchers and so much more. For more information, check out the EasyMock home page.
Now a few caveats with regards to mocking. It is very easy to degenerate some tests into what we call a mockery, where we end up testing mocks and their interaction instead of the actual class we want to test. So Don’t overuse mocks. Use them when you have to test something which depends on Database or expensive service calls. Also if your test ends up exercising a bunch of mock calls and nothing else, that might be a hint that your class really does not belong. And of course, it goes without saying that don’t mock the class you are testing.
Also, don’t set up Mock layers where a class which indirectly depends on some service object uses the mock layer. You should always mock the classes which your Class Under Test directly depends on, and not classes which it indirectly depends on. And sometimes, a mock might not be what you are looking for. Instead, a simple Stub or a Fake might be more useful. I might talk more on this or Dependency Injection in my next post.
The Testability Explorer cometh….
So last time, we explored on how to find hotspots and untested code in your code base. But then you start looking at your code, and then you realize there is a reason why you didn’t test the darn thing. The code’s untestable. Whoo hoo..
Well, there is no such thing as untestable code. Or rather, all untestable code can be refactored to make it much nicer and easier to test, through a variety of techniques. The first and foremost reason for untestable code ends up being “Constructor doing work.” If the constructor of a class does anything more than stuff like “this.x = x” or if it tries to call a constructor itself or use a *GASP* static factory, bingo, you have a problem.
But fixing that isn’t the target of this post. That will be covered in a later one, cause its a doozie. No, in this post, I want to talk about how to find these untestable code snippets without any manual effort. Every code base has atleast a few of these gems, which turn up being a nightmare to test, and in turn make everything depending on it a nightmare as well. Well, fear not, for the Testability Explorer cometh…
The Testability Explorer (http://www.testabilityexplorer.org) is an open source tool which looks at classes and does cyclomatic complexity analysis on it. What does that mean ? Well, it looks for things which make testing hard, like conditionals, and recursively dives in to objects it instantiates to find their testability score. In that respect, it is a static and recursive analysis of a code base. It takes all these into consideration and assigns a score to each class. Based on these scores, a class is either
- An excellent to test class
- A Good class but could use some work
- A horrible class to test, needs a lot of work.
The following image, taken from the Testability Explorer website, shows a sample report generated by the tool :
As you can see, it generates html reports with bar graphs and pie charts. It can even, depending on the options you specify, allow you to dig in deep into the problem classes and find the method and line which causes you the most problem to test. This can give you great insight on deciding what classes need refactoring first to make it testable. A lot of times, fixing the most problematic one causes a ripple effect, which fixes a bunch of problems in classes depending on it.
Another great thing is that the Testability Explorer can take jar files, so if you don’t want to expose your code directly to it, you have an option. Though sadly, the Testability Explorer is currently only available for Java code. It stands to reason that something similar could be done for C++, though you Javascript guys are out on your own.
All in all, a great tool. But don’t depend solely on it. It is great as one tool in a repertoire of tools, but not just by itself. Testability Explorer is also a great way to notice trends, of whether your code is growing more testable or untestable, and other great things, just like code coverage. Though leading you to nothing more than testable code, you would be surprised at how much positive impact increased testability and tests can have on the quality of a project.
So go check it out. And enjoy.
Of EMMA's and Eclipse's
So last time I covered the joys of testing. Now if you do Test Driven Development, then you never have to worry about what you have tested and what is untested, but what about the scores of projects which aren’t developed in a TDD fashion ? How do I figure out which among my thousands of classes needs tests most urgently right now ? Is the class which usually gets the most bug fixes my prime target, or are there even worse classes that should be tested ?
One solid easy way to identify this is to run some code coverage analysis on your code. What is code coverage, you ask ? It is one of the single most brilliant things which gives you coverage information about your code. Coverage information in this case basically provides you with knowledge on how much of your code and classes are executed by your tests, and which hotspots in your code are completely ignored by the test. Though you can run it without having a suite of tests and running your program manually, it provides the most bang for your buck (especially considering it is free) when you run it along with the tests. Code coverage tools generally provide information on a per package, per class, per method, per block and per line basis, so you can dig in as deep as you like.
EMMA is a free, open source tool which allows you to generate code coverage information for Java code. And if you have some issues with providing some random tool with your source code, fear not, for you can provide it with a jar file which it can instrument and generate code coverage information for. And it generates nice Html reports if you prefer, which you can again dig into as deep as you like.
The above image, grabbed from EMMA’s official website, shows a sample html report for a single class. Notice how nicely it highlights the class. The green lines represent code which was covered by one test or the other, the red ones were lines not covered at all and the yellow ones represent code which was partially covered. EMMA is smart enough to distinguish partial matches, as in the mutli condition statement above.
This image, also grabbed from EMMA’s official website, shows code coverage information on a package level. Notice how it breaks down the information to a method, block and line level. So this report can be used to easily identify classes lacking in testing and allows surveyors to tackle these hotspots.
To make it even easier for developers, EMMA is available as a plugin to most IDEs, including Eclipse. Available at EclEmma. This tool can be run along with the tests to generate code coverage information. SO you can instant feedback on any new test that you have written, instead of having to come out and run EMMA separately. This can also help give you feedback on your test, to ensure that you are testing the code paths that you intended.
Generally, it has been found that projects with code coverage less than 50 – 60% generally tend to have much more bugs and fixes than projects with higher coverage. And projects which are developed using TDD tend to end with high code coverage numbers, generally above 80%.
But this comes with a few caveats. Even if you do attain 100% code coverage, it does not mean that your job is done. In the end, code coverage is a statistic and can be bent or twist by a knowledgeable person. A high code coverage number ensures you are hitting a lot of your code paths and your tests exercise a lot of the system, but it does not necessarily mean that you covered all possible cases nor does it mean that your source code itself is testable or maintainable. It is also entirely possible to write as few tests as possible which exercise bigger amounts of systems and don’t provide much value, rather than writing small fast unit tests which exercise just a small part of the system and still end up with good code coverage numbers.
But that said, code coverage is an excellent tool when used as part of a greater set of tools to evaluate your project and can reveal startling trends about your projects. Maintaining a historical trend of how your code coverage grows is an interesting metric and can reveal the practices of your developers as well. And considering how it is free for Java, I don’t see any reason to not start using this for your projects.
Recent putbacks