Thursday, 4 September 2014

FAQ: Should I use JUnit or TestNG, which is better?

Since I do receive this question via email quite a lot, I will answer it as an FAQ.

"Should I use JUnit or TestNG? Which is better? You use JUnit in your training, so is that the better option?"

I use JUnit in my online training courses, and in my book. I do not do this because I consider it 'better' than TestNG. I do it because I consider it simpler to use and understand than TestNG, and that it has better documentation online.

In my experience of production work, I have only used TestNG on one project. On all other projects JUnit was the test execution tool that we used. So I extrapolate from that, that people are more likely to encounter JUnit experience in their day to day job, than TestNG experience. And that experience is useful to draw upon, when learning.

TestNG and JUnit are both very similar for simple actions, so most of what you need to know on my courses applies to both tools.

I have worked with TestNG on one project. And on that project, it was only the test team that used it. Everyone else used JUnit. The test team used it because of its branding as a "Functional" test tool, and not just a Unit test tool. e.g. the tagging and dependencies etc. This was years ago and there were considerable differences between the tools at this point. I think JUnit was still JUnit v 3 and the TestNG book had just been released. Also the people making the decision were not particularly experienced in the test execution frameworks. In fact, it was I that chose to use it, because I saw many people online saying it was better for 'functional' testing e.g GUI testing and API testing.

In the end, we moved over to JUnit. Not because TestNG was 'bad', but because JUnit was a better fit for our applications, process and environment:

  • JUnit 4 came out
  • We could do pretty much what we wanted in either tool
  • Our developers knew and use JUnit so we could use their experience
  • It dropped one barrier from having shared code ownership with the development team
  • JUnit documentation online was easier to use and understand than the TestNG book

In my production work now, I try to keep my test code as simple as I can, so I often avoid the advanced features of JUnit, and the advanced features of TestNG, meaning there is less to distinguish between them.

When I look at the maintenance and documentation over time, I see JUnit as having undergone more change and maintenance than TestNG, with JUnit deliberately trying to focus on its core strengths and delegate 'side' functionality (e.g. assertions and matching) off to Hamcrest. Thereby keeping it simpler.

JUnit data driven testing isn't simple. But I found TestNG data providers were not that simple to get my head around when I started either. But JUnit data driven does pretty much what I need, and I can always try JUnit params if I think I need to make it easier.

So, to summarise, I use JUnit on my training because:

  • I use it in production
  • Most of the production teams I have worked with use JUnit
  • I think JUnit is easier for beginners to work with
  • JUnit has more online resources to help
  • I think the basic JUnit skills you learn are transferrable to TestNG, if you use that on your site

Feel free to evaluate the features of both on your site and make your own informed decision, once you have learned the basics of the automation you need to do.

For related writing on this topic see: