Thursday, 4 September 2014

FAQ: Should I use JUnit or TestNG, which is better?

Since I do receive this question via email quite a lot, I will answer it as an FAQ.

"Should I use JUnit or TestNG? Which is better? You use JUnit in your training, so is that the better option?"

I use JUnit in my online training courses, and in my book. I do not do this because I consider it 'better' than TestNG. I do it because I consider it simpler to use and understand than TestNG, and that it has better documentation online.

In my experience of production work, I have only used TestNG on one project. On all other projects JUnit was the test execution tool that we used. So I extrapolate from that, that people are more likely to encounter JUnit experience in their day to day job, than TestNG experience. And that experience is useful to draw upon, when learning.

TestNG and JUnit are both very similar for simple actions, so most of what you need to know on my courses applies to both tools.

I have worked with TestNG on one project. And on that project, it was only the test team that used it. Everyone else used JUnit. The test team used it because of its branding as a "Functional" test tool, and not just a Unit test tool. e.g. the tagging and dependencies etc. This was years ago and there were considerable differences between the tools at this point. I think JUnit was still JUnit v 3 and the TestNG book had just been released. Also the people making the decision were not particularly experienced in the test execution frameworks. In fact, it was I that chose to use it, because I saw many people online saying it was better for 'functional' testing e.g GUI testing and API testing.

In the end, we moved over to JUnit. Not because TestNG was 'bad', but because JUnit was a better fit for our applications, process and environment:

  • JUnit 4 came out
  • We could do pretty much what we wanted in either tool
  • Our developers knew and use JUnit so we could use their experience
  • It dropped one barrier from having shared code ownership with the development team
  • JUnit documentation online was easier to use and understand than the TestNG book

In my production work now, I try to keep my test code as simple as I can, so I often avoid the advanced features of JUnit, and the advanced features of TestNG, meaning there is less to distinguish between them.

When I look at the maintenance and documentation over time, I see JUnit as having undergone more change and maintenance than TestNG, with JUnit deliberately trying to focus on its core strengths and delegate 'side' functionality (e.g. assertions and matching) off to Hamcrest. Thereby keeping it simpler.

JUnit data driven testing isn't simple. But I found TestNG data providers were not that simple to get my head around when I started either. But JUnit data driven does pretty much what I need, and I can always try JUnit params if I think I need to make it easier.

So, to summarise, I use JUnit on my training because:

  • I use it in production
  • Most of the production teams I have worked with use JUnit
  • I think JUnit is easier for beginners to work with
  • JUnit has more online resources to help
  • I think the basic JUnit skills you learn are transferrable to TestNG, if you use that on your site

Feel free to evaluate the features of both on your site and make your own informed decision, once you have learned the basics of the automation you need to do.

For related writing on this topic see:

Tuesday, 17 June 2014

Getting mvn setup to persist on a Mac

I installed Java and Maven on my mac.

The default Java install for mac worked fine and setup my JAVA_HOME etc., all fine.

I installed Maven and followed the instructions on my mac. (

Again fine...

Until I restarted my mac or created a new terminal.

Then I found the additional secret instructions, which I'd forgotten because I've been away from Unix for so long.

I needed to edit the bash profile.

As explained in this StackOverflow article "mvn-command-not-found-in-osx"

I needed to:

vi ~/.bash_profile

Then add the lines

export M2_HOME=/usr/local/apache-maven/apache-maven-3.2.1export M2=$M2_HOME/binexport PATH=$M2:$PATH

I skipped the JAVA_HOME because when I typed echo $JAVA_HOME in the terminal it had already been exported correctly. If you had to add the JAVA_HOME variable during your installation then you might need to add the export for that in your bash_profile as well.

Remember, with vi, to save the file, press 'esc' and then type ':wq'

esc :wq

I then opened a new terminal, and typed mvn and all was well.

Friday, 13 June 2014

Source code for Java For Testers Book Available on GitHub

I knocked off one thing on my todo list:

[x] release source code for Java For Testers book 

 You can find it over on github:

It will still change

The source code is likely to change in a number of ways:

  • package restructuring
    • I haven't decided if the current package structure supports the book reader or not
  • code will be removed
    • Some of the code classes are not relevant to the book. I use them because I have excerpts from them in the book, but some of them represent intermediate forms of the code in the book, rather than the final forms you find in most code repos e.g. classes without the methods yet, because I show the class without any methods in the text of the book
  • more code
    • I'm likely to add more code because I'm still working on the book. I'm doing my second draft at the moment, and I've been adding extra examples and more exercises, all of which require code.

This is the code in the book

I made a decision when I finished Selenium Simplified that I would never write another book with code in it, unless I had a way of 'pulling' the code out of my main code base, so that I would never have any more copy and paste, or find and replace, editing errors.

So I wrote a macro parser which scans my source code, extracts snippets to text files, and I include the text files in the book.

This means the code in the book is taken directly from the working code base.


I didn't want to release that code to the world as it isn't particularly readable since it has a whole bunch of nested macros in it.

And that was why I haven't released the source code until now.

I expanded my macro processor to kick out a version of the code base without the macros, and that is what I have released to github.

More to come

I'm still working on the edits for the book. So the code will change. As will the book.

The book "Java for Testers" is available to purchase from leanpub with a starting price of $4.99 (I really must get around to increasing the price - perhaps when I've finished my edits)

Monday, 3 March 2014

How to fix Java Language Level Issues

When using other people's source code you might discover that you receive an error like.

"java: strings in switch are not supported in -source 1.6
(use -source 7 or higher to enable strings in switch)"

Or something similar.

Define Language Levels in the pom.xml

I tackle this in two ways. One for the command line and one for the IDE.

For the command line I add the source and target to the maven compiler plugin


By default the maven compiler is set to Java 1.5

I tend to want to use features in 1.6 or 1.7, so I set it to 1.7

Having the above in your pom file should mean that when you import the project into your IDE it sets the language settings appropriately.

And should run properly when you run "mvn clean compile"

Adjust the Compiler Settings

In Settings we have the Compiler settings

  • File \ Settings \ Compiler \ Java Compiler
We might need to set these up to use the level we need.

Adjust the Project Settings

When running in the IDE we might need to change a few things

In IntelliJ you find the language settings in a few places.

Project level we can define the SDK and the Language Level.

  • File \ Project Structure | Project Settings \ Project 
We can override the SDK or set the Project Language Level

Staying on Project we want to check the Platform Settings

And that we have an SDK of the required version setup and in use.

Check Modules Settings as well

You might even need to check in the Module settings on the sources tab
  • File \ Project Structure | Project Settings \ Modules