TestNG Migration

The past couple days at work, I've been migrating all of our JUnit 3 tests to TestNG. The main motivation was the ability to easily create arbitrary collections of tests. When working on a single bug or feature, it's common to write a test that only exercises the code you're working on so it doesn't take minutes between test runs. I'd previously been manually editing the suite() method to comment out all but the test I wanted, but a couple of times recently I'd forgotten to uncomment them before running all of the the tests and merging to source control. The other related thing is having test methods that get accidentally removed from suite, so they're never run and it's not obvious that they're not running. Beyond the greater feature set and flexibility of TestNG, this was enough to motivate a migration.

Also, I recommend the book Next Generation Java Testing by C├ędric Beust and Hani Suleiman. I haven't read the entire book yet, but the parts I have read are very good, much better than other testing book I know of.

The migration was simple using the JUnitConverter utility class included with TestNG. The main issues encountered during migration were:

  • Indent for the "@Test" annotations is set at at 2 spaces, and our codebase is 4
  • Any "assert(String message, String, String)" calls need to be reversed. Fortunately, I was lazy when I wrote most of the tests, so they didn't have messages. And, I wrote most of the two-arg asserts in the wrong order, so now they're correct.
  • The "assert" methods are static in Assert, so they need to be qualified with the class name or static imported in every class. Most of our tests extended a base class that extended TestCase, so I copied all of the methods in Assert into this base class, and via the magic of vim and regex capture, created a method for each that would just pass through to the equivalent Assert method. I then used a grep/sed script to change the few classes that inherited TestCase directly so they inherited my base class. An alternate solution would have been to do replace of all of the "import org.junit.*" with "import static org.testng.Assert.*;". I left all of the junit imports in so that the now-unused suite() methods would continue to compile, and I'm going to go back later and remove all of them.

After migrating the tests, I started to setup the ant targets to call them.
First I tried:

   <target name="run-testng" depends="init,compile" >
       <testng classpathref="common.class.path" groups="fast">
           <classfileset dir="${twork.sdk2}" includes="test/**/*TestCase.class"/>

However, this gave the error:

  [testng] Exception in thread "main" org.testng.TestNGException:
  [testng] Cannot load class from file: /scratch/FirstTestCase.class
  [testng]     at org.testng.TestNGCommandLineArgs.fileToClass(TestNGCommandLineArgs.java:691)
  [testng]     at org.testng.TestNGCommandLineArgs.parseCommandLine(TestNGCommandLineArgs.java:232)
  [testng]     at org.testng.TestNG.privateMain(TestNG.java:831)
  [testng]     at org.testng.TestNG.main(TestNG.java:818)

This is causd because I hadn't included the compiled test case class files on the classpath with which the testng target was called. Adding the classes using the classpath tag (same as in the junit ant tasks) fixed it:

   <target name="run-testng" depends="init,compile" >
       <testng groups="srg">
                               <pathelement path="${twork.sdk2}"/>
                               <pathelement path="${common.class.path}"/>
                               <pathelement location="${sdk2.tsrc.dir}"/>
           <classfileset dir="${twork.sdk2}" includes="tests/**/*TestCase.class"/>

The component I work on is a library that is indented to be used inside of a JEE container, so we have an EJB that runs all of the tests inside of it. It depends on some external files, so it gets passed a map with all of these variables in it, which it then sets the system properties with so the tests can get to them. (Yes, there is probably a better way to do this, but I wrote this about 2 1/2 years ago and it works). I changed our EJB method 'runTests' to use the programmatic TestNG interface and a custom TestListenerAdapter that would just return the the EJB client a string of dots, 'F's, and 'S's:

   public String runTests(Map map){

           for (Map.Entry entry: map.entrySet()){
               System.setProperty(entry.getKey(), entry.getValue());

           TestNG tng = new TestNG();

           tng.setTestClasses( new Class[] {
           } );

           final StringBuilder sb = new StringBuilder();

               new TestListenerAdapter() {
                   @Override public void onTestFailure(ITestResult tr) { sb.append("F{" + tr.getName() +"}"); }
                   @Override public void onTestSkipped(ITestResult tr) { sb.append("S{" + tr.getName() +"}"); }
                   @Override public void onTestSuccess(ITestResult tr) { sb.append("."); }


           return sb.toString();

Finally, someone else had modified several test files since I had started the conversion, so I need to find the one test missing the @Test annotation. I updated from the source control system and then ran this:

find . -name '*.java' | xargs grep -A 2 "@Test" | grep "public void" | sed -e s/java-/java:/g | sort > out
find . -name '*.java' | xargs grep "void test" | sort > out2
diff out out2

The result was several lines long since it include a few tests that had been entirely commented out and therefore weren't annotated, but more importantly it included the one test method that had been added.

Overall, the process was smooth and I'm quite happy with how it went.

Shards in your Latte

Here are the slides from my "Shards in your Latte" PJUG presentation on January 15:


The Josh Bloch / Bill Pugh puzzlers talk from JavaOne given at Google. The Elvis example comes straight from here, and they do a better example of describing it than I did.

The Java Puzzlers book.

A couple of things that came up at the talk:

  • There's no version of String.replaceAll that takes a Pattern. I said I thought there was without thinking for more than a second about it.
  • Until the most recent version of Groovy, it didn't have a classical C-style 'for' loop. There was a lot of disagreement when I mentioned it, and I should have been more precise when I said that as to not imply that Groovy had no 'for' loop constructs.

Please add anything you think I got wrong, or anything you found interesting.