TestNG Migration

The past couple days at work, I've been migrating all of our JUnit 3 tests to TestNG. The main motivation was the ability to easily create arbitrary collections of tests. When working on a single bug or feature, it's common to write a test that only exercises the code you're working on so it doesn't take minutes between test runs. I'd previously been manually editing the suite() method to comment out all but the test I wanted, but a couple of times recently I'd forgotten to uncomment them before running all of the the tests and merging to source control. The other related thing is having test methods that get accidentally removed from suite, so they're never run and it's not obvious that they're not running. Beyond the greater feature set and flexibility of TestNG, this was enough to motivate a migration.

Also, I recommend the book Next Generation Java Testing by Cédric Beust and Hani Suleiman. I haven't read the entire book yet, but the parts I have read are very good, much better than other testing book I know of.

The migration was simple using the JUnitConverter utility class included with TestNG. The main issues encountered during migration were:

  • Indent for the "@Test" annotations is set at at 2 spaces, and our codebase is 4
  • Any "assert(String message, String, String)" calls need to be reversed. Fortunately, I was lazy when I wrote most of the tests, so they didn't have messages. And, I wrote most of the two-arg asserts in the wrong order, so now they're correct.
  • The "assert" methods are static in Assert, so they need to be qualified with the class name or static imported in every class. Most of our tests extended a base class that extended TestCase, so I copied all of the methods in Assert into this base class, and via the magic of vim and regex capture, created a method for each that would just pass through to the equivalent Assert method. I then used a grep/sed script to change the few classes that inherited TestCase directly so they inherited my base class. An alternate solution would have been to do replace of all of the "import org.junit.*" with "import static org.testng.Assert.*;". I left all of the junit imports in so that the now-unused suite() methods would continue to compile, and I'm going to go back later and remove all of them.

After migrating the tests, I started to setup the ant targets to call them.
First I tried:

   <target name="run-testng" depends="init,compile" >
       <testng classpathref="common.class.path" groups="fast">
           <classfileset dir="${twork.sdk2}" includes="test/**/*TestCase.class"/>
       </testng>
   </target>

However, this gave the error:

run-testng:
  [testng] Exception in thread "main" org.testng.TestNGException:
  [testng] Cannot load class from file: /scratch/FirstTestCase.class
  [testng]     at org.testng.TestNGCommandLineArgs.fileToClass(TestNGCommandLineArgs.java:691)
  [testng]     at org.testng.TestNGCommandLineArgs.parseCommandLine(TestNGCommandLineArgs.java:232)
  [testng]     at org.testng.TestNG.privateMain(TestNG.java:831)
  [testng]     at org.testng.TestNG.main(TestNG.java:818)

This is causd because I hadn't included the compiled test case class files on the classpath with which the testng target was called. Adding the classes using the classpath tag (same as in the junit ant tasks) fixed it:

   <target name="run-testng" depends="init,compile" >
       <testng groups="srg">
           <classpath>
                               <pathelement path="${twork.sdk2}"/>
                               <pathelement path="${common.class.path}"/>
                               <pathelement location="${sdk2.tsrc.dir}"/>
           </classpath>
           <classfileset dir="${twork.sdk2}" includes="tests/**/*TestCase.class"/>
       </testng>
   </target>

The component I work on is a library that is indented to be used inside of a JEE container, so we have an EJB that runs all of the tests inside of it. It depends on some external files, so it gets passed a map with all of these variables in it, which it then sets the system properties with so the tests can get to them. (Yes, there is probably a better way to do this, but I wrote this about 2 1/2 years ago and it works). I changed our EJB method 'runTests' to use the programmatic TestNG interface and a custom TestListenerAdapter that would just return the the EJB client a string of dots, 'F's, and 'S's:

   public String runTests(Map map){

           for (Map.Entry entry: map.entrySet()){
               System.setProperty(entry.getKey(), entry.getValue());
               }

           TestNG tng = new TestNG();

           tng.setTestClasses( new Class[] {
               com.foo.FirstTestCase.class,
               com.foo.SecondTestCase.class,
           } );

           final StringBuilder sb = new StringBuilder();

           tng.addListener(
               new TestListenerAdapter() {
                   @Override public void onTestFailure(ITestResult tr) { sb.append("F{" + tr.getName() +"}"); }
                   @Override public void onTestSkipped(ITestResult tr) { sb.append("S{" + tr.getName() +"}"); }
                   @Override public void onTestSuccess(ITestResult tr) { sb.append("."); }
               }
           );

           tng.setGroups("srg");
           tng.run();

           return sb.toString();
   }

Finally, someone else had modified several test files since I had started the conversion, so I need to find the one test missing the @Test annotation. I updated from the source control system and then ran this:

find . -name '*.java' | xargs grep -A 2 "@Test" | grep "public void" | sed -e s/java-/java:/g | sort > out
find . -name '*.java' | xargs grep "void test" | sort > out2
diff out out2

The result was several lines long since it include a few tests that had been entirely commented out and therefore weren't annotated, but more importantly it included the one test method that had been added.

Overall, the process was smooth and I'm quite happy with how it went.

7 thoughts on “TestNG Migration

  • February 22, 2008 at 9:29 am
    Permalink

    Thanks! Excellent entry, saved the day for me. All and all, gotta love TestNG.
    Cheers,

    Kantorn

  • February 11, 2009 at 1:07 pm
    Permalink

    Thanks, I was coming to the same conclusion myself. The TestNG docs could us a little bit a work when it comes to ANT integration.

  • April 8, 2009 at 4:37 am
    Permalink

    Hi, Thanks for the info. During migration, I'm facing this problem. In JUint setup() i can get the name of the testcase just by calling getName(), how can i get the same within the @BeforeMethod ?

    Thanks

  • April 22, 2009 at 9:03 pm
    Permalink

    You just have to specify a parameter of type java.lang.reflect.Method in your @BeforeMethod and you will get the executed test method.

  • May 26, 2009 at 3:52 pm
    Permalink

    Thanks for this post…helped me with some issues I was having with Ant & TestNG!

  • November 2, 2009 at 10:39 am
    Permalink

    it works.thanks

  • February 4, 2010 at 1:48 pm
    Permalink

    Phil – thanks for the post, the hit on "cannot load class from file" helped me realize my omission of the test classes themselves from the classpath, doh! To add to your post, we've been using TestNG for several years now, when JUnit 3.x wasn't supporting 'groups'…it was for this reason we switched over, now we're able to classify subsets of tests and run those easily. We have lots of fast 'unit' tests that all developers run for each local build, then the CI machine runs the db-heavy 'integration' tests nightly against our code in SVN. Very very helful…

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">