Double dereferenced properties in Ant with Groovy

The TestNG report system is nice, but sometimes you need to integrate the TestNG output with some other test reporting system. At work, this other system requires that a single ".suc" or ".dif" file written to a common directory for each "test" you run, where "test" is defined however you want. In our case, we do one test for each TestNG group we run.

Due to Ant's immutable properties, I found it difficult to get exactly behavior I wanted. The simple way was to set a property "has.failure" if the testng-failed.xml file existed, but then would have the effect of appearing that a bunch of tests had failed, rather than a single one. It wasn't that important of an issue, since we it it was easy to see which test had actually failed afterward.

I finally got around to fixing this today. I screwed around with ant tasks for a while, but finally decided to use Groovy, which I'll choose first next time. The main issue was with the double dereference — I wanted to create a property based on the groups the test was running, and then reference it the same way, e.g., create property ${foo}.failed, and then access it like "${${foo}.failed}" (which doesn't work). Ant lets you create this property, but then requires you to jump through some as-yet-unknown-to-me hoops in order to actually reference the property. However, this is simple in Groovy, shown below.

This basically replaces the "condition" tasks in my description of calling the testng task in this post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<target name="run-testng" depends="" >
    ... call testng here
 
       <condition property="${groups}.has.failure" value="true" else="false">
          <available file="${testng.report.dir}/${groups}/testng-failed.xml"/>
       </condition>
 
       <antcall target="process-results">
          <param name="infix" value="${groups}"/>
       </antcall>
</target>
 
<target name="process-results">
       <groovy>
          def infix = properties['infix']
          def dest = properties['RESULTS_DIR']
          def suc = new File("${dest}/product.name.${infix}.suc")
          def dif = new File("${dest}/product.name.${infix}.dif")
          if (properties["${infix}.has.failure"] == "true"){
            dif.write('Pass')
            suc.delete()
          } else {
            suc.write('Pass')
            dif.delete()
          }
    </groovy>
</target>

And, yes, the framework we have requires "Pass" in the dif file– don't ask me.

Aggregated code coverage with Emma and Groovy

This post describes a script I wrote to take XML Emma output and produce multi-package aggregated statistics. One of the drawbacks of Emma's HTML reporting is that it does not allow you to get aggregated coverage information across packages. For instance, if I have packages "com.foobar.sdk.interface" "com.foobar.sdk.impl", there's no automated way to get coverage information for all packages starting with "com.foobar.sdk". Most larger projects are logically grouped like this, so having these "superpackage" groupings is useful. My previous method of getting this was to cut-and-paste the HTML from a browser into a text file, run a Ruby script on it to convert it to CSV, import the CSV into Excel, and add the necessary formulas to the sheet to get the measurements I wanted. Having it simply printed out at the end of the Emma run is much simpler.

First, the setup of Emma. Inside the <report> tag, I put the following output descriptions:

<html outfile="${emma.coverage.dir}/foobar/coverage.html"
    columns="name,class,method,block,line"
     sort="+name,+class,+method,+block,+line" depth="method"/>
<xml outfile="${emma.coverage.dir}/foobar_coverage.xml"
    columns="name,class,method,block,line"
     sort="+name,+class,+method,+block,+line" depth="method"/>

These create both the full Emma HTML report and an XML document with the same results. After calling the report target that includes this, I then use the <groovy> Ant task to call a script which parses the Emma XML and produces some output.

<echo message="------------EMMA Summary----------------" />
<groovy src="${test.scripts.dir}/EmmaParser.groovy">
  <arg value="${emma.coverage.dir}/rules_coverage.xml" />
  <arg value="com.foobar.sdk:SDK,com.foobar.tools:SDK,com.foobar.engine:ENGINE,com.thirdparty:ENGINE"/>
</groovy>
<echo message="----------------------------------------" />

The format of the second argument is comma-delimited set of Java package prefixes and "superpackage" names for which we want aggregate coverage. In the above example, all packages that start with "com.foobar.sdk" and "com.foobar.tools" are grouped into the "SDK" aggregate, and "com.foobar.engine" and "com.thirdparty" are grouped into "ENGINE". For each superpackage, the total number of lines, number of lines covered, and percentage covered are printed.

Below is the groovy script which does the EMMA XML work. A few comments on it:

  • The Groovy XmlParser class was a joy to use and vastly simplified accessing the XML document.
  • The regex was the hardest part to get right. I most commonly write regexes in vim, which requires different escaping that Groovy. It involves both captures and parenthesis in the expression. In Groovy regexes, you escape the parens you want in the expression and don't escape the capture parens. This really tripped me up on the next groovy project after this one, where I reversed the meaning when looking at this regex.
  • Closures are such a nice feature to have when parsing with XmlParser like this. Their use in iteration and assignment of local variables makes the code much shorter to read and understand.

The script:

def filename = args[0]
def config = args[1]

def pkgmap = [:]
def spkgs = [:]
def cmap = [:]
def tmap = [:]

// split the config string by comma, then by colon
config.split(',').each { entry ->
  (entry =~ /(.+):(.+)/).each { all, pkg, spkg ->
      pkgmap[pkg] = spkg
      spkgs[spkg] = ''
  }
}

// init the package map
pkgmap.each { k, v -> cmap[v] = 0; tmap[v] = 0; }

// parse the report
def report = new XmlParser().parse(new File(filename))

// get the stats for the "line" coverage for each package
// packages are non-bundling, so pkg.foo does not contain stats for pkg.foo.bar
report.data[0].all[0].'package'.each() { pkg ->
  pkgmap.each { pkgname, sname ->
      if ((pkg.'@name').startsWith(pkgname) ) {

          (pkg.coverage[3].'@value' =~ /\d+%\s+\((\d+\.*\d*)\/(\d+)\)/ ).each {
              all, cov, total ->
                  cmap[sname] += Float.valueOf(cov)
                  tmap[sname] += Integer.valueOf(total)
          }
      }
  }
}

// print summary stats for each super-package
spkgs.each { sname,x ->
  if (tmap[sname] > 0) println "," + sname + "," +
     String.format("%.2f",cmap[sname]*100/tmap[sname]) + "%," +
        cmap[sname] + "," + tmap[sname]
  else println "," + sname + ",0%,0,0"
}

Ant and Groovy Fun

In my last post, I included Ant code calling a separate Groovy script. Here is an example of an inline Groovy script inside the <groovy> tags.

The purpose of this target was to query a running Oracle app server using asctl and then regex out a specific container website port. asctl is called and the resulting text is put in the property 'ports.list'. One line in string value of this property is contains "http-web-site", possibly some whitespace, a pipe, possibly more whitespace, and then a multi-digit port string. The embedded groovy calls the regex matcher operator on the value of this property via the 'properties' map, performs a capture on the port string, and then invokes a closure that sets the 'properties' map value of 'http.web.site' to the captured value.

  <target name="set-ports">
      <exec executable="${ORACLE_HOME}/bin/asctl" dir="${T_WORK}" outputproperty="ports.list" >
          <arg line="listPorts -topoNode ${INSTANCE_NAME}/${OC4J_COMP_NAME}" />
      </exec>

      <echo>${ports.list}</echo>
      <groovy>
          (properties['ports.list'] =~ /http-web-site\s*\|\s*(\d+)/).each {
              all, port -> properties['http.port'] = port
          }
      </groovy>

      <echo>http-web-site port is ${http.port}</echo>
    </target>

We only expect one match, so 'each' really isn't the right method here, but it looks goods, works, and I don't know what else to do. Please comment if you have another way to do this, I'm always looking to up my groovy-fu.

Testing inside a servlet with Ant, TestNG, and Groovy

In a previous post, I talked about how I run my TestNG unit/integration tests from within an EJB. The EJB implemented the old 2.0 standard, which meant that maintaining all of the configuration metadata was a continual effort sink. I recently moved it to simply use a servlet, which I should have done from the beginning. Since I could no longer use the EJB client code, I had to also write an HTTP client to invoke the servlet and process the results. This post describes the Java code invoking the TestNG tests from within a servlet, the Groovy HTTP client that invokes this servlet, and the Groovy Ant task configuration and code to invoke the client script.

First, the test servlet class was created, which extends HttpServlet and implements it with the following method (this is nearly the same as the previously posted code in the EJB):

    public void doGet(HttpServletRequest request, HttpServletResponse response)
                                        throws ServletException, IOException {

        // Sets the content type of the response
        response.setContentType("text/html");
        ServletOutputStream out = response.getOutputStream();

        try {

            TestNG tng = new TestNG();

            tng.setTestClasses( new Class[] {
                OneTestCase.class,
                TwoTestCase.class
            } );

            final StringBuilder sb = new StringBuilder();

            tng.addListener(
                new TestListenerAdapter() {
                    @Override public void onTestFailure(ITestResult tr) { 
                        sb.append("F{" + tr.getTestClass().getName() + "." + 
                                          tr.getMethod().getMethodName() +"}"); }
                    @Override public void onTestSkipped(ITestResult tr) { 
                        sb.append("S{" + tr.getTestClass().getName() + "." +
                                          tr.getMethod().getMethodName() +"}"); }
                    @Override public void onTestSuccess(ITestResult tr) { 
                        sb.append("."); }
                 }
            );

            tng.setGroups("srg");
            tng.run();

            out.println(sb.toString());

        } catch (Exception e){
            StringBuilder ex = new StringBuilder();
            for (StackTraceElement ste : e.getStackTrace()){
                ex.append(ste.toString() + "\n");
            }
            out.println("Aack! " + e.toString() +"  " +  ex);
        } finally {
            out.close();
        }
    }

I find the Groovy Ant task to be an excellent way of enhancing the power of Ant. Because of it's XML structure, there are many things that are difficult or impossible to do in an Ant task. Even a simple if/then is awkward. This make complex Ant code not only difficult to write, but more importantly difficult to read. Groovy allows to easily break out of the Ant jail and write like a Real Programmer. The concision of the Groovy language, closures, and built-in data structure syntax allows you to express in a couple of lines withing the Ant file something that would a page of Ant code or several lines of Java. Most importantly when working with many other developers, Groovy is enough like Java that most programmers can infer the meaning of most Groovy code even if they have no experience with the language.

I couldn't find any instructions with "cut and paste this text into your Ant build file to use the Groovy task", so here's what I use:

<property name="ant.home.dir" value="" />
<property name="junit.jar" value="" />
<property name="tools.dir" value="" />

<property name="groovy.dir" value="${tools.dir}/groovy" />
<property name="groovy.lib.dir" value="${groovy.dir}/lib" />
    
<path id="groovy.lib">
    <pathelement location="${ant.home.dir}/lib/ant-1.7.0.jar" />
    <pathelement location="${ant.home.dir}/lib/ant-junit-1.7.0.jar"/>
    <pathelement location="${ant.home.dir}/lib/ant-launcher.jar"/>
    <pathelement location="${junit.jar}"/>
    <pathelement location="${groovy.lib.dir}/antlr-2.7.6.jar"/>
    <pathelement location="${groovy.lib.dir}/asm-2.2.jar"/>
    <pathelement location="${groovy.lib.dir}/asm-analysis-2.2.jar"/>
    <pathelement location="${groovy.lib.dir}/asm-tree-2.2.jar"/>
    <pathelement location="${groovy.lib.dir}/asm-util-2.2.jar"/>
    <pathelement location="${groovy.lib.dir}/bsf-2.4.0.jar"/>
    <pathelement location="${groovy.lib.dir}/commons-cli-1.0.jar"/>
    <pathelement location="${groovy.lib.dir}/commons-logging-1.1.jar"/>
    <pathelement location="${groovy.lib.dir}/groovy-1.5.4.jar"/>
    <pathelement location="${groovy.lib.dir}/jline-0.9.93.jar"/>
    <pathelement location="${groovy.lib.dir}/jsp-api-2.0.jar"/>
    <pathelement location="${groovy.lib.dir}/mockobjects-core-0.09.jar"/>
    <pathelement location="${groovy.lib.dir}/mx4j-3.0.2.jar"/>
    <pathelement location="${groovy.lib.dir}/servlet-api-2.4.jar"/>
    <pathelement location="${groovy.lib.dir}/xpp3_min-1.1.3.4.O.jar"/>
    <pathelement location="${groovy.lib.dir}/xstream-1.2.2.jar"/>
</path>

<property name="groovy.lib" refid="groovy.lib" />

<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy"
         classpathref="groovy.lib"/>

I think you can exclude the junit jar if you're not using it, but there's a version included with the Groovy distro if not.

The Groovy Ant task allows you to either call a separate Groovy file or embed Groovy code directly between the <groovy> tags. Below we'll call a separate Groovy script, and in a future post I'll do some embedded Groovy (specifically, querying the server to set the http.port property dynamically).


  <property name="http.port" value="" />
  <property name="app.name" value="" />

  <target name="run-tests" >

      <groovy src="${test.scripts.dir}/servletclient.groovy" >
          <arg value="http://localhost:${http.port}/${app.name}/"/>
      </groovy>

      <echo message="result running test: ${test.result}"/>

      <if>
          <equals arg1="${test.result}" arg2="true" />
          <then>
              <touch file="${test.results.dir}/t.foo.servlet.dif" />                   
              <delete file="${test.results.dir}/t.foo.servlet.suc" />               
          </then>
          <else>
              <touch file="${test.results.dir}/t.foo.servlet.suc" />                   
              <delete file="${test.results.dir}/t.foo.servlet.dif" />               
          </else>
      </if>
  </target>

The <arg> tag values are accessed in the groovy script via the array "args", with the indexing beginning at 0. Values are returned to the Ant context by setting values in the map named 'properties'. All properties in the Ant context are passed into the Groovy script in this variable, and are automatically unmarshalled back to Ant with any new entries (since Ant properties are immutable, you can change the values of the existing keys, but they won't be retained).

The Groovy HTTP client I use to access the servlet is below. This prints the content retrieved from servlet to standard out (so it appears in the ant log) and sets the value of the ant property test.result via the 'properties' map. If the output contains at least one '.' (meaning a test ran) and no instances of 'F' or 'S' (indicating a failed or skipped test), the result is 'true'. I'm certain this code could be shorter, but it's based on the longer DEWD client from Tony Landis, so I only stripped it down as far as needed to work for my purposes.

// Run tests within a J2EE container by calling a servlet.
// print the retrieved output to standard out

uri = new URI(args[0])
method='GET'
socket = new Socket(uri.getHost(), uri.getPort())

contentLen = 0
writedata = "GET " + uri.getPath() + " HTTP/1.0\r\n" +
"Host: " + uri.getHost() + "\r\n" +
"Content-Type: application/x-www-form-urlencoded\r\n" +
"Content-Length: " + contentLen + "\r\n\r\n" +
"Connection: close\r\n\r\n"  
writer = new PrintWriter(socket.getOutputStream(), true)
writer.write(writedata)
writer.flush()

// read and throwaway header
reader = new DataInputStream(socket.getInputStream())
c = null
while (null != ((c = reader.readLine()))) if(c=='') break

// read content
def row
content = ''
while (null != ((row = reader.readLine()))) content += row + "\n"  
// Response from the servlet should consist  of a string of periods
// (each period representing a successful test).
properties["test.result"] = (! (content =~ /[FS]/)) && (content =~ /\./)

println content

reader.close()
writer.close()
socket.close()

Ant, TestNG, and Groovy — such a powerful trio!