Phil Varner

I spent a few hours this week trying to figure out why some date manipulation methods I was writing weren't working. In my test case, I had two instances of GregorianCalendar which I was comparing, an original and one that had been round-tripped through some conversion methods, via oracle.jbo.doman.Timestamp. Using the equals method, they were returning false. I used the toString method to see if there was something obviously different, but the strings were exactly the same. I even when so far as to step though equals in the debugger, until it disappeared into a JDK implementation class for which the source was unavailable. Even stranger, I discovered using compareTo returned 0. I set about printing each property of the instances until I discovered problem: the GregorianChange property.

In the tests, I was using XMLGregorianCalendar to take an ISO 8601 date string (e.g., 2006-09-22T00:00:00.000-00:00) and create a GregorianCalendar. The intention of this was to allow users to create statements like "if Order.timestamp > DateLib.from("2006-09-22T00:00:00.000-00:00"), do something". Below is the reduction of what was happening in the test and application code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import java.util.GregorianCalendar;
import javax.xml.datatype.DatatypeFactory;
 
public class CalTest {
    public static void main(String[] args) throws Exception {
 
        String dtstring = "2006-09-22T00:00:00.000-00:00";
        GregorianCalendar gcFromXmlgc =
            DatatypeFactory.newInstance()
                .newXMLGregorianCalendar(dtstring).toGregorianCalendar();
        GregorianCalendar gcFromCons = new GregorianCalendar();
 
        gcFromCons.setTimeZone(gcFromXmlgc.getTimeZone());
        gcFromCons.setTimeInMillis(gcFromXmlgc.getTimeInMillis());
 
        System.out.println("equals? " + gcFromXmlgc.equals(gcFromCons));
        System.out.println("compareTo? " + gcFromXmlgc.compareTo(gcFromCons));
        System.out.println("XML GC: " + gcFromXmlgc.getGregorianChange());
        System.out.println("new GC: " + gcFromCons.getGregorianChange());
 
    }
}

Running this produces the following output:

equals? false
compareTo? 0
XML GC: Sun Dec 02 08:47:04 PST 292269055
new GC: Thu Oct 04 16:00:00 PST 1582

equals is false, but compareTo is 0? That's curious. The last two lines show why, now explained.

The core of the problem is XMLGregorianCalendar is intended to represent the W3C XML Schema 1.0 date/time datatypes. The date/time type in XML Schema essentially implements ISO 8601, with a few minor exceptions. The important part here is that it represents a "proleptic Gregorian calendar".

The Gregorian calendar is the calendar used exclusively in the West and by most of the global business world. Most people in the West don't even know there's even a name for the calendar, they just know it as "the calendar". The Gregorian calendar was adopted in 1582 to correct for the gradual drift caused by the previous Julian calendar. This change set the nominal date back 10 days and added a more astronomically appropriate leap year rule. The "proleptic" Gregorian calendar is then the Gregorian rules applied to dates before the calendar was adopted, as if the calendar had always been in effect. Therefore, all Western dates after 1582 are according to the Gregorian calendar, but any date before this can be in either the Julian calendar or the proleptic Gregorian calendar, with one always being specified explicitly.

Knowing this, one would then assume that the class GregorianCalendar implemented a proleptic Gregorian calendar. However, belying it's name, it actually represents a hybrid Julian-Gregorian, with each type having a set of rules as to the order of different dates, and the ability to set a date which defines when the rules cutover. The well-known Joda-Time library names these correctly, with a proleptic Gregorian, a postleptic Julian, and a hybrid GregorianJulian. The JDK 1.5 documentation includes this description, but it's not really useful unless you know what the semantics behind it mean. GregorianCalendar.toString() doesn't print it out, so you have to know it exists and to explicitly examine it when debugging. For most code using dates, the cutover date doesn't matter, since most software isn't going to be used for dates in the distant past, and if it is, there's hopefully a domain expert on hand to specify this sort of thing.

A default instance of GregorianCalendar implements a non-proleptic calendar — dates before to 4 October 1582 are in Julian, dates after are Gregorian. However, XMLGregorianCalendar implements a proleptic Gregorian calendar as specified by ISO 8601, so it sets the change date to Long.MIN_VALUE. The Timestamp class uses millis since the epoch (1970), so it will always give us a representation of the same date, but not the same GregorianCalendar. In Sun JDK 1.5, the only difference in equals between Calendar and GregorianCalendar is the change date:

1
2
3
4
5

 public boolean equals(Object obj) {
        return obj instanceof GregorianCalendar &&
        super.equals(obj) &&
            gregorianCutover == ((GregorianCalendar)obj).gregorianCutover;
    }

GregorianCalendar inherits compareTo (implementing Comparable) from Calendar, which simply compares the millis since the epoch. GNU Classpath 0.18 has a different implementation, as it doesn't take into account the change date:

1
2
3
4
5
6
7

  public boolean equals(Object o) {
    if (! (o instanceof GregorianCalendar))
      return false;
 
    GregorianCalendar cal = (GregorianCalendar) o;
    return (cal.getTimeInMillis() == getTimeInMillis());
  }

In writing my tests, I have made a huge error– I was testing for object equality, when I should have been testing for value ordering. I shouldn't have cared if the objects were the same, what I really cared about was that there was no distinguishable ordering between them– that both objects represented the same value.

I didn't actually write the tests from scratch, I had actually translated them from other date tests written in Oracle Business Rules RL, which as semantics similar to Java, but not exactly. The relevant difference here is that you can use the relative operators on Comparable implementors, which then are converted to calls to compareTo at runtime. I made the mistake of regex-replacing a bunch of obj1 != obj2 to !obj1.equals(obj2), when I should have done obj1.compareTo(obj2) != 0. If these had been any of the relative operators, I would have used compareTo, but != took me to !equals. Big mistake.

I learned a lot from this distraction, not just about calendars, but more importantly about equals and compareTo. I recently started reading the new version of Effective Java, so the sections on comparison and equality have new and deeper meaning for me now.

The past couple days at work, I've been migrating all of our JUnit 3 tests to TestNG. The main motivation was the ability to easily create arbitrary collections of tests. When working on a single bug or feature, it's common to write a test that only exercises the code you're working on so it doesn't take minutes between test runs. I'd previously been manually editing the suite() method to comment out all but the test I wanted, but a couple of times recently I'd forgotten to uncomment them before running all of the the tests and merging to source control. The other related thing is having test methods that get accidentally removed from suite, so they're never run and it's not obvious that they're not running. Beyond the greater feature set and flexibility of TestNG, this was enough to motivate a migration.

Also, I recommend the book Next Generation Java Testing by Cédric Beust and Hani Suleiman. I haven't read the entire book yet, but the parts I have read are very good, much better than other testing book I know of.

The migration was simple using the JUnitConverter utility class included with TestNG. The main issues encountered during migration were:

• Indent for the "@Test" annotations is set at at 2 spaces, and our codebase is 4
• Any "assert(String message, String, String)" calls need to be reversed. Fortunately, I was lazy when I wrote most of the tests, so they didn't have messages. And, I wrote most of the two-arg asserts in the wrong order, so now they're correct.
• The "assert" methods are static in Assert, so they need to be qualified with the class name or static imported in every class. Most of our tests extended a base class that extended TestCase, so I copied all of the methods in Assert into this base class, and via the magic of vim and regex capture, created a method for each that would just pass through to the equivalent Assert method. I then used a grep/sed script to change the few classes that inherited TestCase directly so they inherited my base class. An alternate solution would have been to do replace of all of the "import org.junit.*" with "import static org.testng.Assert.*;". I left all of the junit imports in so that the now-unused suite() methods would continue to compile, and I'm going to go back later and remove all of them.

After migrating the tests, I started to setup the ant targets to call them.
First I tried:

   <target name="run-testng" depends="init,compile" >
       <testng classpathref="common.class.path" groups="fast">
           <classfileset dir="${twork.sdk2}" includes="test/**/*TestCase.class"/>
       </testng>
   </target>

However, this gave the error:

run-testng:
  [testng] Exception in thread "main" org.testng.TestNGException:
  [testng] Cannot load class from file: /scratch/FirstTestCase.class
  [testng]     at org.testng.TestNGCommandLineArgs.fileToClass(TestNGCommandLineArgs.java:691)
  [testng]     at org.testng.TestNGCommandLineArgs.parseCommandLine(TestNGCommandLineArgs.java:232)
  [testng]     at org.testng.TestNG.privateMain(TestNG.java:831)
  [testng]     at org.testng.TestNG.main(TestNG.java:818)

This is causd because I hadn't included the compiled test case class files on the classpath with which the testng target was called. Adding the classes using the classpath tag (same as in the junit ant tasks) fixed it:

   <target name="run-testng" depends="init,compile" >
       <testng groups="srg">
           <classpath>
                               <pathelement path="${twork.sdk2}"/>
                               <pathelement path="${common.class.path}"/>
                               <pathelement location="${sdk2.tsrc.dir}"/>
           </classpath>
           <classfileset dir="${twork.sdk2}" includes="tests/**/*TestCase.class"/>
       </testng>
   </target>

The component I work on is a library that is indented to be used inside of a JEE container, so we have an EJB that runs all of the tests inside of it. It depends on some external files, so it gets passed a map with all of these variables in it, which it then sets the system properties with so the tests can get to them. (Yes, there is probably a better way to do this, but I wrote this about 2 1/2 years ago and it works). I changed our EJB method 'runTests' to use the programmatic TestNG interface and a custom TestListenerAdapter that would just return the the EJB client a string of dots, 'F's, and 'S's:

   public String runTests(Map map){

           for (Map.Entry entry: map.entrySet()){
               System.setProperty(entry.getKey(), entry.getValue());
               }

           TestNG tng = new TestNG();

           tng.setTestClasses( new Class[] {
               com.foo.FirstTestCase.class,
               com.foo.SecondTestCase.class,
           } );

           final StringBuilder sb = new StringBuilder();

           tng.addListener(
               new TestListenerAdapter() {
                   @Override public void onTestFailure(ITestResult tr) { sb.append("F{" + tr.getName() +"}"); }
                   @Override public void onTestSkipped(ITestResult tr) { sb.append("S{" + tr.getName() +"}"); }
                   @Override public void onTestSuccess(ITestResult tr) { sb.append("."); }
               }
           );

           tng.setGroups("srg");
           tng.run();

           return sb.toString();
   }

Finally, someone else had modified several test files since I had started the conversion, so I need to find the one test missing the @Test annotation. I updated from the source control system and then ran this:

find . -name '*.java' | xargs grep -A 2 "@Test" | grep "public void" | sed -e s/java-/java:/g | sort > out
find . -name '*.java' | xargs grep "void test" | sort > out2
diff out out2

The result was several lines long since it include a few tests that had been entirely commented out and therefore weren't annotated, but more importantly it included the one test method that had been added.

Overall, the process was smooth and I'm quite happy with how it went.