Skip to content

your brain on λ, part 1

A couple more links to articles which have been influential to me:

Can Your Programming Language Do This?
by Joel Spolsky
, in which Joel describes the value of functional abstraction. I find Joel's thought on the function-imperative divide particularly interesting considering his background in Lisp from his Yale undergrad, but that he's build his software empire around various Microsoft languages, which until recently were firmly in the imperative camp. These are arguments firmly for the effect FP languages have on the way you think about programming,

John Backus's 1977 Turing Award Lecture "Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs". The book "Comparative Programming Languages" by Leslie B. Wilson and Robert G. Clark has a good description of the FP language (2nd Edition, don't know about the 3rd).

Kenneth Iverson's Turing Award page, with a link to his paper "Notation as a Tool of Thought"

Execution in the Kingdom of Nouns by Steve Yegge, an exposition of the value of verbs in languages

Your brain on λ, part 0

In this series, I plan to talk about some of the reasons why learning new programming languages is good for you as a developer, even if you never have a chance to use those languages in your work. This train of though began with preparation for a talk at my local JUG. As was obvious to anyone who was there, I don't have any answers. I'm still refining the questions, and probably will for the foreseeable future.

In future parts, I'll expound upon some of the ideas in the talk, but for now I'm just going to link to some of the material that has inspired me.

Beating the Averages by Paul Graham, in which he discusses Blub

Closures and Higher-Order Functions by Reg Braithwaite, a good simple introduction to the core of idea of functional programming

A Definition of Closures by Neal Gafter

Functional Pattern System For Object-Oriented Design by Thomas Kuehne, with summaries of the patterns on C2.com

Structure and Interpretation of Computer Programs by (and (* Sussman 2) Abelson), full text online, a/k/a the Wizard book. Videos of lectures are also available.

The Adventures of a Pythonista in Schemeland/1

Practical Common Lisp by Peter Seibel on YouTube. I watched this a couple of years ago and until recently had forgotten entirely that he talks about Sapir-Whorf.

Sapir-Whorf on wikipedia.

Alex Miller's blog post on the closures proposals, including links to each one.

Functional Java, a project to provide libraries using the BGGA closure syntax.

lambdaj, a library for doing functional-style things with existing Java constructs

Scala homepage

Java Map interface impl for composition

One of the mantras of object-oriented programming is that you should always favor composition over inheritance. However, it's always annoying when you want to implement Map but don't want to copy-paste-edit a bunch of methods that are just pass-through to the underlying map. This is code to do that, so I won't be tempted to just extend HashMap anymore.

  private final Map<String,String> m_map = new HashMap<String,String>();
 
  // begin Map impl
  public void clear(){ m_map.clear(); }
  public boolean containsKey(Object key){ return m_map.containsKey(key); }
  public boolean containsValue(Object value){ return m_map.containsValue(value); }
  public Set<Map.Entry<String,String>> entrySet(){ return m_map.entrySet(); }
  public boolean equals(Object o){ return m_map.equals(o); }
  public String get(Object key){ return m_map.get(key);}
  public int hashCode(){ return m_map.hashCode(); }
  public boolean isEmpty(){ return m_map.isEmpty(); }
  public Set<String> keySet(){ return m_map.keySet(); }
  public String put(String key, String value){ return m_map.put(key,value);}
  public void putAll(Map<? extends String,? extends String> m){ m_map.putAll(m) ;}
  public String remove(Object key){ return m_map.remove(key);}
  public int size(){ return m_map.size(); }
  public Collection<String> values(){ return m_map.values(); }
  // end Map impl

Books, January 2009

Thinking in Java by Bruce Eckel — I've literally been reading this book for 9 years. This was the book I learned Java from many years ago when I was a student, through the graciousness of Eckel making it available for free electronically. I accidentally bought two copies, so I now have one at work and one at home. I finally got around to reading the chapters on annotations, enums, inner classes, and generics.


Getting Things Done: The Art of Stress-Free Productivity by David Allen
— Mr. Allen, where have you been all my life? As he freely admits, a lot of the stuff in the book is common sense, but its the implementation that people get hung up on. I've started writing down everything I need to do, which alone has made me more productive with lower stress. Highly recommended.

The Little Schemer by Daniel P. Friedman and Matthias Felleisen — This series is one of the weirdest. The Socratic style is off-putting to some, including me, at least initially. Some of the text borders on cheesy, but I'm growing my appreciation of whimsy. I still don't understand they Y combinator, so I'm going to need to revisit this in a few months.

Practices of an Agile Developer: Working in the Real World (Pragmatic Programmers) (Jul 1, 2005) by Venkat Subramaniam and Andy Hunt — Another great book in the "Pragmatic" tradition. I was expecting a book with more on agile software development methods, but this is more individual things developers can do to increase their productivity. While I already do a lot of these, it's always good to be reminded of what you're naturally doing and why it works.

After The Gold Rush by Steve McConnell — Recommended for anyone interested how software development can become a true engineering discipline instead of the craft that it currently is in most of the industry.

Waltzing With Bears: Managing Risk on Software Projects by Tom DeMarco and Timothy Lister — From the authors who wrote "Peopleware. While the word "agile" isn't used in the book, a lot of the topics here strongly related to agile practices.

Bargaining for Advantage: Negotiation Strategies for Reasonable People 2nd Edition by G. Richard Shell — An excellent "soft skills" book. I've read "Getting to Yes" and "Getting Past No", and this book was an excellent complement to those two. Lots of concrete strategies for negotiation and guidance for what works in what situations.

Books, December 2008

Recommended:

Effective Java by Joshua BlochTHE book for honing your Java-fu. If you haven't read this, stop reading and go buy it now. There's so much good stuff in this book, I savored it over about three months.

Peopleware: Productive Projects and Teams by Tom DeMarco and Timothy Lister
— A classic in software development management. Much of what Joel Spolsky has used to build Fog Creek and which he writes about is derived from this book. A warning though, if you work in a cube for a large company, you will probably be tempted to quit :)

Groovy in Action by Dierk Koenig, Andrew Glover, Paul King, and Guillaume Laforge — Groovy is a fantastic language, built around the desire to be useful and productive. This is a good introduction from some of the project leads. I would like to have more content on Groovy's metaprogramming capabilities. The MarkupBuilder section of the book was very handy when I was re-implementing a mess of a Java/XSLT report generator, ending up with about half the lines of code and far greater readability.

The Productive Programmer by Neal Ford — Following in the tradition of "The Pragmatic Programmer", this book provides a bevy of ideas of how to improve your programming. The treatment of many of the subjects is relatively shallow, but everything from window launchers to code coverage is somehow hit. Ford also mentions specific software packages various tasks, which takes the great risk of quickly dating the book but makes it much more useful for the reader.

Not Recommended:

Differentiate or Die: Survival in Our Era of Killer Competition by Jack Trout and Steve Rivkin — I was hoping for a book on how to differentiate your product in the marketplace based on creating innovative products, but this is more "how to lie to people about your product when it's just like everybody else's." "It's Toasted" indeed.

Bad UI: Avery DesignPro

Avery is mainly know for their blank printer labels. Many people use Word for creating labels, but Avery also has their own program, Avery DesignPro. I used this last week to print some Christmas card labels, and found an excellent example of poor UI component design.

In these two screenshots, can you look at them and tell which means "print a full sheet of this label" and which means "print a different individual labels"? The relevant part of the UI is the All Same "button".

picture-2-21-34-47

picture-3

The answer is actually neither. The entire area of "All Same"/"On"/"Off" is one big button, rather than a label and a toggle. Clicking changes the value, rather than performing the action that the button indicates. This is very confusing to users, since it's rare that you click an off button to turn something on. A better design would to have two radio buttons with "all labels the same" and "different individual labels", or some such text, so that it would be clear which of these was currently selected and which one you were changing to.

Code coverage is like compiling

Several months ago, I began a concerted effort at work to get our code coverage numbers up. This was prompted by an upper management target of 85% code coverage by a certain date, which I initially saw as unrealistic within any timeframe. I hadn't done much work with code coverage, but I did know the primary drawback, in that most tools simply show that code was executed and not that all paths through the code were executed (branch coverage). Any simple metric has the potential to be abused by naive management, since it's easier to measure code coverage than measure if the code is actually being tested correctly and fulfills the desired usecases (assuming they even exist!).

Several months later, now at 85%, I have a more positive and specific view:

Coverage is to testing as compiling is to coding.

That is, it doesn't ensure that your testing is complete, or correct, or anything, but it does make sure that nothing is completely wrong. If code is never even executed, you have zero assurance that it is correct, just as if code that won't compile has zero assurance of being correct. That doesn't mean it is correct, just that it has a non-zero probability of being correct. The realities of software development allow you to only increase the probably of correctness, so this is one more tool to do this.

I found large blocks of code that weren't being run at all, for various reasons. There were a few methods that intended to override superclass methods (and weren't annotated with @Overrides because the code was originally 1.4 based), but had subtle name or signature typos. Some code had subtle logic errors in branches which prevented one way from executing.

Code coverage metrics were particularly useful in the case I was in, where I had inherited a large amount of complex code from another developer who hadn't provided the most thorough set of tests. I could easily see what code wasn't being executed, and then devise test cases to cover these. One has to be very careful when doing this, since you only get one chance to test that section of code correctly, since after the first test which covers the code, you no longer have the obvious warning of uncovered code. This presents the developer with a moral hazard, since they can write the simple test to get their code coverage numbers up or they can write exhaustive tests which correctly test the code and contribute to genuine code quality. You only get one chance to do the right thing.

Double dereferenced properties in Ant with Groovy

The TestNG report system is nice, but sometimes you need to integrate the TestNG output with some other test reporting system. At work, this other system requires that a single ".suc" or ".dif" file written to a common directory for each "test" you run, where "test" is defined however you want. In our case, we do one test for each TestNG group we run.

Due to Ant's immutable properties, I found it difficult to get exactly behavior I wanted. The simple way was to set a property "has.failure" if the testng-failed.xml file existed, but then would have the effect of appearing that a bunch of tests had failed, rather than a single one. It wasn't that important of an issue, since we it it was easy to see which test had actually failed afterward.

I finally got around to fixing this today. I screwed around with ant tasks for a while, but finally decided to use Groovy, which I'll choose first next time. The main issue was with the double dereference — I wanted to create a property based on the groups the test was running, and then reference it the same way, e.g., create property ${foo}.failed, and then access it like "${${foo}.failed}" (which doesn't work). Ant lets you create this property, but then requires you to jump through some as-yet-unknown-to-me hoops in order to actually reference the property. However, this is simple in Groovy, shown below.

This basically replaces the "condition" tasks in my description of calling the testng task in this post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<target name="run-testng" depends="" >
    ... call testng here
 
       <condition property="${groups}.has.failure" value="true" else="false">
          <available file="${testng.report.dir}/${groups}/testng-failed.xml"/>
       </condition>
 
       <antcall target="process-results">
          <param name="infix" value="${groups}"/>
       </antcall>
</target>
 
<target name="process-results">
       <groovy>
          def infix = properties['infix']
          def dest = properties['RESULTS_DIR']
          def suc = new File("${dest}/product.name.${infix}.suc")
          def dif = new File("${dest}/product.name.${infix}.dif")
          if (properties["${infix}.has.failure"] == "true"){
            dif.write('Pass')
            suc.delete()
          } else {
            suc.write('Pass')
            dif.delete()
          }
    </groovy>
</target>

And, yes, the framework we have requires "Pass" in the dif file– don't ask me.

Java is dead/alive

Funniest thing I read today:

"Autoboxing was a misguided effort to paper over Java’s early decision to have a segregated type system for primitives and objects. It was Java’s Plessy v. Ferguson decision that pretended primitives and objects were separate but equal; but the claim was no more true in Java than it was in American jurisprudence."

I do love a good Supreme Court ruling analogy.

From one of my favorites, Elliotte Rusty Harold, in Java is Dead! Long Live Python!

Proleptic apoplexy

I spent a few hours this week trying to figure out why some date manipulation methods I was writing weren't working. In my test case, I had two instances of GregorianCalendar which I was comparing, an original and one that had been round-tripped through some conversion methods, via oracle.jbo.doman.Timestamp. Using the equals method, they were returning false. I used the toString method to see if there was something obviously different, but the strings were exactly the same. I even when so far as to step though equals in the debugger, until it disappeared into a JDK implementation class for which the source was unavailable. Even stranger, I discovered using compareTo returned 0. I set about printing each property of the instances until I discovered problem: the GregorianChange property.

In the tests, I was using XMLGregorianCalendar to take an ISO 8601 date string (e.g., 2006-09-22T00:00:00.000-00:00) and create a GregorianCalendar. The intention of this was to allow users to create statements like "if Order.timestamp > DateLib.from("2006-09-22T00:00:00.000-00:00"), do something". Below is the reduction of what was happening in the test and application code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import java.util.GregorianCalendar;
import javax.xml.datatype.DatatypeFactory;
 
public class CalTest {
    public static void main(String[] args) throws Exception {
 
        String dtstring = "2006-09-22T00:00:00.000-00:00";
        GregorianCalendar gcFromXmlgc =
            DatatypeFactory.newInstance()
                .newXMLGregorianCalendar(dtstring).toGregorianCalendar();
        GregorianCalendar gcFromCons = new GregorianCalendar();
 
        gcFromCons.setTimeZone(gcFromXmlgc.getTimeZone());
        gcFromCons.setTimeInMillis(gcFromXmlgc.getTimeInMillis());
 
        System.out.println("equals? " + gcFromXmlgc.equals(gcFromCons));
        System.out.println("compareTo? " + gcFromXmlgc.compareTo(gcFromCons));
        System.out.println("XML GC: " + gcFromXmlgc.getGregorianChange());
        System.out.println("new GC: " + gcFromCons.getGregorianChange());
 
    }
}

Running this produces the following output:

equals? false
compareTo? 0
XML GC: Sun Dec 02 08:47:04 PST 292269055
new GC: Thu Oct 04 16:00:00 PST 1582

equals is false, but compareTo is 0? That's curious. The last two lines show why, now explained.

The core of the problem is XMLGregorianCalendar is intended to represent the W3C XML Schema 1.0 date/time datatypes. The date/time type in XML Schema essentially implements ISO 8601, with a few minor exceptions. The important part here is that it represents a "proleptic Gregorian calendar".

The Gregorian calendar is the calendar used exclusively in the West and by most of the global business world. Most people in the West don't even know there's even a name for the calendar, they just know it as "the calendar". The Gregorian calendar was adopted in 1582 to correct for the gradual drift caused by the previous Julian calendar. This change set the nominal date back 10 days and added a more astronomically appropriate leap year rule. The "proleptic" Gregorian calendar is then the Gregorian rules applied to dates before the calendar was adopted, as if the calendar had always been in effect. Therefore, all Western dates after 1582 are according to the Gregorian calendar, but any date before this can be in either the Julian calendar or the proleptic Gregorian calendar, with one always being specified explicitly.

Knowing this, one would then assume that the class GregorianCalendar implemented a proleptic Gregorian calendar. However, belying it's name, it actually represents a hybrid Julian-Gregorian, with each type having a set of rules as to the order of different dates, and the ability to set a date which defines when the rules cutover. The well-known Joda-Time library names these correctly, with a proleptic Gregorian, a postleptic Julian, and a hybrid GregorianJulian. The JDK 1.5 documentation includes this description, but it's not really useful unless you know what the semantics behind it mean. GregorianCalendar.toString() doesn't print it out, so you have to know it exists and to explicitly examine it when debugging. For most code using dates, the cutover date doesn't matter, since most software isn't going to be used for dates in the distant past, and if it is, there's hopefully a domain expert on hand to specify this sort of thing.

A default instance of GregorianCalendar implements a non-proleptic calendar — dates before to 4 October 1582 are in Julian, dates after are Gregorian. However, XMLGregorianCalendar implements a proleptic Gregorian calendar as specified by ISO 8601, so it sets the change date to Long.MIN_VALUE. The Timestamp class uses millis since the epoch (1970), so it will always give us a representation of the same date, but not the same GregorianCalendar. In Sun JDK 1.5, the only difference in equals between Calendar and GregorianCalendar is the change date:

1
2
3
4
5
 public boolean equals(Object obj) {
        return obj instanceof GregorianCalendar &&
        super.equals(obj) &&
            gregorianCutover == ((GregorianCalendar)obj).gregorianCutover;
    }

GregorianCalendar inherits compareTo (implementing Comparable) from Calendar, which simply compares the millis since the epoch. GNU Classpath 0.18 has a different implementation, as it doesn't take into account the change date:

1
2
3
4
5
6
7
  public boolean equals(Object o) {
    if (! (o instanceof GregorianCalendar))
      return false;
 
    GregorianCalendar cal = (GregorianCalendar) o;
    return (cal.getTimeInMillis() == getTimeInMillis());
  }

In writing my tests, I have made a huge error– I was testing for object equality, when I should have been testing for value ordering. I shouldn't have cared if the objects were the same, what I really cared about was that there was no distinguishable ordering between them– that both objects represented the same value.

I didn't actually write the tests from scratch, I had actually translated them from other date tests written in Oracle Business Rules RL, which as semantics similar to Java, but not exactly. The relevant difference here is that you can use the relative operators on Comparable implementors, which then are converted to calls to compareTo at runtime. I made the mistake of regex-replacing a bunch of obj1 != obj2 to !obj1.equals(obj2), when I should have done obj1.compareTo(obj2) != 0. If these had been any of the relative operators, I would have used compareTo, but != took me to !equals. Big mistake.

I learned a lot from this distraction, not just about calendars, but more importantly about equals and compareTo. I recently started reading the new version of Effective Java, so the sections on comparison and equality have new and deeper meaning for me now.