Alex headshot

AlBlue’s Blog

Macs, Modularity and More

Literate functional testing

2006 Test

Robert Chatley and Tom White (from Kizoom) have given a presentation on literate functional testing, which is using literate programming for creating tests. The idea is that you're telling humans what you're trying to do, not explaining to the computer what you're trying to do (apparently a quote from Donald Knuth).

They have created a framework (soon to be at code.google.com, but not at the time of writing) that allows tests to be readable by non-coders and coders alike. The example that they gave was:

goToTheIndexPage();
assertThat(page,has(title("Google"));
clickOn(link("Images"));
enter("kizoom",into(textField());
assertThat(page, has(text("Kizoom summer party"));
assertThat(it,has(image(),withUrlThat(endsWith("summer04.jpg")));

The idea is that you can read the code to explain what happens to not only coders, but also business analysts and people who want to understand what it's saying, rather than how it's implemented. As a neat extension, they had an Eclipse filter that removed the punctuation, so that it ended up as:

goToTheIndexPage
assertThat page has title "Google"
clickOn link "Images"
enter "kizoom" into textField
assertThat page  has text "Kizoom summer party"
assertThat it has image   withUrlThat endsWith "summer04.jpg"

It seems to have a lot of similarities with other literate programming ideas (e.g. AppleScript). However, like AppleScript, only a subset of programmers get how to write it well (and I'm in the category of people who would like to be able to write AppleScript, but doesn't 'get' it completely -- actually, if there are any AppleScript authors reading this blog entry, please write a book on how to script applications like Mail and iCal/iChat; why is it every example is about scripting the Finder? I don't care about the folders, but I do care about manipulating data in my iLife applications).

It will be neat to see what the framework contains, as well as the different terms that are used. Apparently, it's quite extensible (e.g. you can look for things other than Links in the page, although not if there's JavaScript in the page). However, the framework is an extension of the Html/HttpUnit systems; and so, it's quite based on the assumption that you are testing a web based application. The idea is certainly extensible to other types of environment, even if the specific implementation is based on web tests at the moment. Of course, it will be interesting to see what other dynamic languages (e.g. Ruby, Python) can do in this area. The video is now available.