Alex headshot

AlBlue’s Blog

Macs, Modularity and More

QCon Day 3

2011 Infoq Qcon

And with the final day bringing this year's QCon London to a close, it's time to wrap up another set of presentations and sign off for this series of blog write-ups. Thanks to all my new followers on twitter!

Keynote

The keynote for the final day was Rod Johnson, founder of SpringSource, on the trials and tribulations on starting a company and some of the challenges that they faced. The challenge that I faced was getting out of bed in time to make it in for the keynote, which I failed badly at; but, like Patrick said yesterday, failing fast is always an option. I hear from others that it was vaguely interesting, but since it was a historical look back of what had happened, there weren't any major technical points to take away; though one item I did hear was that they invested too fast and hard in the DM Server and Roo; both of those still seem to be going on comfortably (DM Server became Virgo at Eclipse). Anyway, back to our regular programming

Single Page Apps

Single Page Apps was Michael Mahemoff's talk on the new features and functionality available in HTML5, and in particular, the HTML5 history API and how it affects hashbang URLs.

The problem with hashbang ajax crawling (first proposed on google webmaster blog) is that it changes the URL to effectively be a global single reference e.g. twitter.com/#!/alblue. When bookmarked, the basic twitter.com page will be accessed which won't have any specific reference to specific pages (including any Google crawlers).

The HTML5 history API can help here; using history.pushState() you can push a triple of state,title,path onto the stack. The browser's URL path will be replaced with path, and it will look like a normal page transition – except that the page won't be refreshed as a whole.

Changing the URL is only one part of the situation, though. The app will also have to recover from URL changes. When the URL is changed (whether back/forward button or manual intervention) there will be a window.onpopstate change. This will return the memento associated with it from the history before (or null if the user has typed in the URL manually), which the AJAX application can parse and generate the appropriate data for the call. The example cited was http://rampage.mahemoff.com, which is a pure AJAX application which derives all its data from a single JSON file: http://rampage.mahemoff.com/monsters.json. Interestingly (and somewhat orthogonally to the talk) it was using the mustache templating library, so-called because of the { and } used to delineate the template values. Navigating between pages appears to be a normal website, but you'll note that the top of the page remains the same for all instead of requiring a page refresh, also given away with the cross-fades between images.

A couple of other “just because you can” links - marquee url and url hunter.

Node.js: Asynchronous IO

Stefan Tilkov covered the basic premise behind Node.js as a server-side network server framework (“with a DSL that looks like JavaScript”). By using asynchronous IO – in other words, by never blocking on an IO operation – it's possible to get high throughput whilst consuming minimal memory.

The key is to have one (or a small number) of threads servicing requests from a network source, but only servicing those that are ready instead of having one thread blocked per waiting client. Most operating systems have some variant of select (or in OSX's case, the dispatch_async can be used for any tasks) which allows a function to be triggered if any data is available. Provided that this function creates additional selects in the future (instead of performing blocking IO) then the state of the connection can be maintained independently of the thread servicing it.

Node.js is a set of technologies, such as the V8 JavaScript engine (used inside Google Chrome) as well as libev, libeio, http_parser with some C++ wrapping to provide a JavaScript layer which can invoke callbacks registered by JavaScript programs. When data is available, a JavaScript callback will be invoked using inversion of control; by doing this, a Node.js server is able to sustain a high load in relatively few lines of code.

Stefan has posted some node-samples on GitHub which demonstrate some of the ideas behind the framework. Many additional modules exist, including those to interact with asynchronous requests to MySQL and Postgres databases.

To facilitate ordered operations (where one must complete asynchronously before the next starts) a general step function can be used (yes, like a Monad) to make it easier to chain successive operations together. Since these will all be called back asynchronously, it is possible to parallelise over a data set and have the data members called back with individual elements from the arrays, and then piped into the next farm of functions.

Combined with decent http support, where it's possible to write a (non-caching) HTTP proxy in ten lines of code, and chunking support for streaming back data, Node.js looks like a high performant mechanism of writing services. Note that the key benefits here are twofold:

  • Programs are written in JavaScript, which makes it easy to prototype/iterate/test
  • Using asynchronous behaviour can be an efficient way of dealing with high load

(It's worth noting that Jetty supports asynchronous continuations in a Java runtime and has many of the advantages that Node.js has. The advantage is in the asynchronous nature of calls rather than the implementation language.)

For aged developers such as myself, it may be tempting to write off JavaScript as “that toy language you first saw in a web browser”. For a number of developers, especially with the mobile web, JavaScript is the first production language that they get used to. Those with longer memories may recall similar disparaging remarks being made about Java in its early days; yet with investment in JITs and runtimes, it has become faster than C. So don't write off a language because of its past, but look to its future.

HTML5 @ Facebook

Facebook do a lot of work in HTML5; for some older browsers, certain features (like Chat) are not available. They have a Using HTML5 today post which explains where it is currently being used.

Some problems exist – like the lack of a video codec standard – but libraries such as VideoJS allow fallback to Flash if standard video support is not available.

For implementing games, you typically need to be able to display 5 sprites at 30fps for a board game, 25 sprites for classic arcade games, 50 for shoot-em-ups and 100 for higher performance 2d/2.5d games. But there's different ways to load sprites, such as divs with background images, divs with foreground images, individual files, sprite maps and so on. To investigate what the performance characteristics of each type of mechanism are, they have a JS Game Bench to see what the effect of each different type of mechanism is.

WebGL, if supported, provides an order-of-magnitude difference for game performance. Some cross-platform APIs – like PhoneGap are available, but there's a list of devices and what they support as well listed via Wurfl and other data.

Secure Distributed Programming on EcmaScript 5

Mark Miller talked about the theoretical and implementation details of SecureEcmaScript. The problem being solved is a combination of adding modularity as well as revocation into the EcmaScript language.

To achieve this, a generic proxy to wrap all interaction between two objects. In normal operation, a function is forwarded onto the wrapped object; but a disconnect (or delete) method releases the contained proxied object and subsequent calls become a no-op.

To implement a module system, not only are public objects wrapped, but also all arguments passed in and return results passed out. Subsequently, any connections the module may have had can be terminated externally, and thus allow the module to be unloaded.

The SecureEcmaScript also provides wrappers around built-in functions (like eval) and replaces them with safe implementations. Since this can be done by running an initialisation script, any subsequent uses of eval can be guarded.

With a suitable module system, and safe eval and other core operations, it's possible to view distributed processes in the same way as disconnected modules.

Finally, the talk introduced futures in EcmaScript or 'promises', which were used in the implementation of the distributed system, including a couple of examples in implementing an infinite queue as well as a mechanism to uses these futures as a means of performing a shared bank transfer.

Whilst proxies are in the latest FireFox 4 Beta, and will be part of a future EcmaScript specification, some of the other parts are not yet production ready and remains an ongoing research project.

HTML5 and the dawn of Rich Mobile Web Applications

James Pearce gave an excellent overview of the state of HTML5, reinforcing a lot of the earlier content. Although the iPhone kickstarted a new generation in 2007, other devices are now in play with an estimated 35% iPhone, 35% Android and 35% RIM in the United States (with Nokia having a greater showing in the UK). Although the initial strategy of building per-device specific applications seemed good at first, the reality is that building a new app for each new type of device is a difficult process.

These days, web applications can be just as good as native applications (with some caveats) for interaction with a user. Just as desktop applications have been superseded in some cases by web based applications, so will some native applications be replaced by web applications on local devices. The web is evolving:

  • Documents becoming Applications
  • Declarative HTML becoming Programmatic DOM
  • Templates becoming APIs
  • URLs becoming Arguments
  • Request/Response becoming Data Synchronisation
  • Thin Client becoming Thick Client

As an example, TravelMate is an app which might be mistaken at first glance for a native app, but it's implemented in HTML, JavaScript and CSS. Not only that, but it can be cached locally – although the data it uses needs to be acquired from a network connection, it can cache previously looked up phrases for later use.

Frameworks take the pain out of mobile devices, such as SproutCore and SproutCore Touch, Sencha and tools like PhoneGap can help make cross-platform mobile devices. Knowledge bases such as http://caniuse.com, http://modernizer.com and http://deviceatlas.com can give information about what a device's HTML implementation supports.

Wrapup

QCon is always a blast. As with any conference, it's not just the content of the presentations (which is great) but also the people that you meet, and the technologies that you're exposed to. The only problem is you feel you don't get to clone yourself three or four times to go and see everything; but even if you didn't manage to see everything you want, a fair number of the presentations are recorded and are released over the year at www.infoq.com for subsequent viewing.

The key takeaways for me this year were:

  • HTML5 and mobile web apps are already here
  • Keeping things simple in order to fail fast saves money long term; but that doesn't mean keep it like that forever
  • Separating clients out from data acquisition services over REST (or similar) API means you can innovate in both clients and services

I hope you had a good time following my blogs and my tweets; feel free to come back here some day, or read my content on http://www.infoq.com/author/Alex-Blewitt.