Mac doesn't get much love

Recently, Doug posted that Mac Gets No Love on the Eclipse platform – and rather than replying as a comment, it makes sense to follow up with a separate posting.

I first got into helping Eclipse back in 2002 when the first port of the Mac application was being readied for Eclipse 2.1 on Mac. (I mentioned at the time that using Cocoa rather than Carbon was the way forward; it took another five or so years for that advice to be heeded :-) Although I'd been using Eclipse since the 1.0 days (having worked for IBM on Visual Age for Java, and subsequently its evolution into Studio), it was the migration to Mac that brought me into the Eclipse community as opposed to a silent user. However, I'd been using Macs since the OSX transition, and before that, Nextstep; so I wanted to bring that to the table. (This was in the days before OSX was trendy; by comparison, the first iPod had launched a year before.)

Before going on largely to agree with Doug, I do want to call out specific mention to the SWT folks who have ported their widget set to Mac (twice, now) and without whom we wouldn't have an Eclipse platform at all. Many notable people have worked on the Mac port, including the current Scott Kovatch (first at Apple, then at Adobe, now at IBM), but also past luminaries such as Kevin Barnes and Andre Weinand who did a lot of the original porting. There are specific things which make an Eclipse app feel more Mac-like, including putting Preferences under the application menu, but most of these are SWT team fixes.

However, the problem is that outside of the SWT team, there is virtually no Mac knowledge, or until recently, user base. No-where is this more visible than the release engineering and build projects; what we have out of the box is a project laid out specifically for Windows users. And until you've used Macs for a while, it's not even clear that there is a problem; after all, if it works in Windows, it must be OK for everyone else, right?

Over four years ago, I blogged about 10 things I hate about Eclipse, including the fact that it's not a proper Mac layout. Much like Unix has configuration in /etc and system binaries in /bin, Mac applications have a specific layout as well. Furthermore, there's even an eclipse executable at the top level because the releng build fails if there isn't a file there with that name; despite this not being used by the Eclipse.app itself (it has the copy tucked safely inside it). It also means that you can't drag-and-drop a Mac install into any ~/Applications directory, because the features and plugins directories are in the wrong place. I even started Maclipse as a way of better wrapping up the Mac platform (DMG, files in the right place, drag-and-drop install) to try and show how these problems could be addressed – but there was no interest in making it happen at Eclipse. I seem to recall the conversation trying to get it to work was “it works this way on all other platforms; we're not doing anything different for the Mac” though that was in the days when Macs hadn't got the popularity surge that they have today.

There's also a lot of issues caused by decisions of downstream teams which may look OK on their platform, but not on Macs. Mylyn's use of CCombo boxes, for example, makes the UI on a Mac look ugly when the CCombo offers no functionality that the Combo does on the Mac platform. Heck, it's even been fingered as a memory leak and yet nothing is still done about it. Here's what it looks like on Windows versus Mac OSX:

Then there's little things, like the warning message shown in JDT's “sort members” dialog. Because it uses Label.setEnabled(false) rather than Label.setVisible(false), the text is completely missing on the dialog but with a giant blank space on Eclipse 3.4. (I not only provided the code fix for that but also argued that using the enabled state wasn't right for all platforms; or even that the Mac platform should have a special-case. But that was overruled by a non-Mac user.) Since then, the Eclipse 3.5 SWT on Cocoa now ignores the Label.setEnabled() call, which is why it now shows up on Eclipse 3.5 onwards. (The issue may still be present on the Carbon builds.) Here's what it looks like on Windows and Mac OSX:

Another example: when creating an Eclipse workspace, if you put ~/Projects into the workspace selection dialog, rather than creating them in ~/Projects as you might expect, the workspace is actually created in /Users/alex/Applications/Eclipse-3.6M5/Eclipse.app/Contents/MacOS/~/foo. The use of ~ to denote the user's home directory (and it's available under user.home) is a widely used Unix extension; so the fact that it's not supported is a minor annoyance. And yes, I raised a bug ages ago.

Finally, many Mac applications have the ability to install additional plugins by dragging-and-dropping them into an appropriate folder; typically ~/Library/Application Support/Eclipse/PlugIns. In fact, Mac applications generally have three levels of location to search for installing such items:

1. A ~/Library/Application Support/* directory
2. A /Library/Application Support/* directory
3. The application bundle (i.e. inside the .app itself)

(Technically there's a fourth; /Network/Library/Application Support/*, but this is rarely searched on application launch due to potential delays with the network; and in fact, these days, is rarely used at all.)

Wouldn't it be nice if one could create an Eclipse (or OSGi) runtime which would allow a user to install components by dragging-and-dropping the bundles into the right directory? After all, this works for the majority of other applications (see Doug's scripts for iTunes for a lot that work with iTunes; you can also extend Mail.app with Hawk Wings plugins; even Contacts can launch Google Maps).

However, this will never happen. P2, the bane of many, has a dropins directory which is almost exactly right for this job – except that it's a singleton. You can't have more than one dropins directory. (Well, OK, you can have two; a dropins directory somewhere else, and the one that's in the eclipse folder.) This enhancement has been categorically ruled out from ever appearing in Eclipse P2:

Installing by just adding a folder was just plain wrong and resulted in many problems in UM at the time because ppl would break themselves and then complain. I'm against reintroducing something like that and so are the other p2 committers, which is why I closed the original bug as WONTFIX. p2 provides bundle pooling which provide a much better way to do this.

... except this has nothing to do with bundle pooling. It's about making life easy for those that want to drag-and-drop install. Which is pretty weird, because the dropins folder already allows you to drop and install in one place; so why not several? Sadly, this kind of attitude is prevalent when it comes to improvements to the Eclipse platform for Mac. But hey, building a decent Mac app bundle keeps getting raised periodically.

So, I think it's important to praise the work done by the SWT team, and in particular, the improvements with the Cocoa build; but there's a lot of difference between “uses native widgets” and “is a proper Mac application”. The SWT team is doing great work on the former; but the latter requires greater understanding from the way that projects use those components.

More insanity with 331

After yesterday's piece on the collateral damage in 331, a number of other impacted applications and technologies have surfaced.

• Epic's Unreal engine which uses Lua script via LScript in its game play engine
• SQLLite or any kind of SQL query, though presumably CoreData as a compatibility layer is allowed
• OpenCL since this is not C, Objective-C, C++ or JavaScript
• Apple's PLists since these are not C, Objective-C, C++ or JavaScript
• Interface Builder's XIBs since these are XML representations of how to instantiate an object that isn't C, Objective-C, C++ or JavaScript
• Any game by Sketch Nation, because despite being very cool indeed, the generated games are not games written in C, Objective-C, C++ or JavaScript
• Game engines like Cocos2D, Irrlicht, OolongEngine and Sio2Engine since these all were ported from other platforms and so would be treated as a compatibility layer

For completeness, here's yesterday's list as well:

• TouchXML since it's a compatibility layer for NSXML
• Unity As a 3D tool, this is used on multiple platforms and can accelerate game development. However, this is a “compatibility layer or tool” which is verboten
• XML or JSON Any code that uses either XML or JSON as an intermediate format is guilty of writing an application which is not “originally written in Objective-C, C, or C++”
• 3D graphics tools to export models in generated source
• Parsers by Bison/Yacc DSLs are verboten
• Macros Since any turing-complete language can be transformed into any other turing-complete langauge, macros can't be used either

Collateral Damage in 331

This post is not about whether banning Flash applications on the iPhone is desirable or ethical. As a long-term Objective-C developer, I have no issues with the lack of Flash (transliterated or otherwise) on the iPhone. However, as I noted yesterday the change in the terms is very concerning. As a reminder, the new clause states:

Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

Like a chemotherapy, this targets far more than the cancer it is intended to take out and has wide-reaching implications, for iPhone development in general. For example, the statement “must be originally written in Objective-C, C, C++ or JavaScript” leads to many vague interpretations, the strictest of which prevents nearly all development. Quite apart from Objective-C++ being missed from that list (how else do you get C++ code onto an iPhone?), what does “originally written” actually mean? Does it imply that sketches on an iPhone or iPad sized notepaper disqualify an application? What about pseudocode written in python or ruby to test out a game's AI before being written in Objective-C? The term “originally written” is so vague you could drive a bus through the loopholes this can cause.

The second part ... “only code written in C, C++ and Objective-C may compile directly an link against the Documented APIs” seems entirely orthogonal to the next part “e.g. Applications that link to Documented APIs through an intermediary translation or compatibility layer to tool are prohibited”. As an additional example, is this covered by the legal agreement? Normally, the legalese is more strict than this (including, but not limited to...) when citing examplar information. Furthermore, the first part doesn't prevent indirect linking, though the example stated clearly does desire indirect linking to be captured.

Here's what would be outlawed for future iPhone development with this new SDK agreement in place:

• TouchXML As a wrapper for XML processing, including NSXML-like functionality, which links to the Documented APIs. Any application using that is violating the example of “compatibility layer or tool”
• Unity As a 3D tool, this is used on multiple platforms and can accelerate game development. However, this is a “compatibility layer or tool” which is verboten
• XML or JSON Any code that uses either XML or JSON as an intermediate format is guilty of writing an application which is not “originally written in Objective-C, C, or C++” as all of these are not languages supported. One could argue the edge case of JSON being a subset of JavaScript (it is JavaScript Object Notation after all) – but you then fall foul of “as executed by the iPhone OS WebKit engine” which the majority of JSON replies don't. Oh, and goodbye every Twitter, News reader, Buzz client out there.
• 3D graphics tools Many graphics developers will use a 3rd party tool to do 3D graphics; in the open-source world, Blender is the tool of choice. You can even use Blender to export C code for the models – but goodbye being able to do that, as the generated code is not “originally written in” the specified languages; but rather generated by a “compatibility layer or tool”
• Parsers by Bison/Yacc Sometimes a custom DSL is used to represent state (though these days, mostly the JSON forms are used). However, this is a “compatibility layer or tool” and is verboten under the new guidelines. Goodbye DSLs.
• Macros The Objective-C language syntax is described without reference to the pre-processor. Any program that uses Macros is going against the spirit of the Objective-C language, and is not found in any of the standard texts on the language. Does this mean macros are verboten too?

The ultimate irony, of course, is that Objective-C started life out as a set of pre-processor statements. The language was built around the concept of macros and macro expansion. That's how DSLs grow – by exploiting the power of preprocessors to do things that the original language designer didn't anticipate, including the growth of an entirely new language. If I use an entirely macro-based DSL to generate Objective-C code, am I still writing in Objective-C? Arguably, the compiler will see the same thing, regardless of whether I code it in full or code it with macro definitions. Yet, at some point, the balance of “originally written in” tips the other way.

Too much of this debate has centred around the extreme edge cases – from “Go screw yourself, Apple” to “And so, Adobe, through a mix of incompetence, belligerence and emotionalist hypocrisy...” without actually focussing on the real issue, which is how much of a negative impact of this will cause both from the PR and also the ramifications of the Enterprise aspect.

Although it's been claimed that these apply to the Apple app store alone, no business in their right minds is going to agree to a set of terms “which Apple might not enforce against Enterprise users”. An agreement is an agreement, that both parties have to hold to. You can't just ignore parts of it on a vague hope that they're not going to be enforced. This is particularly true in the case of software licensing, where large fines have been levied on companies who use unlicensed software. Even if there are companies willing to abide by these terms and sell via the AppStore, the damage to the Enterprise market has already been done.

Update: see the follow up post for more insanity in 331

New blog domain

I've recently moved over my blog to a custom domain name, so if you come via alblue.blogspot.com then you should get redirected to alblue.bandlem.com instead. The feeds continue to work as before, so you shouldn't notice an interruption in service – but because there's a new domain name, they're showing up in searches as fresh posts. Fortunately, the post ID is the same (remember: cool IDs in Atom feeds don't contain the domain name; they're immutable once created) so if you're using a compliant Atom feed browser (or aggregator) you shouldn't see a dump of new posts.

iPhone OS 4.0 roundup

Yesterday, Apple announced the upcoming features in iPhone OS 4.0 (the keynote is available, and also in the iTunes podcast). Almost all of the features were well received, though it's worth highlighting what many already have in Section 3.3.1, as Daring Fireball notes:

Prior to today’s release of the iPhone OS 4 SDK, section 3.3.1 of the iPhone Developer Program License Agreement read, in its entirety:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs.

In the new version of the iPhone Developer Program License Agreement released by Apple today (and which developers must agree to before downloading the 4.0 SDK beta), section 3.3.1 now reads:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

Whilst it's clear they're taking a pot-shot at Adobe's Flash generator, the fact is that the answer is not so clear cut. For example, what happens if you have an external tool that generates data to be uploaded into a program, like Blender? Jeff LaMarche has a good tutorial on importing blender models into OpenGL – is that now disallowed? And what about libraries that extend Apple's limited behaviour in either the XML parsing (like TouchXML) or Networking (MobileTwitterific) – are those disallowed too, since they don't directly link to the underlying libraries?

Or, what if someone were to generate an SWT like binder on top of the UI, wrapping existing components? The wrappers might well be written in Objective-C, satisfying that condition – but the condition isn't transitive, so code that calls those wrappers doesn't need to be.

Finally, what if I develop a series of macros to make life easier? The language isn't strictly Objective-C any more – you could write a whole program that way – but still be compilable by an Objective-C compiler. Is that still valid?

New APIs

The coverage of what's coming is already in various blog posts and on the iPhone OS keynote. However, it looks like the biggest change is the addition of the various background APIs. Instead of allowing general background processing, Apple has created a number of highly specific background runners. For example, you can now do:

• Background music playing outside of the app – though whether the data must be fully downloaded or permits access by ongoing streaming isn't clear
• VoIP calls in the background, including what looks like the beginnings of an iChat API which would permit continued logins whilst external to the app itself
• Finishing background tasks once the app has quit
• State persistence to restore where you last were when the app is opened
• iAds will be a great (and consistent) way of generating ongoing revenue for freemium apps
• iBooks coming to the iPhone is nice, but it'll give you iStrain
• Folders seems like an enhancement that's been waiting to happen for ages
• Background updates for location changes with different levels of accuracy
• Wireless distribution/install of iPhone apps for enterprises

Conspicuously absent were the items relating to new iPhone hardware (though there appears that there's software support for hardware camera flash in the future), the ability to wirelessly sync, or being able to use the iPhone as a wireless disk (and/or access app data from a remote computer). Given the lack of iDisk and wireless syncing so far, it's unlikely that they'll be in with the iPhone 4.x stream; it seems that the mobile department are keeping up with the current stuff rather than features that have been missing since the start.

All in, the iPhone 4.0 update should be a great step forwards for those with iPhone 3Gs devices (or newer), and those with iPhone 3G will be able to use some of the APIs (but no background execution). Other than missing out on Your Favourite Feature Here, the only stain on Apple's card is their change to 3.3.1; is that here to stay, and will it be enforcable?