Running headless VMs under Mac OS X

I recently needed to be able to install some software which wasn't available as a pre-compiled binary for Mac OSX, but was for a variety of Linux distributions. As a result, I decided to finally roll out a virtual Linux box running on my Mac OSX box in order to be able to test against a Linux environment as well as an OSX one (which, of course, is already UNIX based).

I'm using a run-of-the-mill Mac Mini, dual-core 2GHz with 4GB of memory (only 3 accessible, I suspect) with 10.5 (no, I've not updated to Snow Leopard yet). Getting an installation of Debian up and running on a virtual environment was actually pretty easy; the following should apply equally well for other (Intel x86) operating systems. It should be noted that since I'm on 10.5, I'm using a 32-bit kernel, along with 32-bit app and guest OS. I don't know how well it would fare if the guest OS and/or OS were not matched.

I downloaded VirtualBox for Intel x86 (it has two separate binaries, one suffixed -x86, and one suffixed -amd64, which has the x86_64 binary type; so I guess it would work on a 64-bit OS). Normally, this is a client-app; installing it gives you /Applications/VirtualBox.app but it's possible to run headless as well.

Inside /Applications/VirtualBox.app/Contents/MacOS there's a plethora of executables; but the two to look out for are VBoxManage (for creating and looking after VMs) and VBoxHeadless (for running them). I created a daemon user to run the VM images, and set that up in the PATH for the login, but you could run them from that directory as well if you want.

Setting up the disks. In order to have a VM, I needed a (virtual) disk image. VBoxManage createhd --filename Debian.hd --size 10000 creates you a file (in ~/Library/VirtualBox/HardDisks) which can be subsequently mounted. For reasons best left to random conjecture, once you've created it, you have to then do VBoxManage openmedium disk Debian.hd before you can do anything useful with the image.

Getting the Boot CD. Most Linux distros come with a bootable CD (or DVD) with everything you could want on it. Typically, they'll end in .iso; you might also see .cdr, but it's pretty much the same thing. You can mount this as a file (no need to burn it unless you feel like it) by doing: VBoxManage openmedium dvd /path/to/linux.iso. In the case of Debian, I used the NetInst CD although there's no reason necessarily to use that instead of the larger ones. If you end up defining networking, it'll pull the rest down as needed anyway. If you're not doing networking, then you might want to download one of the larger CD or DVD images; but the netinst is preferrable, because it will go out of date less quickly than the others.

Creating the VM. Once we've got the (virtual) media together, we're ready to go onto our (virtual) box. VBoxManage createvm --name Debian --ostype Debian --register will give you a new VM defined with the name “Debian”. The OS type Debian is one of the IDs in VBoxManage list ostypes, though in reality, one distro is pretty much like another as far as virtualisation is concerned (I suspect it's got more to do with installing the host extras, which I largely suspect are unnecessary for a headless/server box).

Tweaking the VM. Once you've got an empty box, we'll need to configure it to tell it where the hard disks are and what other parameters to use. There's a wealth of data in the user doc (in the VirtualBox.app/MacOSX folder, of all places...) but we do need to get a few things configured. VBoxManage modifyvm Debian --memory 512 --acpi on --vram 128 --hda Debian.hd --dvd /path/to/linux.iso --nic1 bridged --bridgeadapter1 "en0: Ethernet" --macaddress1 0123456789.

Quite a mouthful, and some parts should be obvious. The networking setup defines adapter 1 to be bridged (i.e. this VM is visible to the outside world) and based upon the ethernet connection "en0", which is the wired connection on the box. For some reason, specifying "en0" didn't work, but "en0: Ethernet" did. I suspect a regular expression is looking for the colon, and complaining if it's not there; the stuff to the right I believe to be a comment. Incidentally, you can specify a Mac address (otherwise it changes each time) which will help with those using DHCP to map to a fixed network address. For reasons best left to Sun, the second digit in the mac address must be even (0, 2 etc.) - if it's odd, you'll get "ERROR: Invalid MAC address format". Obviously.

Getting ready to go. So, we can turn on our machine now, by running VBoxHeadless -s Debian. However, this will boot from the DVD, and we can't see what's going on. VirtualBox has a nifty remote desktop featurette which gives a remote viewing experience. Unfortunately, it's based on Microsoft's RDP, but that's probably a commercially sensible plan anyway. You can download a Microsoft RDC client for Mac (the page is pretty poor; the 'Downloads' section is hidden in a pane in the bottom middle; and when you click that link, it populates a pane on the right which you have to click again to get going).

To run with a remote connection, kill (Ctrl+C) the running box and start again with VBoxHeadless -s Debian -v on. This will give you a port 3389 remote desktop connection (the default) and you can simply put the name of the host OS in the remote desktop client. (If you have port collisions, you can specify -p 1234 and then connect to hostos:1234 from remote desktop).

Now that you've booted from the DVD, you can go through the joyous steps of installing your favourite Linux flavour.

Time passes. ...

Once you're ready to finish the installation, you need to modify the VM to get rid of the DVD. Simply run VBoxManaage modifyvm --dvd none to eject the DVD (I did this after having stopped the install) and then start again.

Once you're convinced that everything is running, you can stop the RDP with VBoxHeadless -s Debian -v off and continue to log in remotely via SSH or whatever flavour of remote access you're going to use.

Summary. Installing is easy, configuration takes a bit of work. If you prefer, you can set up the image locally on a machine tow which you have a GUI, and then copy the settings over to the remote host. And don't forget to eject the installation CD when you reboot to prevent going round the loop again.

Here's a summary of the steps:

• Downloads
  • Download VirtualBox.
  • Download Debian NetInst CD or other distribution to taste
  • Download Microsoft Remote Desktop Client for Mac
• Setup
  • VBoxManage createhd --filename Debian.hd --size 10000
  • VBoxManage openmedium disk Debian.hd
  • VBoxManage openmedium dvd /path/to/linux.iso
  • VBoxManage createvm --name Debian --ostype Debian --register
  • VBoxManage modifyvm Debian --memory 512 --acpi on --vram 128 --hda Debian.hd --nic1 bridged --bridgeadapter1 "en0: Ethernet" --macaddress1 123456789012 --dvd /path/to/linux.iso
• Running
  • VBoxHeadless -s Debian -v on -p 1234
  • VBoxHeadless -s Debian -v off

Modular Java series at InfoQ

InfoQ have published my first article in a series on Modular Java. In this article, I look at what modularity means, and why there's a difference between the compile-time classpath and the run-time classpath, and why it's an important concept for not only large complex systems but also for library providers as well. Lastly, I take a quick look at the current state of modularity in Java.

OSGi 4.2 released

The OSGi 4.2 specs have been released, and they bring with it some goodies. It looks like Felix is well on their way to officially passing the TCK, and I doubt that Equinox can be far behind.

I've written up a piece on InfoQ (and a question on StackOverflow) to cover what's new and noteworthy for this release.

There's no Snow Leopard here.

So, I'm still on 10.5. Why, I hear you ask, when the world has gone Snow Leopard crazy? Should I worry?

Well, generally not. Of course, it doesn't help that Apple bungled Snow Leopard with an insecure version of Flash with known exploits, but let's face it, every Apple 10.x.0 product has had its own subtle set of flaws in their own way. One lets the great unwashed try out the .0 (and .1) releases first, and then after .2 comes out (and importantly, the retail discs are re-cut to have a default version of 10.x.2 on them) is it worth buying.

But even in this case, I'm still not getting Snow Leopard. How come? Well, let's count the ways in which Snow Leopard will improve my life:

OK, let's now count the ways in which it will impact it:

1. I will have to buy a new PowerMac (sorry, MacPro) because the G5 isn't 64-bit enough for Apple's Intel fascination. This is despite the fact that the hardware still has a good few years of life left in it, and the fact that since it has an attached ADC monitor, replacing the PowerMac will also require the purchase of some new displays to go along with it. That's a lot of moolah for a point upgrade.
2. There's no ZFS support. Yes, I've written about ZFS many times before on here (including packaging up a decent installer) but that's gone the way of the Firewire. Yes, that came back after Apple relented and provided a feature that everyone wanted (much like the much-derided missing Matte screen option). So ZFS may turn up again in 10.7, in which case I'll have saved the purchase price of 10.6 in the meantime. Given that I use ZFS exclusively for data (both on my disk storage array and on my laptop, where it provides me with a compression ratio of 1.36x as well as providing block-level integrity, thankyou very much), not being able to access that would be a major downside.
3. There's still likely to be an app market for the 10.5 stream for some time. Apple's apps may always require the latest and greatest, and as completely new versions of apps may get released, there's very likely to be a combined Universal and 10.5 compatible version of the program. Apple's pricing for the 10.6 update is expected to get as many people writing for 10.6+ only, so that when 10.7 comes on stream, anything less than 10.5 can be written off as a statistical anomaly. That gives me a year (or two) of breathing space in the meantime.
4. Who needs 64-bits, anyway? Not for memory, at least on laptops and small devices; my G5 has less memory than my laptop does. Yes, more efficient, registers etc. but frankly, it's better to access data than to not be able to access data faster.
5. Security features are veritably worthless. Apple's gone overboard on writing up the changes, like address space randomisation, but the start up of the dynamic linker is in exactly the same place as it always was. Can't find that return-to-libc exploit in memory? Just load libc via the dynamic linker, and away you go. It's a bit like saying that you've put shutters up over every window with high-tech frikkin' lasers over everything, but neglected to actually close the front door on the way out.
6. My wife thinks I spend too much on techy stuff as it is.

What 10.6 is is a developer release. It's like a massive-scale beta early-access test of the next version of 10.7. Instead of it being available to just a handful of developers who pay through the orifice to get their mitts on the latest and greatest. Well, 10.6 is the latest and greatest, and it'll only set you back a few quid. And it's got all sorts of cool technologies (yes, GCD is cool, but NSOperation does much the same thing – at the high level for Cocoa in 10.5, and in 10.6, it's based on GCD anyway). But most of those won't be immediately visible to the user of the system. Granted, it's got a few benefits for the kind of mass data crunching apps (the Photoshops of the world, which in any case are still encased in Carbonite) – but in reality, a lot of subtle touches make it feel slightly snappier. But that doesn't make Leopard any slower, and in any case, the only time I ever shutdown and restart is when I'm installing a system update, frankly, which is once every month or two.

Meanwhile, rabid fanatics like RoughlyDrafted are giving even hardened fanbois the chance to cringe in embarrassment with statements like "As jingle-pundits desperately try to denigrate Snow Leopard as a Service Pack ... " when in reality, it's not even that. There may be a lot of foundation shifting under the hood (and 10.6 is likely to be the last 32-bit release for a Mac OS X computer; though hardware devices like the Apple TV and iPhone are likely to continue with 32-bits for a while) which will set up for a great 10.7. And by the time that comes, my dual-core G5 will probably have reached the end of its useful life in any case and I'll bite the bullet to upgrade to a shiny-new 16-core system in 2011. By then, it will either have ZFS or the equivalent Apple invention (though HFS+++ should really die a horrible death, and hopefully pretty soon) and I'll be able to continue on as before.

Draft Scala modularity requirements posted

I've just posted a set of draft Scala modularity requirements to the scla-lang-internals mailing list. If you're interested in Scala, Maven, OSGi and modularity in general, please read through and send comments to the scala-internals mailing list.

Signed Updates for Mac OS X

This is something that I hadn't seen before. I'm not sure whether it's new, or whether I've not noticed it, but updates from Apple are signed with Apple's public key. At the top right of the installer image, there's a small icon that (on close inspection) appears to be a mini 'certificate' icon. Clicking on it brings a drop-down sheet which confirms that the key is valid, and when it was issued. The ability to digitally sign (and therefore verify) updates should have been done long ago, but it's good to see that the automatic process now shows this. However, the question of whether this is almost utterly invisible, combined with the willingness for people to download-and-install anything on the computer, means that in practice whether it's signed or not is probably a moot point for most people. Now, if we knew whether the softwareupdate would refuse to install anything that wasn't signed, or wasn't signed with an appropriate Apple key, then we might have some semblance of security.

Java Security update for Mac

Apple has released Java 10.5 Update 5. There's no corresponding Snow Leopard release; the buggy versions of Java are 1.6.0_13 and 1.5.0_19; the update takes it to 1.6.0_15 and 1.5.0_20, which Snow Leopard may already have.

This can be accessed by using Software Update from the command line or the GUI. Since these include security updates which are remotely exploitable, updating should be of paramount importance.

There's no news yet about any updates for 10.4 – this has the same vulnerability though. The updater doesn't appear to have any restrictions on the version of the OS, and the URL is JavaForMacOSX10.5Update5.pkg, so if you've got a 10.4 era Mac, you might want to give that a download and see if it'll install on your Mac or not.