Alex headshot

AlBlue’s Blog

Macs, Modularity and More

Migrating from AFP to NFS (there and back again)

2006, crap, mac, rant

Some time ago, I migrated my network home directories from NFS to AFP. This co-incided with the switch to Mac OS X Server 10.4 (from Mac OS X Server 10.3). I mistakenly believed that using the Apple Filing Protocol would be a good idea, since I could use LDAP to specify where the home directories were mounted from. I also (foolishly) thought that using AFP in conjunction with kerberos would result in a more secure mounting infrastructure.

It's been a baptisim of fire, and I can heartily recommend anyone setting up a new Mac OS X server system that they *do not* use AFP for any mounting processes. It's just unbelievably problematic, and even with the latest 'n' greatest patches (the 10.4.7 patch came out recently, once again, with patches for 'afp file sharing' ...)

Here are just some of the many problems with AFP automounted network home directories:

  • Can't use a local mount. Yup, that's right. If you try and mount a share that's coming from the same computer, you get an error. So, sometimes afp://server/share will work; on the other hand, if you're logged into server, then it won't. This seems to be unique in any network mounted protocol -- samba, nfs, afs (not afp), webdav -- all of those work mounted locally or remotely. Why the AFP server/client can't get into some negotiation to transparently map the calls, I don't know -- it's not like the automounter doesn't make heavy use of symbolic links anyway.

  • Craptacular ways of specifying mounts for users. So this is a knock-on effect to the previous. Because an Apple client needs to know whether it's on the local or remote machine, you have to specify two URLs; one for remote access, and one for local access. So when I log on, it either uses afp://server/Users/me or /Users/me, both of which are stored in my LDAP/Kerberos record. In fact, you specify it in (get this) three parts; the top-level share afp://server/Users, the subsequent path me and the local path /Users/me. This is definitely crap in the Mac OS X server administration utility, but it's crap because AFP is crap.

  • Unbelievably craptactular way of mounting the frigging point. When an Apple client mounts a user's share, it mounts the share (in this case, afp://server/Users as /Network/Servers/server/Users). It then sets up any necessary symbolic links, like /Users/me to /Network/Servers/server/Users/me. Of course, this fucks up royally when you've got Fast User Switching enabled, because as soon as the 2nd user you tries to log in, it tries to mount afp://server/Users, realises that it's already been mounted, and then tries to write into your home directory. However, since the share has been mounted by me, and me doesn't have write access to you, there's no way that mounting the share as a whole will ever work. It's not just FUS -- any kind of multiple login where the person logged onto the machine is on the same share as the other one will fail just as gratuitously. (Don't believe me? Set up two users with network mount points, get one to log onto the machine and then ssh in as the other one. The ssh user won't be able to write to their directory.)

  • Shite implementation of permissions over AFP. I've been fighting with an issue for a while where my wife's account hasn't been able to persist the fact that we live in the UK (and you know, speak English and use sensible sized paper. F'ing annoying, because every time she prints something, she's got to hit a button on the printer to tell it to use the paper that we keep telling Office that we've got.) Of course, this was easily attributable to Office -- but digging deeper when trying to get iTunes onto her iPod until 4am yesterday morning, it turns out that AFP is just shit. For example, spot the difference between the two sets of permissions:

    server[shite] pwd /Groups/us/shite server[shite] ls -l total 0 -rw-r--r-- 1 me us 0 Aug 13 01:31 crap bigmac[shite] pwd /Volumes/Groups/us/shite bigmac[shite] ls -l total 16 -r--r--r-- 1 me me 0 Aug 13 01:31 crap
    afp_00dj8822ZQpG001Eic06SdO0-1.2c000011 on /Volumes/Groups (nodev, nosuid, mounted by me)

    Spot the difference? Yup, that's right -- despite the fact that me mounted the file, and on the server, it's me who owns the file, somehow it's translated into a read-only file by the time it gets to the client. Log into server; I can edit the file. Log into bigmac; I can't. Also note that the group has been buggered; no longer is it in the us group; nope, it's owned just by me now. Guess that will screw up anyone who wants to look at that file, then.

    And now, just to give you all a laugh, what happens when I do chmod g+w on the client? Fuck all. That's right. Nada. Zip. Nothing. Well, no surprise there; it's not like AFP is going to work well anyway. So what about if I do chmod g+w on the server, and look again?

    server[shite] pwd /Groups/us/shite server[shite] ls -l total 0 -rw-rw-r-- 1 me us 0 Aug 13 01:31 crap bigmac[shite] pwd /Volumes/Groups/us/shite bigmac[shite] ls -l total 16 -rw-r--r-- 1 me me 0 Aug 13 01:31 crap

    Fuck me! It's using the group permissions for the user permissions, whilst at the same time applying general read-only access for the group (which is my default group, me, rather than the group it's supposed to be). How screwed up is that? So it turns out that Office is saving new files and deleting the old one; each write causes a -rw-r--r-- to be created, which then magically transforms into a -r--r--r--, and thus can't be saved a second time. For once, the problem is squarely on the filing system, and not on Office. (Note: the mount for /Volumes/Groups was set via the Command+K from Finder, and it's shared under normal sharing guidelines from the server.)

So, I'm moving back to mounting /Users and /Groups as NFS mounts. I've set up kerberos so that only kerberized logins will allow NFS mounts; but at least now they have the right permissions for all concerned. I can then put all this AFP rubbish behind me.

Incidentally, I did get a few things sorted: I can now FUS between my account and my wife's over network home directories. I modified the kerberos record to use /UsersMe and /UsersYou as the mount, so now we get independent mounts (which results in my home directory being /UsersMe/me and hers in /UsersYou/you. Nicely, she can send stuff to my public folder via /UsersYou/me/Public/DropBox and vice-versa. But it's incredibly ugly, and to allow logging in on the server, I had to create symbolic links from /UsersMe->/Users and /UsersYou->/Users. Of course, we wouldn't need to do this if we had a decent mount point used by pretty much any other automounter known in the universe. The big problem is that it tries to mount the entire /Users share as a single person, rather than /Users/me as its own independent share. But unfortunately, there's no way of fixing this problem; it's a fundamental flaw in the way that the AFP works.

I note from the WWDC that there is some sneak peek from Leopard server that there's improvements to file services. Realistically, you've seen in almost every security update the file protocols being updated. I think that it would make more sense to use your own version of NFS or an even better filing system like AFS. On the other hand, it may be that by switching to AutoFS, they have fixed these problems.

But at the end of the day, I had absolutely zero problems with using NFS. The problems started -- and started in droves -- when I switched to AFP. And don't get me started on über-anal applications like iTunes that throw a major wobbly if the file permission isn't exactly -rw------ (which when you're sharing your un-DRM MP3 collection with other members of your family is a pain in the fricking arse). Now, when I want to run iTunes (or iPhoto) on another account, I have to chmod -R the entire media tree because the application is craptacular. I just don't think I want to be bitten again by this. I'll probably format an old Mac, and hook it up with a Linux-mounted NFS server, and ditch Mac OS X Server. It's more effort than it's worth. And the less said about the frigging Finder turding .DS_Store files all over the place. I'd pay someone money to write a file system driver for Mac that (a) allows transparent mapping of users, (b) appears to the OS as a local drive even if it's network mounted, (c) automatically rejects any attemtps from Finder to create .DS_Store files. (Deleting them afterwards isn't really a sensible option. Using another finder like PathFinder might be.) On the other hand, I'm surprised that a new finder wasn't announced at WWDC -- maybe there will be a decent Finder announced at MacWorld Expo, with Server that uses a decent automounting file system that avoids these problems. But nothing's going to get me back the five or so hours I was fighting the frigging group-is-you problem.