Alex headshot

AlBlue’s Blog

Macs, Modularity and More

Eclipse bug reporting

2009, eclipse

I recently postulated the question: “Raising bugs against #eclipse - worthwhile or a waste of time? Discuss.” So I thought it would be interesting to see where my bugs were in the RESOLVED state, by resolution:

So out of the bugs I've raised over the years, half of them are fixed, quarter are DUPLICATEs, and the other are either ones that have been fixed already (WORKSFORME), ones that are not valid (INVALID) or ones that simply aren't going to get fixed, through ideolgy, compatibility or other such (WONTFIX). What the graphics don't show is whether those DUPLICATE bugs have been fixed, closed or other; but in my experience, the reason a bug gets marked a DUPLICATE is because it's been a long-outstanding problem and keeps getting reported repeatedly. A query suggests DUPLICATE bugs are fixed around 60% of the time, with around 25% of bugs still open. The remainder are misc items like WONTFIX, LATER, or WORKSFORME.

Perhaps more interesting is the overall proportion of raised bugs versus resolution. Omitting duplicate bugs (for simplicity), here's the results of my bugs:

There's certainly an inconsistency between how bugs are (or aren't) moved between RESOLVED->VERIFIED>CLOSED; but either way, the graphic suggests that 15% are still open (with ~5-10% being marked a non-fixed resolution like WONTFIX). On the face of it, then, it looks like the issues are being resolved. However, one item which is misleading is that bugs raised against older versions tend to stay old. A different report shows that my open bugs range over a variety of different versions, with Platform and JDT collecting the majority of stale bugs (i.e. those that are open against non-maintenance versions and likely never to be closed):

Platform JDT

Putting that into perspective, some of these bugs are still marked as NEW despite being opened in 2002. My oldest NEW bug is bug 26814, although bug 26809 is in the RESOLVED NEVER category, and has had a number of duplicates raised against it. (bug 206113 was marked as a dupe in 2007, but sensibly re-opened since this feature was never fixed.) Although I've used Eclipse since the 1.0 days (and previously, the Visual Age series of products) as an IBMer, I didn't get involved in the Eclipse community until 2.1, which was when the initial Mac port came out.

So, what's the conclusions that you can draw from this? Well, even though there's a lot of open bugs in Eclipse generally, my personal response rate has been pretty good. There's probably more chance of the smaller projects being agile and fixing bugs than the larger ones, which tend to not be able to address all the issues in each release. If you've got bugs against the ECFs, XSLT, Mylyn type projects, then they're well worth filing; you'll probably get a good turnaround or discussion. Enhancements against some of the larger projects are the most likely to go AWOL; frankly, it's probably not worth raising an enhancement request against JDT or Platform unless you're prepared to supply a patch yourself (and even then, I've supplied patches to JDT to change sort ordering of members and kerberized CVS support which have gone stale due to being ignored for years).

Distributed version control systems and tools, like eGit, may help to prevent patches going stale in time. They may even help forking for fixing bugs that (for whatever reason) don't get fixed upstream; though obviously the desire is for forking to not happen. But the larger the upstream project, the less likely patches make it in from the outside world; and some teams are either in maintenance mode or just don't respond to changes any more.

Lastly; bugzilla triage and assistance from the community. There's a lot that could be done to help out projects which have a large amount of outstanding or potentially invalid bugs that could be cleared up by the community. Unfortunately, the current setup of Bugzilla is very much a partitoned, you-can-only-change-your-own-bug kind of model. That means that practically, you can't change anyone else's bugs, even when the bug may have been filed by someone else (or worse, when your bug is marked as a duplicate of theirs) whose e-mail address is invalid. The typical way of making changes is to add a comment suggesting that perhaps some features of the bug be changed (or marked invalid, or whatever). In practice, this is basically a waste of time (both of the triager and the committer who then applies that change). Not only that, but inter-team inconsistency in how Bugzilla is used causes expectation problems for the community who may use a number of different Eclipse components. For example, the RESOLVED vs VERIFIED vs CLOSED noted above is badly handled; some projects use CLOSED whilst others only get as far as RESOLVED. The version number of the bug is also inconsistent; when a bug is raised against a newer version (and then marked duplicate of an older version) doesn't help anyone. Who knows if a bug filed against 2.1 is still a 3.5 bug (like, for example, the packaging of Mac apps) if the version number is not updated? Yet there are projects who steadfastly refuse to update bug version numbers when any kind of filter will simply hide those from prying eyes. (The argument goes 'but we need to know what version it was raised against', but bug history shows you that. If it's really that important, perhaps a new Bugzilla field "First reported against" could explicitly record the initial occurrence of a problem whilst leaving the Version field to be updated when the bug is confirmed to be in a current version.) Sadly, I don't see this improving; there's a lot of inertia in the system; no-one wants to agree on a standard way of handling bugs; committers complain about volume of e-mail when you change things; hiding bugs by pretending they're old and irrelevant cuts down the amount of bugs that committers look at regularly; and lastly, Bugzilla itself doesn't permit changes easily or reduce the amount of minor noise that gets generated (like updating the version number of bugs).

So, is raising bugs worth it? Well, I like to think I've made a (positive) difference to the Eclipse platform, both on the Mac and all operating systems generically. Since I started raising bugs about 6½ years ago, I've raised 417 bugs, which works out on average around one bug every 17½ days. I've contributed some code to the platform (Project->PDE Tools>Open Manifest was mine, as was the Sort Members cleanup) but the majority has been finding issues or making suggestions about how the platform as a whole can be better. Sometimes I push hard for what I believe in; but in all cases, I'm aiming to make Eclipse a better and more inclusive platform. And, like Doug's hobby of Wascana, all my Eclipse work is hobby-based. I've not worked with Java for a good four or five years now; my day job doesn't involve Eclipse or any of its derivatives; and when I go to EclipseCon I have to fund myself to get there (and back). Oh, and writing for InfoQ (and before that, EclipseZone) about what's happening in the Eclipse world. And that's before any Eclipse-based hobbies like ObjectivEClipse or eGit come into play. And no, I'm not an Eclipse committer on any project.

So, to answer the original question: “Raising bugs against #eclipse - worthwhile or a waste of time?” It can be a worthwhile and rewarding experience. It can also be frustrating at times. There are improvements to the bug reporting (and triaging) process that could be done, and harmonisation of the way Bugzilla is used could be of benefit to the community as a whole. But taken as a whole, one person can make a difference and bugs (or patches) are the way to help.