weblog.masukomi.org

Distributed Bug Tracking January 3, 2008

[Update 2: Ditz is the newest player in town and it looks to be the best current option.  Go check it out.]

[Update: josef on reddit and Paul in the comments here point to another distributed bug tracker called DisTract but that only seems to work with Monotone, which seems to be the least known or used distributed version control system. Also, I didn’t mean to imply that Bugs Every was unique or the first. It was just one of the only ones I found and anyone who is tackling this problem at the moment is a pioneer.]

Panoramic Feedback* has created an interesting tool called Bugs Everywhere (link below) which combines bug tracking with distributed development. The idea is interesting, no, check that. The idea represents a freaking paradigm shift in how we track bugs and one that is really needed. In a world where developers are constantly creating and merging branches it’s entirely common, to find a bug in one branch, but create a new branch just to fix it. A normal bug tracker can’t support this. As far as they’re concerned the bug is either fixed or it isn’t. But, in the distributed world, which bugs happen to be fixed, or exist, in your current branch is entirely dependent upon how the current branch came to be and what other branches you’ve merged into it.

Let’s look at an example. You create a “new_feature” branch off of the trunk to work on a new feature. In the process of creating the new feature you create a new bug. That bug exists in the new_feature branch but not in the trunk. If you never merge that branch back into the trunk does the bug really exist? What if you’re the only one with a copy of that branch. With a centralized system you’d have to file the bug but you’d be the only one who could reproduce it. If a coworker finds a different bug in the trunk and creates a bug_fix_a branch to fix it does it count if he hasn’t merged it back into the trunk yet? What if you merge bug_fix_a into your new_feature branch. Now the fix from bug_fix_a exists in two branches but neither of them are the trunk. If you merge new_feature into the trunk now the trunk has the fix, but unless you were paying close attention and didn’t forget that your new_feature also included the patch from bug_fix_a you might not know you should close out the ticket that was fixed in the bug_fix_a branch (that was never directly merged into the trunk).

A centralized bug tracker is just not capable of accurately managing a bugs status when developers start using lots of branches, even if they’re doing so in a centralized version control system like Subversion, CVS, or Perforce. They can’t even handle it well when you’ve only got a handful of branches like “trunk”, “qa”, and “production”. If you find a bug in “production”, you’d first fix it in “trunk”, but then what’s the real status here. Is it fixed or not? Even if you’ve had 20 people verify your fix it’s still broken for your end users because they’re running code from the “production” branch.

Bugs Everywhere solves this problem. The idea is that the tracking info gets versioned right along with each branch. New bugs are tracked within the repo, which means that you don’t have to keep track of which branches had which bugs and which ones had the fixes too. It allows you to take full advantage of the greatest feature of distributed version control systems (cheap branching and merging) without having to keep track of the bugs. When you merge the branches the lists of present and fixed bugs goes along for the ride. But, there are some problems:

Where we need to improve:

No Windows support

Bugs Everywhere does it’s magic by calling out to the chosen Source Control Manager via the command line, which means it’ll pretty much never work on Windows. I think that there are two solutions that won’t hit this barrier. One is to intimately tie the bug tracker to the version control system so that it will work on all the same platforms. The other solution, and I think this is the better one, is to simply make the bug tracker into a local server that waits for commands via a standard API from whatever wants to talk to it.

Limited Bug Visibility
Sometimes you really do want to see *all* the outstanding bugs. Bugs Everywhere has a TurboGears based webapp which appears to be capable of listing / managing the bugs in multiple branches but, you’d have to tell it which branches to watch and it would most likely have to have direct access to them, which means each developer would probably have their own instance of the webapp, and then one that ran on a central server against your main branches. The latter is only a half-way solution because if a developer discovers a bug in a branch they made from one of the main branches they’d only have a record of it in their local branch until they merged back into the main one again. But everyone should be aware of the fact that the bug exists in that main repo.

If, however, we have a local bug server, then we can configure it with the url of a central reporting server where bugs will all filter up to when they’re created or modified (or when the computer regains net access). Or, you could have it act as a client on a peer-to-peer network, and the peers can share bugs for reporting purposes. Anyone could ping all the current clients for a list of their bugs. The former is probably the better solution since it’s much easier to write and debug and all the peers wouldn’t have to keep track of bugs on other boxes in case the other boxes go away for a while. In both solutions, when a bug is fixed any of the other local servers that is tracking that bug could be notified that the bug is fixed and they could, in turn, notify the user of who has the fix and, if possible, where to get it.

In Summary

Bugs Everywhere is a great start, but it’s got some conceptual issues that I believe the proponents of distributed version control need to address in a new, replacement, application. I am, however, very glad that it was created. It represents a workable solution to a problem that no-one else has addressed and people have been blind to, or ignoring, for years now.

Bugs Everywhere can be found here, but, before you go there, please note that the page is horribly out of date. It says Bugs Everywhere only supports Arch and Bazaar, but it actually supports Mercurial (hg) too and be sure not to download the .tgz version that’s release 123 (which probably doesn’t support Mercurial) whereas (at the time of this writing (Jan 2 2008) ) the repo is up to 211 (last update on July 30 2007). Don’t believe the command line help from the repo about supported apps either. It also hasn’t been updated. Oh, and it says that it’s easily extendable. That’s actually true, but you won’t find any docs on how to do it and there are no comments or internal documentation in the code. If you’re interested check out the rcs.py and bugdir.py files. Which revision control system is being used is determined when you call set-root or if you call ‘be set rcs_name ‘ the latter is handled by set.py

Personally, if I didn’t think Bugs Everywhere needed to be replaced by a new generation of distributed bug trackers, I’d suggest rewriting the whole thing from scratch. There are lots of files which probably should be classes but are instead just collections of methods, there are no unit tests, and other files have multiple classes shoved in them which makes it a lot harder to determine where things happen (and test). The one thing it has going for it is relatively readable code, but that could just be a byproduct of it being written in Python.

* What the hell is “360 degree feedback”? The only thing on Panoramic Feedback’s home page that gives me a hint is a short quote, presumably from a customer, about people giving “…honest and candid feedback to one another.” and a quote that it’s the “…most evolved multi-source feedback system.”

Popularity: 16% [?]