Tag Archives: bug reports

File a bug, but where?

Posted on by
Reply

It’s easy to tell someone to file a bug. Knowing where to file it isn’t quite as simple. In a large project like Fedora, there is no One Tracker to Rule Them All.

Red Hat Bugzilla comes the closest. That’s where most software bugs should be filed. Except for Fedora CoreOS which would rather you use their GitHub tracker. And a few other components, which would very much prefer issues be filed upstream.

But for non-software issues, Bugzilla may or may not be the place. Components still exist for documentation and websites, although those are mostly handled via Pagure repos now. (I should really remove those components.) And some teams discuss software features in their Pagure repo for collaboration and then might open a Bugzilla bug if other tools need one.

Of course, knowing which platform is only the start. You also have to know which component (in Bugzilla) or repo (in Pagure) is right. This isn’t always clear when you’re familiar with the inner workings of the project and is entirely opaque to outsiders. A casual user is not going to know (or care) which URLs happen to be owned by the websites team versus the docs team versus who knows what other team.

And that, dear reader, is one of the key features of a centralized tracker like Bugzilla: mis-filed bugs can be trivially passed to the right team. With independent repos, you force users to know your project’s org chart

Of course, Bugzilla can be pretty heavy and is generally more complicated than you need. There’s no right answer. There’s just how you’re willing to balance the tradeoffs.

In general, I think putting software issues (bugs, features, etc) in Bugzilla and process issues in a team’s Pagure repo is the right approach. But ultimately, teams decide for themselves how they will work.

[If you are intrigued by this half-baked post, you’ll enjoy my book on program management for open source projects, coming from The Pragmatic Bookshelf in 2022.]

When to have a bug tracker

Posted on by
1

Back in April, I wrote a post about how the traditional “file a bug” or “patches welcome” responses to issues weren’t always helpful. For as long as open source software has been something I think about, I’ve been of the mind that bug trackers are a necessary component of a project. After all, if we’re going to take advantage of the “given enough eyeballs, all bugs are shallow” philosophy, someone the eyeballs have to be able to communicate information about the bugs.

But then “magnus” made a comment that caused me to question that requirement.

A free sw project should not have a bug tracker unless there is enough organization behind to actually service the reports IMO. Unfortunately many hosting sites have the mentality that they throw in one for free in every project since it “can’t hurt”.

The more I thought about this, the more I agree with it. I think there’s active harm in having a bug tracker that is ignored. Without a bug tracker, someone might file a bug via email, or maybe not at all. Worst case, they probably just go find another project that meets their needs. But once the door is open to filing bugs, people have an expectation that the bugs will be fixed. Bugs that sit in the tracker forever don’t make people indifferent, they make people angry.

If your project lacks the resources or the desire to manage bugs, then it’s probably best to not have a bug tracker. When I say “manage” bugs, I don’t necessarily mean fixing them. Bugs need to be triaged, they need to be verified, they need to be fixed. Critically, the people filing the bugs need to feel like their contributions are valuable. Of course, the contributions are sometimes not valuable, but that’s just an opportunity to help someone learn how to become a valuable contributor.

magnus went on to talk about how bug trackers are implemented:

Also I don’t know if putting a bulky database system between the users and the developers, which makes sense in the commerical world, really makes the best use of the strength of the free software community.

Something more lightweight and free form would be better in many cases. An interface something like this very blog commenting interface could be appropriate. Here I just write what I think and I don’t feel any social expectation that you will reply to it (but would be fun if you did).

I think Bugzilla is a great platform for certain large projects. It’s absolutely atrocious for a casual user. And it’s complete overkill for small projects. Simpler interfaces like GitHub’s Issues can often be just as valuable with a lot less friction. Something even simpler like Trello might be suitable for small projects. Whatever the developers will use works, so long as they use it.

File a bug!

Posted on by
13

The ability for users to submit patches is often touted by advocates of open source software. “Patches welcome!” is a common refrain on mailing lists. Even if someone lacks the ability to diagnose and fix an issue, they can file a bug in the project’s system. Indeed, there’s an unwritten social expectation that users file a bug.

Sadly, these are often just empty words. “Patches welcome” can be a seemingly-polite way of saying “your problem is not important to me. Go solve it yourself and I’ll accept your solution.” And telling a user to go file a bug can be equally dismissive, especially if the bug-filing process is unpleasant.

I certainly don’t mean to imply that all or even most open source developers use these phrases as a polite cover for their disinterest. There are jerks, of course, but I suspect most developers genuinely want bug reports and patches. Last week, a very busy developer replied to a mailing list post with a terse “file a bug.” Now, I happen to know this particular developer and I know he’s keenly interested in fixing bugs. On this day, he was swamped and didn’t have time to get to the issue right away. Suspecting from the original email that the user who reported the bug wasn’t deeply technical, I took the liberty of reproducing the issue and filing a bug with the details.

Would the person who originally reported the issue have filed the bug done so if I hadn’t? We’ll never know, but I do know that he didn’t by the time I did. After creating a Red Hat Bugzilla account, selecting the right options, and filling out the bug report, he’d have to hope the developer meant it and that he bug would get fixed. As anyone who has been around a while can attest, just because a bug is filed, that doesn’t guarantee it will be fixed in the next few years.

One particular bug that I filed sticks in my memory. It was undeniably a bug, I attached a patch to fix it, and yet the bug remained untouched through several Fedora releases. Granted, it was a relatively minor issue, but if I weren’t involved with the project, I’d have been pretty put off by this. Of course, no one owes me a fix, but as Chris Siebenmann noted, if there’s a social obligation to file a bug, there’s also a social obligation to deal with the bug.

This morning,I asked on Twitter if anyone had what they’d call a “positive” bug reporting experience (as opposed to simply a not-negative experience). I was pleasantly surprised when several people chimed in to say they had. Small projects were generally the ones that drew the good responses (Jordan Sissel was mentioned by multiple people).

So what makes for a positive experience? Rapid acknowledgement and resolution were mentioned several times. Those who replied mentioned courteous and helpful interactions (e.g. asking constructive questions, filing an appropriate bug when the original bug was mis-filed). @phrawzty summed it up well: “Most importantly, however: good intentions were assumed. I never felt treated like a dangerous outsider.”

This is an area wide open for study (and I may end up doing just that), but in the meantime projects should consider how they present themselves. Large projects with more resources should make an active effort to devote some time to bug handling as a community outreach effort (I suspect large projects have worse bug experiences, in part due to the fact that nobody owns the process). When you say “patches welcome” or “file a bug”, consider what you’re really saying and if people will take it the way you mean it.

(Thanks to those who provided feedback: @cpj1, Matt Simmons, Matthew General, Patrick Cable, @phrawzty, Rick Waldron, @SysAdm Dredd, Thomas Ballinger, @Twirrim, and anyone else I failed to mention.)

Getting good reports from users

Posted on by
Reply

Earlier this week, Rob Soden asked an excellent question on Twitter: how do you train your non-technical co-workers to file useful/actionable bug reports? Is it possible?

It is, but this question doesn’t just apply to developers. Operations people need useful incident reports from users as well. Users generally don’t provide unhelpful reports maliciously.. In my experience, reports are often the least useful when users are trying to be the most helpful. A healthy relationship between a customer and a service provider means that everyone is working together. Users often don’t know what information is helpful, they just want to go about their work.

A useful report contains the following:

  • What happened and what the desired result is
  • What the user was doing when it happened
  • Any changes that may have happened
  • If the product/service ever worked and if so, when it last worked
  • If the problem is reproducible and if so, what steps are required to reproduce it

A useful report does not contain speculation about what the solution is. But what, Rob asked, if the lesson doesn’t stick? It gets annoying having to ask over and over again. As with any repetitive task, the answer here is to automate it. One option is to have a canned response at the ready. This way, you polish the text once and you never have to worry about irritation creeping in. A better solution is to have a website that explicitly asks for the information you need. This saves a round of back-and-forth and hopefully increases the time to resolution.

The most important part is to modify the process as you learn. Tweak the questions to make it easier on the users. Provide multiple-choice or other defined-response choices when you can to reduce ambiguity. No good process is ever in a final state.