78% of companies “run on open source”

Black Duck Software recently released the results of their annual Future of Open Source survey. On the surface, it looks pretty good. As the title of this post says, 78% of companies “run on open source”. Open source usage has doubled in business IT environments since 2010. Two-thirds consider open source offerings before their proprietary counterparts.

Not only are companies using open source software, they’re contributing, too. Some 64% of companies participate, with nearly 90% expecting to increase their contributions in the next few years. Half of the companies say more than 50% of their engineers are working on open source. Many companies see open source participation as a recruiting tool as well.

But when you dig a little deeper, there are some issues, too. A majority of companies view open source software as having higher quality and security, but most don’t monitor the code for vulnerabilities. Companies lack formalized policies both for consumption and contribution. A lot of the terms are pretty vague, too. “Participation” in open source can take on a variety of meanings, some of which are basically token involvement for PR purposes.

What I found most interesting, though, was the projects listed as “most valuable”: OpenStack and Docker. I may be biased by my day job, but I see that as a sign of the rise of *aaS. Despite the growth that cloud services have already seen, there’s a lot more market out there to be tapped.

Another interesting item was the increase in venture capital investment, both in gross and per-deal measures. Hopefully, this reduces the issues faced by projects such as OpenSSL and PGP, where a lack of funding puts much of the Internet’s secure communication at risk.

Finally, my initial reaction to the headline was “the other 22% do and don’t know it.” As it turns out, I wasn’t that far off. Black Duck reported that 3% of respondents do not use open source software at all. (Where’s the remaining 19%?) I actually wonder if that’s true. It seems like you’d have to try pretty hard to avoid any of it. This will become increasingly true as time goes on, when even historically hostile companies like Microsoft being open sourcing some of their products.

File a bug!

The ability for users to submit patches is often touted by advocates of open source software. “Patches welcome!” is a common refrain on mailing lists. Even if someone lacks the ability to diagnose and fix an issue, they can file a bug in the project’s system. Indeed, there’s an unwritten social expectation that users file a bug.

Sadly, these are often just empty words. “Patches welcome” can be a seemingly-polite way of saying “your problem is not important to me. Go solve it yourself and I’ll accept your solution.” And telling a user to go file a bug can be equally dismissive, especially if the bug-filing process is unpleasant.

I certainly don’t mean to imply that all or even most open source developers use these phrases as a polite cover for their disinterest. There are jerks, of course, but I suspect most developers genuinely want bug reports and patches. Last week, a very busy developer replied to a mailing list post with a terse “file a bug.” Now, I happen to know this particular developer and I know he’s keenly interested in fixing bugs. On this day, he was swamped and didn’t have time to get to the issue right away. Suspecting from the original email that the user who reported the bug wasn’t deeply technical, I took the liberty of reproducing the issue and filing a bug with the details.

Would the person who originally reported the issue have filed the bug done so if I hadn’t? We’ll never know, but I do know that he didn’t by the time I did. After creating a Red Hat Bugzilla account, selecting the right options, and filling out the bug report, he’d have to hope the developer meant it and that he bug would get fixed. As anyone who has been around a while can attest, just because a bug is filed, that doesn’t guarantee it will be fixed in the next few years.

One particular bug that I filed sticks in my memory. It was undeniably a bug, I attached a patch to fix it, and yet the bug remained untouched through several Fedora releases. Granted, it was a relatively minor issue, but if I weren’t involved with the project, I’d have been pretty put off by this. Of course, no one owes me a fix, but as Chris Siebenmann noted, if there’s a social obligation to file a bug, there’s also a social obligation to deal with the bug.

This morning,I asked on Twitter if anyone had what they’d call a “positive” bug reporting experience (as opposed to simply a not-negative experience). I was pleasantly surprised when several people chimed in to say they had. Small projects were generally the ones that drew the good responses (Jordan Sissel was mentioned by multiple people).

So what makes for a positive experience? Rapid acknowledgement and resolution were mentioned several times. Those who replied mentioned courteous and helpful interactions (e.g. asking constructive questions, filing an appropriate bug when the original bug was mis-filed). @phrawzty summed it up well: “Most importantly, however: good intentions were assumed. I never felt treated like a dangerous outsider.”

This is an area wide open for study (and I may end up doing just that), but in the meantime projects should consider how they present themselves. Large projects with more resources should make an active effort to devote some time to bug handling as a community outreach effort (I suspect large projects have worse bug experiences, in part due to the fact that nobody owns the process). When you say “patches welcome” or “file a bug”, consider what you’re really saying and if people will take it the way you mean it.

(Thanks to those who provided feedback: @cpj1, Matt SimmonsMatthew General, Patrick Cable@phrawzty, Rick Waldron@SysAdm DreddThomas Ballinger, @Twirrim, and anyone else I failed to mention.)

elementary misses the point

A recent post on the elementary blog about how they ask for payment on download created a bit of a stir this week. One particular sentence struck a nerve (it has since been removed from the post): “We want users to understand that they’re pretty much cheating the system when they choose not to pay for software.”

No, they aren’t. I understand that people want to get paid for their work. It’s only natural. Especially when you’d really like that work to be what puts food on the table and not something you do after you work a full week for someone else. I certainly don’t begrudge developers asking for money. I don’t even begrudge requiring payment before being able to download the software. The developers are absolutely right when they say “elementary is under no obligation to release our compiled operating system for free download.”

Getting paid for developing open source software is not antithetical to open source or free (libre) software principles. Neither the OSI’s Open Source Definition nor the Free Software Foundation’s Free Software Definition necessarily preclude a developer from charging for works. That most software that’s free-as-in-freedom is also free-as-in-beer is true, but irrelevant. Even elementary touts the gratis nature of their work on the front page (talk about mixed messages):

100% free, both in terms of pricing and licensing. But you're a cheater if you take the free option.

100% free, both in terms of pricing and licensing. But you’re a cheater if you take the free option.

Simply put, the developers cannot choose to offer their work for free and then get mad when people take them up on the offer. Worse, they cannot alienate their community by calling them cheaters. Of the money the elementary receives, how much of it goes upstream to the Linux Foundation, the FSF, and the numerous other projects that make elementary possible? Surely they wouldn’t be so hypocritical as to take the work of others for free?

An open source project is more than just coders. It’s more than just coders and funders. A truly healthy project of any appreciable size will have people who contribute in various ways: writing documentation; providing support on mailing lists, fora, etc.; triaging bug reports; filing bug reports; doing design; marketing (including word-of-mouth). This work is important to the project, too, and should be considered an in-kind form of payment.

It’s up to each project to decide what they want in return for the work put in. But it’s up to each project to accept that people will take from all of the choices that are available. If that includes “I get it for free”, then the right answer is to find ways for those people to become a part of the community and contribute how they can.

In defense of the bazaar

Earlier this week, I came across a 2012 article from Poul-Henning Kamp entitled “A generation lost in the bazaar“. This is a reference to Eric S. Raymond’s seminal The Cathedral and the Bazaar, which advocates for making the sausage, so to speak, in public. Using the Linux kernel and his own fetchmail program as examples, Raymond emphasizes the benefits of rapid, iterative development and of fostering a user community that acts as co-developers. This stands in contrast to the “cathedral” style of development where a product is worked on by a small number of people until it is ready to be revealed to the public.

Kamp’s point (and subtitle) is “quality happens only when someone is responsible for it,” which I endorse wholeheartedly. However, he is mistaken to blame Raymond’s bazaar for “a clueless generation of IT ‘professionals’ who wouldn’t recognize sound IT architecture if you hit them over the head with it.” What he observes is the democratization of programming, which is due to ever-cheaper hardware, free (as in beer) software, and the Internet. Had The Cathedral and the Bazaar never been written I doubt the world would look dramatically different, at least in this respect.

IT is in its awkward teenage years. It has been around long enough that it can do pretty cool things, but not so long that it has accumulated much wisdom. The fact that anyone can write software (or copypasta snippets from various example sites and fora) and make it available to others is simultaneously a wonderful and terrible thing. Nonetheless, that doesn’t make the bazaar style wrong.

Kamp described the end result of the bazaar as “a pile of old festering hacks,” and I’ll agree that its an apt description for a lot of software. It’s probably just as apt for a lot of software developed in the cathedral style. Raymond devotes a fair portion of his book to quality and good design, and it’s unfair to blame him for people not following that part (assuming they’re even aware of his work at all).

Raymond makes many unsubstantiated claims that the bazaar style of development leads to higher-quality software. That may or may not be the case. My own view is that the bazaar style is well-suited for open source projects. After all, open source is about more than code.

Open source is about more than code

The idea of open source developed in a closed manner is hardly new. The first real discussion of it came, as best as I can tell, in Eric S. Raymond’s The Cathedral and the Bazaar. A culture of open discussion and decision making is still a conscious act for projects. It’s not always pretty: consensus decision making is frustrating and some media outlets jump on every mailing list suggestion as the final word on a project’s direction. Still, it’s important for a project to make a decision about openness one way or the other.

Bradley Kuhn recently announced the copyleft.org project, which seeks to “create and disseminate useful information, tutorial material, and new policy ideas regarding all forms of copyleft licensing.” In the first substantive post on the mailing list, Richard Fontana suggested the adoption of the “Harvey Birdman Rule,” which has been used in his copyleft-next project. The limited response has been mostly favorable, though some have questioned its utility given that to date the work is almost entirely Kuhn’s. One IRC user said the rule “seems to apply only to discussions, not decisions. The former are cheap and plentiful, but the latter actually matter.”

I argue that the discussions, while cheap and plentiful, do matter. If all of the meaningful discussion happens in private, those who are not privy to the discussion will have a hard time participating in the decision-making process. For some projects, that may be okay. A ruling cadre makes the decisions and other contributors can follow along or not. But I see open source as being more than just meeting the OSI’s definition (or the FSF’s definition of free software for that matter). Open source is about the democratization of computing, and that means putting the sausage-making on public display.

How do you measure software quality?

There are two major license types in the free/open source software world: copyleft (e.g. GPL) and permissive (e.g. BSD). Because of the different legal ramifications of the licenses, it’s possible to make theoretical arguments that either license would tend to produce higher quality software. For my master’s thesis, I would like to investigate the quality of projects licensed under these paradigms, and whether there’s a significant difference. In order to do this, I’ll need some objective mechanism for measuring some aspect(s) of software quality. This is where you come in: if you have any suggestions for measures to use, or tools to get these measures, please let me know. It will have to be language-independent and preferably not rely on bug reports or other similar data. Operating on source would be preferable, but I have no objections to building binaries if I have to.

The end goal (apart from graduating) is to provide guidance for license selection in open source projects when philosophical considerations are not a concern. I have no intention or desire to turn this into a philosophical debate on the merits of different license types.

How Richard Stallman got me to ponder extremism

This evening, I had the opportunity to attend a speech by a man whose work over the past decades enters into my life on a daily basis. The Network for Computational Nanotechnology at Purdue hosted Richard Stallman, the founder of the GNU Project and the Free Software Foundation. Stallman is a well-known and controversial figure, not only because of his technical work, but also (primarily?) because of his idealism and activism. His un-nuanced views and public lack of tact have driven fans of his work away from the FSF. I went into the talk expecting some pot-stirring. I didn’t expect to walk out deep in thought.

Stallman opened with a discussion of terminology, drawing a distinction between free (for the purposes of this post, free software means libre, not gratis) and proprietary software.  It is an ethical, social, and political distinction, not a technical one. Free software, Stallman argues, is a contribution to society. Proprietary software is morally unjust. Stallman prefers, given the choice between writing proprietary software and doing nothing, that developers do nothing. Even though free software is often available at no cost, encouraging the adoption of free software should be framed as a moral issue, not an economic or practical one. Software as a Service (SaaS) is morally equal to proprietary software in Stallman’s view, regardless of the licensing of the software, because users have no control over it.

During the question-and-answer session at the end, this view brought a heated discussion from NCN director Dr. Gerhard Klimeck. NCN runs nanoHUB, which is effectively SaaS for nanotechnology simulation. Stallman seemed to argue that it was a niche and not really important to discuss. He also semi-adroitly dodged the question of how developers can make money with free software, only asserting that it is being done without providing the [mostly student] audience any insights as to how.

Stallman’s views are based on his personal morality and seem to be absolute. This is what occupied my thoughts on the walk back to my car. Because I largely agree with Stallman, I’ve been inclined to see his extremism as an annoying, but useful thing. By being on the edge, he defines the middle. But why should extremism that I happen to generally agree with be more valid than extremism that I disagree with? While extremism does help define the middle ground, it also poisons reasonable discussion. I admire and appreciate his technical accomplishments, but I think he hurts his own ideological cause.

CNET considered harmful

In my younger days, I made great use of CNET’s download.com website. It was an excellent tool for finding legal software. Apparently, it has also become an excellent tool for finding malware. An article posted to insecure.org describes how CNET has begun wrapping packages with an installer that bundles unwanted, potentially malicious software with the desired package.

This is terrible, and not just for the obvious reasons. It’s bad for the free software community because it makes us look untrustworthy. There’s a perception among some people (especially in the business world) that software can only be free if it’s no good. I suppose that’s one reason some in the community use “libre” to emphasize the free-as-in-freedom aspect. (Of course, not all free-as-in-beer software is free-as-in-freedom. That’s another reason the distinction can be important.)

When this conveniently-bundled malware causes problems for users, it’s not CNET who gets the blame. Users will unfairly blame the package developer, even though the developer had nothing to do with it. For well-established and well-respected packages like nmap, this reputation damage may not be that important. For a new project just getting started — or for the idea of free software in general — this can be devastating.