Open source quality -

Open source quality

I've been in a few discussions about the quality of programs. Naturally, the debate gets polarized, with more anecdotes than facts in play. One the one side are the people who favor proprietary software. On the other side, the Open Source supporters.

The proprietary software advocates point to the well publicized exploits in free and open-source software (FOSS) like Heartbleed. They talk about how there's no control over who submits patches to an Open Source project. They point to the companies behind the proprietary software, with their customer service departments, someone they can talk to when they run into problems, not someone on a mailing list who answers (if they answer at all) to inquiries with responses that range from irrelevant to rude.

On the other side of the ring, the FOSS advocates point to the zero-day security flaws in Microsoft Windows or the weekly bug fix update to Adobe Flash. They talk about how anyone can download source for an Open Source program and look for bugs, something you can't do with Internet Explorer. The talk about the open bug report databases, where you can see just what problems have been reported and the progress getting the bug fixed.

Who's right and who's wrong? Well, to a significant degree both are right and both are wrong. And to a large degree, both miss important factors contributing to quality programs. I've worked for companies developing proprietary programs, and I've worked for companies developing FOSS programs. I've seen excellent quality programs developed in both environments, as well as poor quality programs in both.

So let's start by knocking down some of these straw men. Let's start with the customer service department taking your bug report. Does that mean that your bug will get fixed sooner than if you posted a similar bug report to an Open Source project's mailing list or bug database? Unfortunately, both proprietary and Open Source development teams are chronically over-committed. Developers want to create new features, not fix bugs. Your bug will be ranked by both in different ways, such as how critical the flaw is to the program's overall performance, how many people are affected, and that ineffable quality “priority”. At one proprietary company, we were supposed to prioritize bugs by the impact on our customer base. I joked that the priority of a bug was directly proportional to the number of zeros on the submitter's annual purchases. In the FOSS world, priority might be based whether the bug affects the developers. If you spend enough with a company, you might be able to have your bug report addressed before others. On the other hand, if you have money, you can hire someone to fix a bug in an Open Source project.

What about the expertise of the people working on programs? The fans of proprietary software would have you believe that everyone developing their favorite software package is an expert in the field. The truth is that companies hire junior staff and they may be the ones working on the program not the experts. They may be working under the mentorship and guidance of a more experienced person, or that person may have moved on to some different project. Or he or she may have left the company. With a private company it's hard to tell who, if anyone, is working on a product. Open source projects also have their share of newbies, but you can tell who is doing what by reading the project's mailing or the notes in their bug tracking database. And, oddly enough, when someone who has been working on an Open Source project changes employers, that often doesn't mean that they stop working on the project. The project retains the developers expertise; it's just that someone else is paying the bill (and perhaps setting goals).

Isn't using a program developed in the Wild West of Open Source, where anyone can submit patches, much riskier than using software from a proprietary business where their reputation is on the line and they have corporate goals of high quality. Well, I hate to be the one to break the news to you, but the first corporate goal is to make a profit, all others are secondary. I've seen companies that had excellent quality control processes as well as those which had none. From the outside, it is virtually impossible to tell which is which. Open Source projects also vary, with some practicing code reviews and other quality-promoting practices, others not.

Companies sometimes feel pressure to release a product before it is complete, whether to meet a marketing requirement or to match a predetermined release schedule. For example, Apple's release of a poorly tested mapping application a couple years ago was likely dictated by the product release schedule, unaffected by quality concerns in one component. Some Open Source projects will delay a release until it meets specified quality criteria, others have the philosophy of releasing what they have working on their scheduled release date, with the definition of working being, shall we say, flexible. Each new release of Fedora, which I use, seems to have a set of new features, sometimes incomplete or breaking previous functionality. These problems get corrected in the next release (or two), but that subsequent release also has its own set of new, not quite cooked features.

The very best of proprietary and Open Source show stability and consistency, where each version adds something new, while maintaining the best of the previous versions. Examples in the Open Source world include the Linux kernel, which has rules about not breaking compatibility. In the proprietary world, one can look at TurboTax, where every annual release seems to work reasonably well. Contrast this with the Open Source Gnome project which caused many user complaints when they completely changed the user interface between releases, just as Microsoft did between Windows 7 and 8.

What is the take-away from all this? That program quality is affected by many factors, including developer expertise, corporate or project culture, and release pressure, among others. These same factors are in play on both sides of the corporate wall. In choosing a program which would be best for a particular purpose, you have to take a close look at your choices. You can't simply say Open Source is always better or proprietary is better.

5 thoughts on “Open source quality

  1. “An excellent, balanced summary.nnHowever I do take exception with one statement: “…the Linux kernel, which has rules about not breaking compatibility. “nnThat compatibility only applies to user space binaries. It does not apply to interfaces insid

    Log in to Reply
  2. “Michael, a very good summary and puts things in a balanced perspective. The other angle I would add to the open source experience is as follows. The quality also appears to be related to the relative number of users. As a hardware engineer's first foray

    Log in to Reply
  3. “It is important to understand that the quality of open source code is very variable. That includes some of the Linux kernel code.nnChip vendor code in particular can be awful.nnMuch open source code progresses like the story of stone soup: https://e

    Log in to Reply
  4. “I think that the number of developers is more relevant than the number of users. Some projects (e.g. bash or ntp) have many users but very few developers. I don't know anything about the Aurdino IDE and the Microchip driver you mention, but I expect tha

    Log in to Reply
  5. “As you've touched on, some companies see “open source'ing” their code as an event and not an experience – flip the license, dump the code (and even some documentation, though not necessarily for the exact device…) over the wall and walk away. nnFor

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.