Are you a superprogrammer?

I’m sure you, gentle reader, are the best software developer on your team. Heck, maybe in the world. Even Linus tries to curry your favor by “liking” or “favoring” your every Facebook post or tweet. Developers of the opposite sex faint when they see you strolling the halls. Well, I should change “opposite” to some complex algorithm now that Facebook recognizes as many genders as Baskin-Robbins has flavors. But you get the idea.

I have never met a developer who thought his/her/etc programming skills were anything less than stellar. My first engineering class warned us that we engineers have enormous egos. Ironically in the same session we were told half of us wouldn’t be there the following year, which only stoked the smugness of those who survived.

All modesty aside, my observation is that it’s true, engineers tend to be very smart. Any rational person observing the engineered world would have to agree: The products we produce are simply astounding. The complexity of a mobile phone, spacecraft, modern car or any electronic-enabled device beggars the imagination. Yet somehow engineers routinely tame the complexity and ship products that work.

But how good are you, really?

“Good” is a hard metric to define. Is it lines of code per hour? Is it completed projects per year? What about defect rate?

The best developer that ever worked for me could be infuriating. He’d spend most days leaning back in his chair, feet up on the desk, hands clasped behind his head, staring at the ceiling. As a manager I wanted to see him cranking out code. But he was better than that; in these days-long nearly trance-like states he was building the system – in his head. Once he had it all figured out it was a simple matter for him to type the code into the editor. It was like dictation; like Mozart scribbling out a perfectly-formed composition. His code nearly always worked. While others slaved over a hot debugger he did little bug fixing at all.

The embedded world suffers from a dearth of studies, but a lot of data has been accumulated for software development in general. Little of that sees the light of day, or at least it escapes the attention of working engineers as it tends to be buried in mind-numbing tomes like “Applied Software Measurement” by Capers Jones and similar works. These books are full of data and are probably interesting only to metrics wonks. But like the details of the Federal budget the numbers are important. Unless we have a quantitative understanding of our profession it will be more art than science.

It turns out we do know how “good” developers are, at least in terms of defect rates. Programmers inject 120.8 defects per 1000 lines of code, on average. (We also know how many bugs are removed per KLOC, but that’s fodder for a future column.)

That’s a 12% error rate. I have no idea how that compares to other endeavors, but consider writing. Here’s a previous paragraph with 12% of the words being incorrect:

The embedded clock suffers from a dearth of studies, but a lot of fudge has been accumulated for idiot development in general. Little of that sees the light of day, or at least it escapes the attention of working milkshakes as it tends to be buried in mind-numbing pencils like “Applied Software Scissors” by Capers Jones and similar rice pudding. These books are full of vice and are probably interesting only to metrics beards. But like the details of the Federal asphalt the numbers are important. Unless we have a quantitative understanding of our noses it will be more art than science.

That’s pretty incomprehensible.

But the numbers are even more interesting, as there is a distribution. The average is indeed a 12% defect injection rate. But the best 25% of developers are twice as good, at 61.8 defects/KLOC. The best 10% are twice again as good at 28.9. And the very best, the top 1%, inject only 11.2 defects per thousand lines of code.

These numbers apply to experienced developers, not students or newbies and represent analysis of 8000 programs written by 810 developers.

(What’s a defect? It’s anything in the code that has to be corrected. That spans outright bugs to incorrect comments. The latter are often dismissed as unimportant, but reflect a lack of care that is also generally reflective of coding mistakes. I would like to even include spelling errors in the comments, but researcher don’t.)

Engineering is a numbers-driven game. Only a fool would design in a new IC without scrutinizing the device’s numbers in the datasheet. Yet, oddly, too few of us collect metrics about our software engineering performance. Is it just that we’re not in the habit of measuring ourselves? Or, are we afraid of what the numbers might reveal?

I would think that a consulting company could score business by advertising their results, starting with the defect injection rate and defect removal rate. Everyone talks about quality, but absent numbers, it’s all just talk.

What about you? What are your defect injection numbers?


Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges, and works as an expert witness on embedded issues. Contact him at . His website is .

4 thoughts on “Are you a superprogrammer?

  1. “Agree about spelling errors: I once saw a comment where there was such an error and it changed meaning of sentence in a very bad way. I had to read code to fix comment! Lucky nobody did the opposite before…”

    Log in to Reply
  2. “A good friend of mine once observed u201cThere are three things you simply canu2019t tell somebody:n 1.tThey are no good at driving a carn 2.tThey are no good in bedn 3.tThey are no good at writing softwareu201dnnThere are a huge range of

    Log in to Reply
  3. “On the other hand, it's very hard to make code idiot proof. Idiots are very very inventive when coming up with incorrect ways of using a product!nnOf course, I write perfect code (until the first customer tries to do something I didn't think of)!n”

    Log in to Reply
  4. “I wonder, if the defect injection rate by itself will be a sufficient metric. The code in question could be inefficiently written (using far more lines of code than actually required), in which case, the defect-per-KLOC will turn out to be less.”

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.