Inspections prior to test

One of the most powerful tools to improve the quality of firmware is the code inspection. They also speed development. A lot of data supports the latter statement; for example, in A Survey of Software Inspection Technologies by Oliver Laitenberger, the author shows that Jet Propulsion Laboratory’s inspections are ten to 34 times cheaper than testing. A wealth of other studies confirm this.

Worse, test doesn’t work. V. Basili and R. Selby, in Comparing the Effectiveness of Software Testing Strategies , showed that test is unlikely to reveal more than half of the defects in a body of code. Glass and others have confirmed these results. (There are exceptions; disciplined code coverage will improve the numbers).

In my travels this Autumn I’ve had a chance to visit several companies that do disciplined inspections, which is laudable, and unfortunately uncommon. Interestingly, the companies I’ve recently visited do the inspection before final commit to the version control system. That is, the developers write and test the code, fix the bugs they find, and then inspect it.

That’s a mistake.

Inspections should take place after cleaning up errors found by automated tools, like the compiler, lint, and static analysis. No test should be done pre-inspection.

It’s completely understandable that developers wish to test first. Who wants their mistakes pointed out in a meeting of their colleagues? Sneak in a couple of hours of test to get rid of the smack-your-forehead-obvious mistakes.

But inspections are cheaper – much cheaper – than test, so test first is akin to burning stacks of hundred-dollar bills. It’s simply bad business.

A central observation of the quality movement is that you can’t test quality into a product. That’s also true in software; a focus on fixing bugs will not lead to quality code. Quality must be designed in from the beginning.

When inspections are done pre-test, the code automatically improves. If I’m writing code that no one will look at it’s quite different from that which will go through an inspection. And that’s part of the goal of the process: encourage the developers to write correct code from the outset. Have them work harder during the coding process, rather than devoting eons of time to fixing all of the bugs – or, at least, all of the ones they can find.

In The Economics of Software Quality , By Capers Jones and Olivier Bonsignour, the authors report that out of 121 known ways to improve software quality, the fifth most effective is pretest inspections.

To repeat: a focus on fixing bugs will not lead to quality code.

We need to break the cycle of writing a lot of code fast and then getting into the tar pit of endless test. Think first. Code carefully. Examine the results. Make it as perfect as possible. Feed it through the tools and fix any errors detected. Inspect it and fix those defects.

Finally, test.

Jack G. Ganssle is a lecturer and consultant on embedded developmentissues. He conducts seminars on embedded systems and helps companieswith their embedded challenges, and works as an expert witness onembedded issues. Contact him at . His website is.

9 thoughts on “Inspections prior to test

  1. “Where I am currently working we do the inspection before the commit. It works.nWatts Humphreys in the Personal Software Process recommends inspecting your code before compile time. Yes, write, inspect then type make. He said you should run the compiler o

    Log in to Reply
  2. “Jack,nnInspection certainly is useful and has an important role to play. nnHowever you don't do your case any favours by refering to Selby's 1987 paper.nnTools, and the complexity of the software being testsed have both moved on since then, renderin

    Log in to Reply
  3. “”K&R C was terrible and quite unlike what we call C today. It was frankly dangerous as a programming tool.” nnNot so! It just requires programmers who know what they are doing and treat powerful tools with respect.”

    Log in to Reply
  4. “Having other people review a document before publishing it is certainly a very good idea. But it is important that the document has been automatically spell-checked and reviewed/revised multiple times by the author herself. In other words: it must be fre

    Log in to Reply
  5. “I completely agree that test — the activity that happens after code is written to find out what works — is not a good way produce quality software.nnOn the other hand, test-driven development helps to prevent defects. Unfortunately the word test is i

    Log in to Reply
  6. “It seems embedded.com could use some TDD. I logged in for the first time in a while to weigh in on this article. My profile is long gone. I entered a new one, but it won't accept my bio. The website does not show my name.nnThis is exactly the kind o

    Log in to Reply
  7. “”Finally, test”nHaving just read this article once again, I must admit that it is so waterfally, it almost makes me think that it came through a rupture in space-time from the 20th century. It is so oblivious of all the achievements of modern software

    Log in to Reply
  8. “Yes Ralf, I'm totally on your page. Designing, prototyping, reviewing, testing, refining, maturing, reviewing or verifying again etc is an interative process where you always go for the discipline you need.Policies (number of developers er task, r

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.