The cost of quality -

The cost of quality

Though the paper is over a decade old, in my opinion “Safety Critical Software and Development Productivity ” by Oddur Benediktsson has some stunning results.

We all know that building safety-critical software is hugely expensive. When the code must be utterly reliable the effort expended in getting it right skyrockets. That's why avionics and similar products are so expensive.

Except that conventional wisdom may not be entirely true.

In the paper Benediktsson looks at IEC 61508, which is a widely-used functional safety standard. It is commonly used (or is sometimes is the father of similar standards ) in the automotive. nuclear, rail and other industries. It defines four “safety integrity levels” (SILs) with consequences as follows:

SIL1 : Minor injuries at worst
SIL2: Major injuries to one or more persons
SIL3: Loss of a single life
SIL4: Multiple loss of life

The standard lists practices that are recommended, or highly recommended, for each SIL. The use of coding standards, for instance, is highly recommended for every SIL. Formal methods are recommended for SIL2 and 3, and highly recommended for SIL4.

The author relates the effort needed – derived using the Constructive Cost Model (COCOMO) method – to the difficulty of verifying correctness at various safety levels. Unsurprisingly that goes up by rather a lot as one goes from the lower SILs to higher ones. SIL3 is about 1.7 times more effort than a nominal, non-safety-critical product.

He goes on to relate productivity to process maturity, using the Capability Maturity Model as an example model. At CMM4 he finds schedules are almost half those for nominal (NOM) products.

Tying it all together results in this table below:

In other words, as one progresses to higher levels of process maturity (the CI values – NOM is CMM1, CI3 is CMM4 ) the effort to build SIL3 systems is no greater than that needed for non-safety-critical systems.

That's a pretty stunning result.

Or, a dysfunctional company can go to high levels of process maturity and get the same crap for half the price.

Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .

5 thoughts on “The cost of quality

  1. A very good and informative article Jack. While it may be true that higher levels of process maturity will lead to reduction in effort for highly safety critical applications, in practical sense it is difficult to achieve. What I have seen (though not in s

    Log in to Reply
  2. This is a single report written in 2000, based on data collected during the previous 3 decades (ie 1970-2000).

    Is that really a valid body of evidence for making comments on the current state of the industry?

    Much has changed. What was once cheap is now

    Log in to Reply
  3. The people who write the standards actually perform their writing at CMMI level 1 and DO-178B/C level D. In other words, to write the standard they simply begin writing. They have no theoretical basis that the document they write will meet the requirements

    Log in to Reply
  4. I would agree with CDHM. Any software development project prior to the 90s should be discounted. We just didn't have the tools and experience we have now at our disposal. In regards to higher levels of CMMI reducing cost and schedule, one must be extremely

    Log in to Reply
  5. The lesson seems to be that as the product complexity rises, the maturity of development group must rise as well. That can only happen through training, and consequent attention to following through on proven development methods and tools. Defect prevent

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.