How embedded projects run into trouble: Jack’s Top Ten – Number Nine
Last week I wrote about the first of the top ten reasons I see for embedded projects crumbling into disaster. Here’s number 9.
The Beatles repeated “number nine” incessantly (for younger readers: The Beatles were a musical group); The Searchers had their Love Potion Number 9. My number nine reason for project failures is a doozy.
9 - Jumping into coding too quickly
In the last two decades I’ve given firmware seminars to 400 companies around the world and have visited countless others for a variety of reasons. One nearly-universal problem I’ve seen is a propensity to start coding as quickly as possible, even well before the functionality and design have stabilized. Yes, in some cases it’s impossible to know much about the nature of a product until a prototype exists, but generally we can get a reasonably-clear idea of what the requirements will be before writing C.
PMI's Pulse of the Profession states that 37% of software project failures are due to inadequate requirements.
In the 1970s my boss came to me and asked me to build a new product. “Uh, what should it do?” His answer: “well, sorta like the last one, but we’ll add features and stuff. You’ve got three months.”
Really? You want me to design something with those pseudo-requirements?
Embedded firmware is a very different sort of beast from other software. We must specify the hardware early in the project, and that can’t be done unless the team has a pretty good idea of what they will build. Get it wrong and costs balloon. Schedules fall apart.
It IS hard to elicit requirements. Sometimes maddeningly difficult. But that doesn’t justify abdicating our responsibility to do as good of a job as possible to figure out what we’re building.
In my travels I see three major reasons why requirements get shortchanged.
First, requirements are boring and hard; coding is fun, so let’s get started now.
This is by far the most common reason coding starts too early. To get them right we’ll have to talk to people – maybe even users. Yuk! It means doing a lot of writing. Who wants to do that? Figure on a lot of hard work to ensure no contradictory requirements.
The schedule must allow for adequate time to figure these out. We do have data on this. The numbers paint a fuzzy picture, but like an impressionist painting we peer through them to get at least a sense of how much time we should devote to this. Fred Brooks, in The Mythical Man Month claims one should spend a third of the project’s person-hours in requirements and design (and only a sixth in coding).
In Flight Software Complexity, NASA, too, found that the effort depends on the project’s size. The round dots are “sweet spots” where the team will get the most benefit:
In The Economics of Software Quality Capers Jones describes what is typical, rather than what is ideal, in the industry. Translating his numbers from function points to lines of C, and rounding to KLOC here are the number of pages of requirements (when they are expressed using English text):
1 KLOC: 14 pages, 97% requirements completeness
10 KLOC: 115 pages, 95% completeness
100 KLOC: 750 pages, 80%
1000 KLOC: 6000 pages, 60%
Again, these are industry averages. Clearly, 60% completeness is not what one would hope for. I did extrapolate to 100% coverage and found that the number of pages is pretty linear with lines of code.
Data Mr. Jones sent me suggest that gathering requirements on a 200 KLOC project consumes, on average, 7.5 person months.
Scary numbers indeed!
The second major reason requirements are not well thought out early in the project is that it’s sometimes impossible to know what they are due to very real unknowns. That’s one of the motivating factors for the agile methods: build something small, get user feedback, then iterate. (While agilists have some deeply-important insights, I object to the deprecation of up-front analysis and design, at least for embedded systems. See Agile! by Bertrand Meyers for a thoughtful analysis of this).
It’s true that on some projects, not many, we really don’t know where we’re going till we build something and get feedback or data.
This leads to a Bad Term we use: R&D. There’s no such thing as R&D. There’s R – research. And there’s D – development based on the results of the R. When people conflate the two I ask for a schedule for the elimination of cancer. R cannot be scheduled as it is a complete unknown, and it’s impossible to schedule an unknown. Too many managers don’t get this and ask us for development costs when there’s still a heavy fog over the product’s functionality.
If some of the requirements are truly a mystery, find a way to solve the mystery. Build a prototype. If there’s confusion about the user interface, maybe prototype it with a friendly tool like LabVIEW. Build it quickly to get early feedback. Suppose the nature of the data isn’t known; get a sample using the smallest framework you can construct.
Do the R, develop the requirements, and then the D.
The final reason requirements are poorly thought-out is the truth that getting them 100% correct is impossible for most projects, so developers give up. Engineers love perfection and are frustrated by compromise.
Perfection is impossible, but that’s no reason to abandon our responsibility to do a really good job. Strive for accuracy but accept the futility of always hitting the bullseye. Get to 95% and then there’s only 5% of the project that will need rework. That sure beats the 60% completeness achieved on big projects.
In school, 90% correctness is an A. That’s true for a lot in life as well.
Next week: Number 8.
The Economics of Software Quality, Capers Jones
Agile!, Bertrand Meyers
The Mythical Man Month, Fred Brooks
Flight Software Complexity, NASA