Estimation is Evil? -

Estimation is Evil?

The always-provocative Ron Jeffries recently wrote a piece titled “Estimation is Evil” . It is part agile manifesto, part screed, and has a lot of decent observations. Unfortunately his conclusions are all wrong. At least in the context of embedded systems, which are, of course, a lot different than most other computer projects.

He starts by saying the worst time to elicit requirements is at the start of the project, since that’s the point in time when we know the least about what we’ll build. The second clause is true, of course. But the conclusion is wrong. Requirements, at least for most firmware projects, have to be nailed down early.

We’ll never get them all, and we’ll never get them all right. Eliciting requirements, at least to around an 80% accuracy level, is not impossible; it’s just hard. Engineering the requirements is a fundamental part of all engineering disciplines. There’s a lot of data we can use to evaluate how much effort should go into up-front requirements and architecture design as a function of program size (e.g., NASA’s Flight Software Complexity briefing). And there are a lot of great tools to aid us.

One of the worst expressions used is “R&D”. There’s no such thing. There’s R – research – when one explores the truly unknown. And there’s D – development – when one has a pretty good idea where we’re going.

In an R environment there cannot be a schedule and there cannot be even an expectation of success, at least in a fashion that will help the company make a profit. Basic science is a great example of R. I have seen so many projects crash because the science wasn’t well understood before development started. The project gets built, but the chemistry, physics, or bio isn’t quite understood so things just don’t work.

If, as many seem to believe, it’s impossible to elicit requirements, we’re not doing development. We’re doing research. Maybe that’s true for designing the next Facebook, but it’s not true for developing a cell phone, MP3 player, or most of the embedded products that fill the world.

Ron says that management typically capriciously shortens estimates, therefore scheduling is a waste of time. He then goes on to say that even a single agile sprint of a couple of weeks will miss that interval’s deadline since management too often overloads the team, demanding a “stretch” goal for the iteration. Thus he oddly seems to lump agile outcomes in the same bin as traditional approaches.

There will always be a tension between management’s schedule needs and the reality provided by engineering. That doesn’t make scheduling evil; it makes the culture – well, evil is too strong a word – but the culture is broken.

His advice is to just starting building something and see how it works out. “Is this a bigger project, possibly taking six months or a year? Then build in two-week iterations for a few cycles. See where you are. If it’s good, go ahead. If it’s not, stop.” That may work in some environments, but most businesses need to operate in a businesslike fashion, and need to make a specific product that can go to market in a reasonable time frame.

Ron does cite some reasons why schedules are important but dismisses them. I disagree; there are hard deadlines. Miss a launch window to Mars and you’ve just lost 20 months till the next one opens. Money is finite; run over budget, especially in a smaller firm, and layoffs are likely. The engineering team does not exist in some ivory tower isolation; their product needs an ad campaign, a production line, a distribution network and a lot more that has to be coordinated. Often these things need to be scheduled well in advance.

Almost all of the embedded firms I’ve worked with do struggle with scheduling. It’s impossible without pretty darn good requirements, and, as I noted, these are hard to elicit. But that doesn’t mean abdicating our responsibility to get them nearly-correct.

Management doesn’t seem to realize that the word “estimate” does not mean “exactimate.” “Estimate” implies a probability distribution function – which is never an infinitely-narrow spike centered on 10:07 AM May 15.

We do a terrible job, in most cases, dealing with feature creep. Changes cost money and/or time. Unmanaged scope changes will destroy the most carefully-produced schedule.

The agile community has made us think hard about traditional development approaches. Much of what they advocate makes a lot of sense. But the idea of just starting to build something and see how things go does not, at least in the embedded space.

Developers sometime complain about the various dysfunctional tendencies of their bosses, especially when it comes to capricious schedules. So my question to them is “How will you behave when you’re the boss?” For your boss is being pressured by his boss, and so forth up the chain. Will you have the strength to fight off the pressures from on high and protect your team?

On another topic, I’m collecting data for a salary survey of embedded hardware and software developers. Managers too. Please take two minutes and fill out the form . If you chose to enter your email address (which is optional) you’ll be entered into a drawing to win one of three copies of my book The Art of Designing Embedded Systems.

Jack G. Ganssle is a lecturer and consultant on embedded developmentissues. He conducts seminars on embedded systems and helps companieswith their embedded challenges, and works as an expert witness onembedded issues. Contact him at . His website is.

6 thoughts on “Estimation is Evil?

  1. No surprise that I agree totally with Jack. Embedded projects are different because we must nail down the processor, which means defining and limiting capabilities, early on. One is parallel development of schematic and PCB layout.
    Another one, often o

    Log in to Reply
  2. I totally agree with Jack and DCH (I'm sorry, Ron…).

    I have been working in engineering firms for the last 23 years and I must include customer's own crazyness and conflicting pressures in the picture.

    For me, having a good set of requirements early on

    Log in to Reply
  3. I can understand Mr. Jeffries points: when one doesn't know what one is doing, and how well it must be done, AGILE is a keyword.

    For some career paths, that may mean obfuscation long enough so you can claim victory, then move on before the truth is discov

    Log in to Reply
  4. Good article. Agile's a bit too trendy these days, and I think many engineering teams find the methodology appealing because it punts on some of the 'hard' stuff. I have no doubt that the principles Mr. Jeffries espouses can be effectively applied, but the

    Log in to Reply
  5. It is a requirements problem. It is always a requirements problem.
    The problem with requirement, is this: if you know exactly what to build, so does everyone else. All the first mover advantages are lost.
    As engineers working on the hardware and software,

    Log in to Reply
  6. I think it is over simplying things to say it is always a requirements problem. It is often also a technical problem. Sometimes we know what to build, just not how to build it.

    Where are the unknowns?

    Frequently unknowns are in the requirements, yes. Of

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.