The show floor at the Embedded Systems Conference is crowded with exhibitors showing all sorts of wares. Booths with glittering gadgets employing billions of transistors sit next to consultancies from third world countries peddling their services. An army of attendees prowl the floor looking for solutions to their problems.
Possibly the biggest problem we face is that of building software that actually works, an effort that gets more intractable every year as firmware grows exponentially in size. A decade ago it was hard to find projects exceeding a few hundred thousand lines of code; today millions aren’t uncommon. The usual heroics work pretty well for smaller projects but don’t scale for efforts requiring large teams.
I’m a great fan of tools – anything that can automate or ease a developer’s work. Especially interesting are static analysis tools. Give me a program with two buttons: “find bug” and “fix bug.” Though such alchemy is not possible today, we’ve long had syntax checkers, Lint, and other contraptions that do find some problems.
A new and evolving class of static analyzers is being shown by at least two companies at the ESC. Products by Polyspace Technologies and Coverity do deep analysis of source code to find those hard-to-isolate problems. Though I’ve yet to take either of these offerings for a test drive the claims made by the vendors are interesting.
Coverity’s products find null pointer dereferences, memory leaks, buffer overflows and much more, all without running the code, all just by analyzing the source tree. The salesman told me that, on average, the tools uncover about 1 bug per thousand lines of code analyzed. That might not seem like much… but since some defects might take days or weeks to find, the benefit is clear. That’s 1000 bugs per megaline; think of the time required to find and fix a thousand bugs! And when a single bug can result in a huge product recall, this sort of insight has vast benefits.
Polyspace’s tools do similar checks, and also find array overflows and other sorts of runtime errors. A new feature lets the user specify min and max ranges of global variables; the tool then insures none exceeds these specifications. All without actually running the program.
These tools perform insanely complex calculations requiring lots of compute time. I’m told Coverity’s products can dissect 40 million lines of code in 3 or 4 hours. Polyspace’s are apparently targeted at smaller projects, and burn something like an hour for a 20 KLOC project. Clearly you wouldn’t invoke either tool from the make file, but it’s reasonable to run a nightly build that includes the tests.
The cost? Coverity prices their product based on the number of lines of code you’ll being examining. Polyspace has a flat $5k per seat price tag. It seems to me that the tools could pay for themselves after finding just a couple of tough bugs.
Yet developers have nearly no budget for software quality tools. I ran a survey where 80% of the respondents complained they’re unable to get more than $500 for a static analyzer or other quality-enhancing product. Yet our EE brethren routinely snag $50k logic analyzers and similar equipment. But accounting can put a property tag on that gear.
What do you think? Is static analysis a valuable idea? Is it worth the money?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .
In a similar vein, I've just been collating results from using thePolySpace Verifier (www.polyspace.com) on a small embedded C project,~8 kSLOC, 4 man years of effort. We reckon that it has saved up to$35k, and 4 man months of bug fixing effort. We also now have /proof/that our code will /never/ raise any run-time exceptions, or exceedany range of values that it shouldn't.
I should add that the project shipped on the due date and to95% of budgetted cost. Better can mean cheaper too!
I'm a long time reader of your column. Thanks for all the years of thought provoking items and insight into the industry.
I had to respond to your recent column regarding departmental budgets. I am an EE, who worked in hardware design, embedded software development, and also SALES!!, most recently.
When I went into sales, I was amazed at the difference in treatment and respect that I received, compared to being an engineer.
You wanna talk about budget???
We would have three day sales conferences in Las Vegas, all expenses paid etc. I attended dinners and parties from our partner corporations that cost many thousands of dollars with nary a product engineer in sight.
I spent a ton on airfare, hotels and meals without scrutiny…. as long as I made my numbers…. which I did.
When I was an engineer for a medical devices company, I wanted to buy an ICE for my project. I sat in front of the CFO with a $20,000 requisition for three emulators plus debuggers, etc. and was asked if I really needed those toys!
The next time an engineer has to requisition capital for debuggers, whatever, if the exhaustive ROI you provided doesn't do the trick, try grilling the CFO on how much the company spent on the last sales meeting. Then, ask him what the ROI was for that expenditure!
– Steve Reese