Verifying embedded software functionality: Why it’s necessary
Economics: the basic software quality driver
Let us illustrate the economic issues that drive interest in software testing and debugging. A report on the “Economic Impacts of Inadequate Infrastructure for Software Testing” published in 2002 by the Research Triangle Institute and the National Institute of Standards and Technology (USA) estimates that the annual cost incurred as a result of an inadequate software testing infrastructure all over the United States amounts to $59.5 billion—0.6% of the $10 trillion U.S. GDP.
Industrial studies on quality control of software have indicated high defect densities. An ACM Crosstalk article reports case studies where on an average 13 major errors per 1000 lines of code were reported. These errors are observed via slow code inspection (at 195 lines per hour) by humans.
So, in reality, we can expect many more major errors. Nevertheless, conservatively let us fix the defect density at 13 major errors per 1000 lines of code. Now consider a software project with 5 million lines of code (theWindowsVista operating system is 50 million lines of code, so 5 million lines of code is by no means an astronomical figure).
Even assuming a linear scaling up of defect counts, this amounts to at least 65,000 major errors. Even if we assume that the average time saved to fix one error usingan automated debugging tool as opposed to manual debugging is 1 hour (this is a very modest estimate; often, fixing a bug takes a day or two), the time saved is 65,000 man-hours (1477 work weeks or 30 man-years.)
Clearly, this is a huge amount of time that a company can save, leading to more productive usage of its manpower and saving of precious dollar value. Assuming an employee salary of $ 40,000 per year, the foregoing translates to $ 1.2 million savings in employee salary simply by using better debugging tools.
A much bigger savings, moreover, comes from customer satisfaction. By using automated debugging tools, a software development team can find more bugs than via manual debugging, leading to increased customer confidence and enhanced reputation of the company’s products. Finally, manual approaches are error-prone, and the chances of leaving bugs can have catastrophic effects in safety-critical systems.
Related Terminology
To clarify the terminology related to dynamic checking methods, let us start with the “folklore” definition of software bug in Wikipedia:
A software bug (or just “bug”) is an error, flaw, mistake, “undocumented feature,” failure, or fault in a computer program that prevents it from behaving as intended (e.g., producing an incorrect result). Most bugs arise from mistakes and errors made by people in either a program’s source code or its design, and a few are caused by compilers producing incorrect code. A program that contains a large number of bugs, and/or bugs that seriously interfere with its functionality, is said to be buggy. Reports detailing bugs in a program are commonly known as bug reports, fault reports, problem reports, trouble reports, change requests, and so forth.
The conventional notion of a software bug is an error in the program that gets introduced during the software construction. It is worthwhile to note that the manifestation of a bug may be very different from the bug itself. Thus, the main task in software debugging is to trace back to the software bug from the manifestation of it. Agood bug report will be able to take in a manifestation of a bug and locate the bug. In case this sounds unclear, let us consider the following program fragment marked with line numbers, written in Java style:
1. void setRunningVersion(boolean runningVersion)
2. if ( runningVersion ) {
3. savedValue = value;
}
else{
4. savedValue = "";
}
5 this.runningVersion = runningVersion;
6. System.out.println(savedValue);
}
Suppose this program is “buggy,” the bug being that the variable savedValue is set to a wrong value in line 4. However, the manifestation of the bug is different— the variable savedValue is printed in line 6, and that is where the bug is indeed manifested. So, naturally there is a “distance” between where the software error is, and where it is observed (possibly via an output or a program crash).
Loading comments... Write a comment