All of the reported warnings were inspected manually to determine if they constituted genuine problems. A number of warnings were discarded during this process, if they were determined never to be the direct cause of a device malfunction.

For example, the static code analyzer issues an "Unused value" warning when a variable is assigned a value that is never used. These types of warnings are typically harmless by themselves, but the tool reports them because some safety-critical coding conventions prohibit their use, and because they could be indicative of poor design and maintenance processes.

Warnings like these were discarded, unless they were suspicious for some other reason. A second group of warnings were discarded because they were false positives. As discussed in the previous section, false positives are impossible to avoid in general. However, some of these could be eliminated by choosing the configuration parameters based on domain knowledge.

As a result of this manual analysis, 127 of the 736 warnings reported were found to be of genuine concern; in other words, they either reflected poor quality control or had the potential to cause the device to malfunction. These warnings were submitted as part of a report to the CDRH compliance group to take further action as necessary.

The total effort expended during the post-market analysis was 210 person-hours. A majority of the effort was expended in configuring the build for the application and manual analysis of the results. While 210 person-hours is still a significant amount in terms of the effort required for the analysis, it is considerably less than what would have been required for a purely manual analysis. Additionally, the static analysis method provides for a much more reliable means for tracing errors in the software as opposed to the manual process.

Preventive medicine
Static analysis is a valuable tool for post-market investigation. By reasoning about potential run-time errors in the software, static analysis provides an independent, standardized, and repeatable inspection of a medical device's software, as part of a broader scientific analysis of the device. Further, providing the precise location of the failure and a corresponding execution trace enables the investigator to trace the root cause of failure to its origin in the source code. This ability not only helps reduce time and effort involved in post-market investigation, but also leads to a more accurate means for post-market analysis, as opposed to manual inspection. Most importantly, the use of static analysis allows the post-market investigator to evaluate the product, in this case the software, and not just the processes involved in developing it.

Much as static analysis helps the investigator, it can be leveraged to even greater effect by medical-device manufacturers. The manufacturers can use static analysis to help find flaws early in the development cycle. Static analysis lends itself readily to verification and validation activities and can easily be incorporated as part of the manufacturers' software-development processes. Doing so facilitates a deeper assessment of the code before releasing it to market and helps establish conformance to good programming practices.

On the basis of this experiment, we have reason to believe that static analysis--whether used in pre-deployment analysis by the manufacturer or during post-market surveillance by an investigator--has the potential to greatly reduce software anomalies and lead to safer, more dependable medical devices.

Raoul Jetley is a researcher at the U.S. FDA, Center for Devices and Radiological Health/Office of Science and Engineering Laboratories. His research interests include formal methods and static analysis of medical device software. Jetley received his PhD in computer science from North Carolina State University. Contact him at raoul.jetley@fda.hhs.gov.

Paul Anderson is vice president of engineering at GrammaTech, a spin-off of Cornell University that specializes in static analysis. He received his B.Sc. from Kings College, University of London and his Ph.D. in computer science from City University London. Paul manages GrammaTech's engineering team and is the architect of the company's static analysis tools. A significant portion of his work has involved applying program analysis to improve security. Paul can be reached at paul@grammatech.com.