Think static analysis cures all ills? Think again.
Static code analysis has been around as long as software itself, but you'd swear from current tradeshows that it was just invented. Here's how to choose the right code-analysis tools for your project.
Static analysis (or static code analysis) is a field full of contradictions and misconceptions. It's been around as long as software itself, but you'd swear from current trade shows that it was just invented. Static analysis checks the syntactic quality of high-level source code, and yet, as you can tell from listening to the recent buzz, its findings can be used to predict dynamic behavior. It is a precision tool in some contexts and yet in others, it harbors approximations.
With this extent of contradiction, it's hard to believe that all of these statements are accurate. Static analysis, a generic term, only indicates that the analysis of the software is performed without executing the code. So, simple peer review of source code fits the definition just as surely as the latest tools with their white papers full of various incantations of technobabble.
There isn't much point in any such analysis existing in isolation since even if code is perfectly written, it's only correct if it meets project requirements. It's therefore also important to understand how well any such analysis fits within the development lifecycle.
No analysis is good or bad simply by virtue of being static or dynamic in nature. It follows that each analysis tool is neither good nor bad or perhaps more pertinently, appropriate or inappropriate just because they're statically or dynamically based. It's, then, important to look past the subtle advertising and self-congratulatory white paper proclamations to consider the relevant merits and demerits of static analysis and its ability to predict dynamic behavior. Can a solid static-analysis engine bypass the need for dynamic analysis? In this article, I explore current technologies and explain how static analysis predicts dynamic behavior. This article will help developers understand which method to use under which circumstances.
We'll look specifically at five key attributes of analysis tools, shown in the sidebars.
Key attributes of static-analysis tools
1. Automated code review—automates the peer review process to enforce coding rules dictating coding style and naming conventions and to restrict commands available for developers to a safe subset. Code review doesn’t predict dynamic behavior, except to the extent that code written in accordance with coding standards can be expected to include fewer flaws that might lead to dynamic failure.
2. Formally-defined language—(such as SPARK Ada) defines desired component behavior and individual run-time requirements. This may be in the form of specially formatted comments in the native language that are ignored by a standard compiler but can be statically analyzed to show that the program is “well-formed,” consistent with the design information included in its annotations, and has certain properties specified in those annotations. The annotations therefore make it possible to precisely predict dynamic behavior via static analysis. The Larch/C++ approach is similar in concept and uses a predicate-oriented interface language.
3. Prediction of dynamic behavior through static analysis—models the high-level code to predict the probable behavior of the executable that would be generated from it. This approach builds an approximate mathematical model of the code and then simulates all possible execution paths through that model, mapping the flow of logic on those paths coupled with how and where data objects are created, used, and destroyed. This approximation is used to predict anomalous dynamic behavior that could possibly result in vulnerabilities, execution failure, or data corruption at run time.
The first three are attributes of static-analysis tools. Notably, these attributes don't comprehensively describe the categories of static-analysis tools, and many tools include more than one of these attributes.
The issue of what's static and dynamic analysis is further confused when there is a requirement to predict dynamic behavior. At that point, the dynamic analysis of code that has been compiled, linked, and executed offers an alternative to the prediction of dynamic behavior through static analysis.
Dynamic-analysis tools involve the compilation and execution of the source code either in its entirety or on a piecemeal basis. Again, while many different approaches can be included, these characteristics complete the list of the five key attributes that form the fundamental "toolbox of techniques."
Key attributes of dynamic-analysis tools
4. Execution tracing (or code coverage analysis)—details which parts of compiled and linked code have been executed often by means of software instrumentation probes that are automatically added to the high-level source code before compilation.
5. Unit testing—snippets of software code are compiled, linked, and built in order that test data (also called “vectors”) can be specified and checked against expectations. Unit testing can be extended to include the automatic definition of test vectors by the unit test tool itself.
Test-tool vendors offer a plethora of combinations of these key static and dynamic attributes and claim their particular combination to be invaluable to the efficient and effective development of software.
Despite their lofty claims, no single vendor touts an offering that embraces all of these attributes. And, attempting to apply every one of the five techniques through a combination of tools would usually be prohibitively expensive both in terms of capital investment for the tools and labor costs for software testing.