Making source code analysis part of the software development process
Modern source code analysis tools (sometimes called static analysis or SCA tools) analyze software programs at the earliest stage of development. SCA tools analyze a program to calculate metrics and find potential flaws and defects in the code.
Unlike tools of the past, which tended to do simple pattern matching, modern SCA tools (Figure 1, below perform sophisticated path and data flow analysis and can find surprisingly meaningful bugs with good accuracy.
|Figure 1: Source code analysis detects problems in the earliest part of development|
From a business perspective, SCA tools hold a lot of promise. By uncovering problems in the earliest part of the development process, SCA tools can dramatically lower the cost of quality and security for a product.
The effort required to get to value is relatively low. For most organizations, just a few hours of analysis will uncover hundreds to even thousands of potential defects. No testcases are required, and the reported defects literally point to the line or lines of code where a problem can occur.
While SCA tools are an easy sell for most organizations, in practice, users often struggle with the tool. Unrealistic expectations coupled with underinvestment result in failed or suboptimal deployments.
Instituting any toolchain change in an organization requires more than just installing it and sending out the URL to the developers. Real change requires effective planning and execution. Instituting SCA is no different.
Everyone has opinion and experience with SCM tools and bug tracking systems. Much fewer have experience with SCA tools. Because modern SCA tools are relatively new, the industry does not have a plethora of reference architectures by which to fall back upon.
What I hope in this article is to convey some of the hard lessons we've learned by working with a number of companies instituting SCA tools. We hope you don't make these mistakes when instituting change in your environment.
This is where most organizations are set up for failure. The majority of times, the goal is not even stated or agreed upon. What defines success and what defines failure? With a clear idea in mind all of the process, organizational structure and supporting technical infrastructure can fall into place.
While everyone will agree that SCA tools will "improve quality by finding bugs earlier in the process," realistic goals need to be instituted to make clear what is expected. Some of the ones we've seen are:
* 100% clean software before release
* No new defects being introduced for each Agile development sprint
One often forgotten mantra is that the value in the tools is not in the number of bugs you find, but in the number (and quality) of the ones you fix. Identifying and addressing the ones worth fixing is the focus.