Software Standards Compliance 101: Tracing code to requirements
In the mid-1990s, a formal investigation was conducted into a series of fatal accidents with the Therac-25 radiotherapy machine. Led by Nancy Leveson of the University of Washington, the investigation resulted in a set of recommendations on how to create safety-critical software solutions in an objective manner. Since then, industries as disparate as aerospace, automotive and industrial control have encapsulated the practices and processes for creating safety- and/or security-critical systems in an objective manner into industry standards.
Although subtly different in wording and emphasis, the standards across industries follow a similar approach to ensuring the development of safe and/or secure systems. This common approach includes ten phases:
- Perform a system safety or security assessment
- Determine a target system failure rate
- Use the system target failure rate to determine the appropriate level of development rigor
- Use a formal requirements capture process
- Create software that adheres to an appropriate coding standard
- Trace all code back to their source requirements
- Develop all software and system test cases based on requirements
- Trace test cases to requirements
- Use coverage analysis to assess test completeness against both requirements and code
- For certification, collect and collate the process artifacts required to demonstrate that an appropriate level of rigor has been maintained.
Phases 6 through 8 are discussed in the main body of this article. System safety and security objectives can only be guaranteed if the original requirements are adhered to, and used as the basis for the software and system level testing. Although presented sequentially in the table above, the phases described here are best started at the time that the system requirements are agreed upon. The principles described here adhere to the mantra “test early, test often”, helping to identify defects and issues as early in the development process as possible, resulting in higher system quality and the lowest possible costs for defect correction.
Standards compliance is not just about checking the code for programming errors. Standards compliance also has a foundation in a solid development process. This article looks at three specific development phases addressed by safety- and mission-critical software development standards:
- Mapping the code under development to requirements
- Generating test cases from requirements
- Mapping test cases back to requirements
From this list, it appears that the code development phases should start only when the code creation process actually starts. But, nothing could be further from the truth. These phases are all about establishing a verification framework that focusses on identifying defects early in the development process in order to reduce the cost of finding and fixing them. These phases create a feedback loop between the requirements, the software being produced, and the test cases used to verify them. The resulting process helps identify defects, ambiguities, and incompleteness early in the product development lifecycle so that they can be addressed in a manner that is both cost-effective and conducive to building quality into safety- and mission-critical systems.
Tracing Code to Requirements
Requirements provide an initial definition of the system under development, providing the reference for how the system is supposed to behave. Once this definition of the system has been translated into code, it is necessary to ensure that the “right system” is built by mapping the code under development back to the original requirements. Not only does this traceability enable project managers to be confident that the features outlined in the requirements are being implemented, but it is especially important for safety- and mission-critical systems as it ensures that the functionality required to meet the system target failure rates has been implemented.
In practice, the traceability process is normally started by mapping requirements to code, but it is the start of the feedback process between the code under development and the requirements. It is, of course, critical that all of the code under development be traceable to the original requirements to ensure that the system be feature complete, but this traceability also helps to ensure that no components are added to the software that are not part of the original system definition.
Extraneous components identified through this traceability process must be evaluated against the original requirements to determine whether they are superfluous to the system design. If they are, then they should be removed or disabled. However, extraneous components identified through this process are not always superfluous. Often they are critical for system functionality, and they therefore become what are referred to as “derived requirements”. As critical components, these components must also be tested to completion, so the requirements documentation must be updated to reflect these derived requirements as they are identified so that test cases can be created for them.
For project managers, this process is also valuable for monitoring development progress. Being able to trace code to requirements helps answer the question, “How much of the system has been completed?” For systems developed under contract, this traceability becomes a critical way to ensure that contractual schedule obligations are met.
An increasing number of software tools work with requirements capture tools to make this process even easier. Figure 1 shows a tool that integrates with requirements capture tools and creates a mechanism not only for mapping requirements to the implementing code but also for assessing the impact of a change in either requirements or code.
click for larger image
Figure 1: A screenshot of TBmanager showing the mapping between requirements and code. (Source: LDRA Technology)