In the mid-1990s, a formal investigation was conducted into a series of fatal accidents with the Therac-25 radiotherapy machine. Led by Nancy Leveson of the University of Washington, the investigation resulted in a set of recommendations on how to create safety-critical software solutions in an objective manner. Since then, industries as disparate as aerospace, automotive and industrial control have encapsulated the practices and processes for creating safety- and/or security-critical systems in an objective manner into industry standards.
Although subtly different in wording and emphasis, the standards across industries follow a similar approach to ensuring the development of safe and/or secure systems. This common approach includes ten phases:
- Perform a system safety or security assessment
- Determine a target system failure rate
- Use the system target failure rate to determine the appropriate level of development rigor
- Use a formal requirements capture process
- Create software that adheres to an appropriate coding standard
- Trace all code back to their source requirements
- Develop all software and system test cases based on requirements
- Trace test cases to requirements
- Use coverage analysis to assess test completeness against both requirements and code
- For certification, collect and collate the process artifacts required to demonstrate that an appropriate level of rigor has been maintained.
Phase 4 is discussed in the main body of this article. As a key topic in many software development standards, this article will not focus on any particular software development standard, but will discuss several standards whose key concepts hinge on a robust requirements capture process.
Every software development standard, across every industry, incorporates requirements capture as a key component to the development process, and justifiably so. Several studies over the past 3 decades, including the “Chaos” reports published by the Standish Institute, seeking to identify the top causes of software project failure have revealed that one of the top causes of failure relates to the requirements elicitation, definition and management processes. Having a robust requirements capture and traceability process should therefore be considered the backbone to a robust software development process. But what are requirements?
There are many detailed technical definitions of the term “requirements”, all based around the concept that requirements are an expression of desired behavior. Probably the most succinct definition derives from a 1979 paper by Barry Boehm entitled Guidelines for Verifying and Validating Software Requirements and Design Specifications . In this paper, Boehm differentiates between validation and verification by defining each via the following questions:
Am I building the right product?”
Am I building the product right?”
Furthermore, he explains that the role of validation is to establish the fitness or worth of a software product for its operational mission. In other words, validation is used to determine whether a software product meets its requirements, so requirements can be defined as an expression of what the “right product” is.
Why are requirements so important? There are several levels of answer to this question. At its most basic, requirements form the foundation for a contract between a supplier and their customer. For the supplier, requirements provide the final reference for determining when the system is actually complete and that scope creep does not occur by implementing more than is expected. During development, requirements also form the baseline reference for verification; i.e. ensuring that the software components have been built right. For safety critical systems where software can cause personal injury (e.g. medical devices developed to meet the standard of the IEC 62304, Medical Device Software – Software Lifecycle Processes ), requirements can also be a key component in defining the limits of a software developer’s liability as in the case of negligence lawsuits regulators will audit development and testing practices that are rooted in the system requirements.
Figure 1: Automate requirements to reduce costs and time. In this specific view, we can see that we want to verify that we have the necessary structural coverage. If the coverage is as expected, then we can observe that the status indicator turns from red (unverified) to green (verified). (Source: LDRA)
As a definition of “right product”, requirements need to encapsulate the needs of all of the stakeholders in a product. However, you can’t design a process that assumes requirements are stable as there will always be some degree of learning what the requirements really are while building the product. This concept is well understood in the development of avionic systems developed in accordance with the DO-178C standard, Software Considerations in Airborne Systems and Equipment Certification . Avionic systems requirements are divided into initial requirements, those that are provided as the original system definition, and derived requirements that are identified while building the product.
Gathering the needs of all of the stakeholders in a product is a process that should be treated with great care and consideration, especially when the stakeholders span international borders. Stories abound about how the assumptions of individual stakeholders have affected the success, or failure, of a given project. For example, NASA lost the $125 Million Mars Climate Orbiter spacecraft when it passed too close to the planet, causing the spacecraft to disintegrate in the upper atmosphere before it reached its operating orbit. The ensuing investigations determined that the cause of the failure was because one engineering team used metric units while another used English units for a key spacecraft operation.
Requirements capture and management tools, from word processors to spreadsheets to purpose built applications, help make the process of documenting and maintaining requirements very straightforward. As the linchpin on which all of the project development is based, it is just as, if not more important, to be able to trace each requirement through the product development lifecycle down to the specific code used to implement the requirement. Similarly, software development standards such as DO-178C state that all software testing be driven from the requirements, so it must also be possible to trace each requirement to the test cases that are used to verify that the requirement has been met. Tying individual requirements to downstream artefacts and activities is the role of requirements traceability; widely accepted as a development best practice to ensure that all requirements are implemented and that all development artefacts can be traced back to one or more requirements.
Figure 2: The amount of requirements traceability tracked varies depending on the risk level of the system. In DO-178C, for instance, the highest risk Level A systems (red line) require traceability right down to the executable object code. Levels B and C (yellow line) ensure bidirectional traceability between high- and low-level requirements and to the source code. Level D (black line) traces between requirements for between system and high-level requirements and then on to the test cases, test procedures and test results. (Source: LDRA)
The traditional view of software development shows each phase flowing into the next, perhaps with feedback to earlier phases, and a surrounding framework of configuration management and process. Traceability is assumed to be part of the relationships between phases; however, the mechanism by which trace links are recorded is seldom stated.
While each individual phase may be conducted efficiently thanks to investment in up-to-date tool technology, these tools seldom contribute automatically to any traceability between the development tiers. As a result, the links between them become increasingly poorly maintained over the duration of projects. The net result is absent or superficial cross checking between requirements and implementation and consequent inadequacies in the resulting system.
To gain true automated traceability requires a Requirements Traceability Matrix (RTM) since the RTM sits at the heart of every project and is a key deliverable within many development standards.
Figure 3: The Requirements Traceability Matrix (RTM) plays a central role in a development lifecycle model. Artifacts at all stages of development are linked directly to the requirements matrix, and changes within each phase automatically update the RTM so that overall development progress is evident from design through coding and test. (Source: LDRA)
An RTM provides traceability between the software architecture and software detailed design, where the software architecture is refined into software units. The RTM helps verify that software units don’t contradict with the software architecture and that all of the static and dynamic verification activities map back to the original requirements.
Using the RTM, the impact of requirement changes and how they affect the system can be estimated. When RTM becomes the center of the development process, it impacts all stages of design from high-level requirements through to target-based deployment. Integrating requirements management and traceability with other tools saves time and avoids rework. Requirements are integrated with defect tracking, visual development and testing tools to jumpstart activities and provide each role on the team with a direct window into end user needs.
For avionics systems development, having a RTM is essential to providing the levels of visibility that are required by the DO-178C standard. This standard requires not only that requirements be traceable down through the code and to the test cases and their results, but also that the reverse is possible. By creating an RTM like the one described in Figure 1, it is possible to get the two-way traceability that the standard requires.
Every software development standard across every industry incorporates requirements capture as a key component to the development process. As a result, it is imperative that they be handled with appropriate care. It is not enough to have a requirements management process that stops at capturing the requirements from the original stakeholders. The process and tools used need to be able to handle the evolution of requirements from the original requirements through derived requirements. To meet the objectives of the most demanding software standards (which also happens to be best practice), it is also necessary to trace the requirements throughout the development process, down to the code and the software verification artefacts. Managing the complexities of this process demands the use of tools designed specifically to address these challenges. Creating and maintaining a Requirements Traceability Matrix (RTM) is key to ensuring that the evolution and application of the captured requirements are managed throughout the software development process.