Software Standards Compliance 101: Critical role of documentation
In the mid-1990s, a formal investigation was conducted into a series of fatal accidents with the Therac-25 radiotherapy machine. Led by Nancy Leveson of the University of Washington, the investigation resulted in a set of recommendations on how to create safety-critical software solutions in an objective manner. Since then, industries as disparate as aerospace, automotive and industrial control have encapsulated the practices and processes for creating safety- and/or security-critical systems in an objective manner into industry standards.
Although subtly different in wording and emphasis, the standards across industries follow a similar approach to ensuring the development of safe and/or secure systems. This common approach includes ten phases:
- Perform a system safety or security assessment
- Determine a target system failure rate
- Use the system target failure rate to determine the appropriate level of development rigor
- Use a formal requirements capture process
- Create software that adheres to an appropriate coding standard
- Trace all code back to their source requirements
- Develop all software and system test cases based on requirements
- Trace test cases to requirements
- Use coverage analysis to assess test completeness against both requirements and code
- For certification, collect and collate the process artifacts required to demonstrate that an appropriate level of rigor has been maintained.
Phase 10 is discussed in the main body of this article, the last in this standards series.
There is a famous cartoon amongst aerospace engineers titled “Dream Airplanes” by C. W. Miller that is featured in the L. M. Nicolai book Fundamentals of Aircraft Design (METS Publishing, 1975). In this illustration, an aircraft design is presented from the perspective of the many engineering disciplines that are required to make such an aerospace project possible. From the perspective of the Wing group, the aircraft design is dominated by oversized wings, while for the Power plant group everything is built around an unfeasibly large engine. The Aerodynamics group’s perspective is a sleek, drag-free design that would be impossible to build, while the Stress group would rather the design consist of a series of steel girders.
The point of this diagram is to enforce the fact that it takes multiple disciplines to make an aerospace project successful, and no one discipline can dominate; there has to be a balance. Like an aircraft project, safety-critical embedded systems built to comply with industry standards are really an amalgamation of many different disciplines, from system safety concepts to process and software engineering, with a particular emphasis on building quality into the final work product. Nowhere is this amalgam more evident than in the project lifecycle documentation that encapsulates the work products that are produced at each phase in the development lifecycle. In addition to the final software product, the certification authorities depend on project documentation to ensure that the software was produced in an acceptable manner to meet or exceed the minimum safety criteria objectives for the project.
This article looks at some examples of the documentation produced during the development lifecycle for safety-critical software, and how the documentation work products from each phase affect and are affected by the work products from other phases. We’ll also look at the role that documentation plays in the certification process for different standards. In particular, we’ll focus on how the use of software tools throughout the development process can streamline not only the document generation process, but also the validation and verification processes that occur as the project evolves.
Documentation starts long before development
Contemporary civil engineering is responsible for an ever-increasing number of engineering marvels, often in places where it was not thought possible. Good examples include the highest bridge in the world, the Millau Viaduct in Southern France, or Burj Khalifa—the tallest building in the world that was erected in Dubai in the United Arab Emirates. Long before construction started on these projects, a significant amount of documentation was generated, and continues to be archived and maintained even after construction is complete. Focusing specifically on the technical side, examples of the documentation required for this type of construction project include:
- Geological surveys
- Geological mitigation designs
- Architectural design
- Structural design
- Mechanical design (utilities)
- HVAC design
- Fire-detection and -suppression system design
It would be unimaginable that even simple construction projects would get off the ground without much of this documentation already being in place. Without them it would be impossible to define what to build, how much it would cost, and how long it would take. The same goes for software.