Model-driven development (MDD) is a growing trend within the embedded computing world including safety-critical industries such as defense and avionics. While some modeling work is often involved in the software development process, the key differences are that specialized modeling tools are used and that the resulting model directly determines how the code is implemented.
For our purposes, we will define MDD as a two-step process. Firstly, a graphical representation of the software and its functionality is created. From this model, source code is generated, either manually, automatically, or by a combination of the two.
In the past, MDD was unable to compete in either reliability or efficiency with manually generated code. However, advances in modeling tools have reduced, and in some cases eliminated, the gaps between hand-generated code and code generated from a model. In addition, supporters of MDD cite as advantages its visual approach, simplification of complex systems, potential for component reuse, and overall flexibility.
Despite MDD’s popularity in safety-critical industries, companies that use it often face challenges in achieving safety certification. In part, these challenges are due to the age of the most popular safety standards.
For instance, when DO-178B, the safety certification standard for the avionics industry, was written, few in the avionics industry had heard of MDD. Despite revisions present in DO-178C to address MDD processes, users still face unique challenges in verification, especially with regard to requirements traceability. These challenges pose several important questions: What are the current issues faced by developers who use model-based design but want to maintain requirements traceability for safety verification and how do modern requirements traceability tools assist in overcoming these challenges?
Requirements traceability has become a key part of the safety verification processes in a variety of markets developing critical software, such as aerospace, defense, transportation, nuclear, and medical, but it has taken a special importance in industries where a high risk is involved in software failure.
Requirements traceability connects the highest-level description of the project with lower-level implementation and design. From there the design can be linked to the implemented code and any associated testing. This enforces the idea that every requirement must be implemented and testable. In addition, for DO-178 and other standards, traceability must be bidirectional.
This type of traceability links the tests to the code they verify and the requirements that the code implements. If successful, bidirectional traceability provides a view into the many relationships between the requirements of a system, the code that implements those requirements, and the tests that prove the requirements are met.
The value of requirements traceability is that it provides the means whereby software producers can ‘prove’ to their client that the requirements have been understood; the product will fully comply with the requirements; and the product does not exhibit any unnecessary feature or functionality.
In the context of safety-critical software, requirements traceability takes on an added dimension of proving that software risks have been minimized and all necessary verification activities have been performed. For instance, software failure for an airplane flight controller or a missile launcher can have catastrophic effects. It is vital that implementation of the requirements be proven with concrete and verifiable evidence. For over two decades, requirements traceability has shown itself to be an important means of collecting, organizing and connecting that evidence.
The Challenge of Models and Requirements
Requirements traceability was originally envisioned for a traditional software lifecycle, involving either requirements database tools or textual documents, however. Integrating MDD into a traceability-based workflow is a relatively new challenge. Compounding these complications for safety-critical applications is the lack of guidance provided by safety certification standards.
For example, when DO-178B was developed for the avionics industry, MDD technologies were not a factor and no guidance was provided. Newer updates to these documents include directions designed specifically to address this issue. There were many lessons learned in the interim time about how MDD should be integrated with requirements traceability. The latest updates take these into account and provide guidance on avoiding some of the potential dangers of incorrect integration.
Fitting the Model into the Requirements Mold
An issue that arose in many companies within the avionics industry, and which DO-178C clarified, was exactly how design models fit within the requirements traceability methodology. Since models can represent a high-level overview of the project, some companies treated them as high-level requirements. Others treated models as source-code since they can be used to generate the actual implementation.
The approach approved for DO-178C, and which likely makes the most sense from the viewpoint of requirements traceability, is that design models are low-level requirements. This means that models should be treated as design documents that show how the implementation is to be performed. It is important to treat design models in this manner to allow for more accurate verification (Figure 1 below ).
Without establishing a high-level definition of the system, it is difficult to ensure that the model is functioning as desired. It is also important to avoid treating the model as the actual implementation. This ignores possible defects introduced by the code generation process, external code elements (hand code) added to the generated code, and verification in the target environment.
Clickon image to enlarge.
Treating models produced by MDD as sets of low-level requirements has important implications for the requirements traceability process. For full requirements traceability to be satisfied, all high-level requirements must be implemented in lower level requirements, and for bidirectional traceability, all low-level requirements must be relatable to higher level requirements.
Moreover, for the traceability to be meaningful the links must be between the actual model elements and the requirement from which it is decomposed. It is tempting to link all high-level requirements to an opaque model (a top-level model with no elements visible to the traceability process). Doing so ignores the purpose of traceability (Figure 2, below ).
Only the portion of the model which represents a given requirement can provide evidence that the requirement has been realized in the design. If a requirement has not been realized, it is important to be able to isolate the exact model element at fault.
Clickon image to enlarge.
Achieving traceability between the model and the higher level requirements can be challenging, especially since the two are usually stored and presented in very different ways. High-level requirements are predominantly textual in nature, while most modeling tools provide graphic interfaces for design. Modern requirement traceability tools can bridge this gap, by pulling high-level requirements and model elements from their respective repositories. These components can then be linked and managed alongside other traceability linkage, such as the connections between requirements and functional tests.
The central innovation of MDD is a separation between the development and coding process. This separation, however, can cause issues with requirements traceability. If the development and coding processes exist independently, establishing traceability for both can be complicated and error-prone. Requirements traceability is meant to be a continual effort spanning the entire software development lifecycle. If implementation and design are not linked, traceability work performed during design needs to be repeated during implementation.
The best-of-breed modeling tools do mitigate this danger somewhat. Many modeling tools provide means of establishing automatic connections between the model elements and their corresponding code. While that automatic mapping works in theory, many modeling tools are combined with customized build processes, additional hand-written code, and custom code generators that make the connections between code and model more obscure.
Even when traceability between code and model has been utilized, there may be disconnects that exist between testing occurring in the model and testing source code in the target environment. In the MDD paradigm, it is important to verify model behavior according to the high-level requirements, and ensure that tests exist to prove that model execution.
One of the clarifications made by the DO-178C MDD supplement is to specify a need for model-verification as part of the overall software verification process. To achieve this level of verification, modeling tools provide mechanisms to test the model within the modeling environment.
While this testing is useful for verifying the design, it does not necessarily constitute actual executable object-code verification. This distinction between source-code level testing and model-level testing means that traceability connections between requirements and model tests does not automatically count for traceability between source-code tests and requirements. Again, the typical solution to this problem is to create parallel traceability chains for source-code tests and model-based tests, leading to redundant testing and the additional complexity of manually integrating the testing results.
It is possible to mitigate the disconnect between model-based design and source code verification. Requirements traceability tools can be utilized that are able to pull requirements and traceability evidence from multiple sources. By gathering together low-level requirements that are represented as model elements and high-level requirements from external documents or databases, traceability links can go directly to the design elements linked to source code.
Moreover, results from both source code testing and model-based testing can also be presented together. Requirements traceability tools allow traceability evidence to be organized as one seamless flow, joining requirements, lower-level requirements, design, source code, and tests. This helps to avoid the added complexity of maintaining and integrating two separate traceability efforts (Figure 3 below ).
Clickon image to enlarge.
In addition, separate testing requirements can be reduced by utilizing test-case reuse. This involves isolating the input and output information used in the model-based testing, and then exporting it to a format that can be read by source-code testing tools. With the proper verification and modeling tools, this process can even been automated.
What is important from the traceability side is that traceability evidence can be re-used. If certain test-case parameters were already shown to be traceable to a requirement, these parameters can then be re-applied on the source code level and the same justifications and artifacts used in the requirements to model test linkage can be used in the requirements to source code test linkage.
Despite its popularity MDD has presented safety-critical applications with several new challenges. One issue of particular importance has been reconciling requirements traceability with a model-driven development approach. MDD blurs many of the traditional lines between design, requirements, and implementation, and so it can be easy to misplace the model in the traceability process.
In addition, model-based design removes the design process from the source-code level testing process, which creates the danger of duplicate and erroneous traceability work. Underneath these new challenges is the question of how to integrate a new method of software development into the critical software development practices. The answer of using innovation and best practices applies here as well.
The addition of the model-based development and verification supplement in DO-178C shows that work is being done to develop best practices for incorporating models into lifecycle traceability. Modern traceability and testing tools are actively innovating to simplify and automate this process.
Requirements traceability, especially when done with an unclear understanding of the traceability process, can be painful. When done right, requirements traceability can provide verification on all levels of the software development lifecycle. The gains of model-based design cannot come at the cost of safety, but with the proper tools, no trade-off is necessary.
John Thomas is a Field Application Engineer for LDRA Ltd. He graduated from Rutgers University. Before working for LDRA, John worked as a developer for several companies and was previously Project Manager for Castel Enterprises, Ltd. He joined LDRA in 2011 and has worked with LDRA clients on coverage testing, target integration and requirements traceability.
Jared Fry is a Field Application Engineer for LDRA Ltd. He graduated from Western New Mexico University with degrees in mathematics and computer science. His career began in the defense industry working for Lockheed Martin. There, Jared served as a software engineer working on various projects ranging from missile and radar systems, training simulations, and software testing. With LDRA he leverages these experiences as a consultant, assisting clients throughout the development process to produce a quality and certifiable product.