A whole industry has developed around verification and validation practices that are championed by functional safety, security, and coding standards such as IEC 61508, ISO 26262, IEC 62304, MISRA C, and CWE. Of course, not everyone is obliged to follow the formal processes and methodologies these standards promote, especially if their software doesn’t need to meet the rigors of these standards. But the standards champion best practices because experience says that they represent the most effective way of achieving high quality, reliable, and robust software.
Best-practice development techniques that follow these standards help ensure that errors aren’t introduced into code in the first place, reducing the need for extensive debugging activities that can slow time to market and add costs. Of course, not all developers have the luxury of the time and budget afforded to the applications seen in the aerospace, automotive, or medical device industries. The techniques they deploy, however, represent a toolbox with huge potential benefits for any development team, whether criticality enforces their use or not.
Types of Errors and Tools to Address Them
Two key types of errors can be found in software and addressed using tools to prevent errors from being introduced:
- Coding errors. An example is code that tries to access outside the bounds of an array. These kinds of problems can be detected by performing static analysis.
- Application errors. These can only be detected by knowing exactly what the application is supposed to do, which means testing against the requirements.
Coding Errors and Code Review
Static analysis is an effective technique for detecting coding bugs, especially when it is deployed from the start of a project. Once the code has been analyzed, there are different types of results that can be viewed. Code review is where the code is checked against a coding standard such as MISRA C:2012, which is what we’ll focus on in this article.
Ideally a safe language such as Ada would be used for all embedded projects. Ada includes numerous characteristics to enforce a thought process that naturally reduces errors (such as strict typing, for example). Unfortunately, it is difficult to find programmers with Ada knowledge and experience, so the majority of companies instead use C and/or C++. These languages, however, present pitfalls even for experienced developers. Fortunately, by performing code review most of these potential pitfalls can be avoided.
The best way to avoid defects in code is to avoid putting them there. This sounds obvious, but this is exactly what a coding standard does. In the C and C++ world, around 80% of software defects are caused by the incorrect usage of about 20% of the language. If the use of the language is restricted to avoid the parts of the language that are known to be problematic, then defects are avoided and the software quality greatly increases.
The fundamental language-related causes of failures with the C/C++ programming languages are undefined behavior, implementation-defined behavior, and unspecified behavior. These behaviors lead to software bugs and security issues.
As an example of implementation-defined behavior, consider the propagation of the high-order bit when a signed integer is shifted right. Is the result 0x40000000 or 0xC0000000?
Figure 1: The behavior of some C and C++ constructs depends on the compiler used. (Source: LDRA)
The answer depends on which compiler you are using (Figure 1). It could be either. The order in which the arguments to a function are evaluated is unspecified in the C language. In the code shown in Figure 2—where the rollDice() function simply reads the next value from a circular buffer holding the values “1, 2, 3 and 4”—the expected returned value would be 1234. There is, however, no guarantee of that and at least one compiler will generate code that returns the value 3412.
Figure 2: The behavior of some C and C++ constructs is unspecified by the languages. (Source: LDRA)
The C/C++ languages present plenty of pitfalls like this, but with the use of a coding standard, these undefined, unspecified, and implementation-defined behaviors can be avoided. Similarly, the use of constructs such as goto or malloc can lead to defects, so a coding standard can be used to prevent these constructs from being used. Many problems occur when mixing signed and unsigned values, which generates no issue most of the time but there may sometimes be a corner case where the signed value overflows and becomes negative.
Coding standards can also check that code is written in a particular style; for example verifying that the tab character is not used, that the indentation is a specific size, or that parentheses are positioned in a specific position. This is important since some manual code review will be required and when the code is viewed in a different editor where the tab character has a different size, then the strange layout distracts the reviewer from concentrating on reviewing the code.
Some developers are guilty of writing “clever” code that may be highly efficient and compact, but may also be cryptic and complex, making it difficult for others to understand. It is much better to keep it simple and let the compiler take care of making an efficient binary. Once again, the use of a coding standard can help prevent developers from creating undocumented and over-complex code.
The best-known programming standards are perhaps the MISRA standards, which were first published in 1998 for the automotive industry. The popularity of these standards is reflected in the number of embedded compilers offering some level of MISRA checking. The latest version of MISRA is MISRA C:2012, which has almost double the number of pages of its predecessor. Most of this additional documentation consists of useful explanations about why each rule exists, along with details of the various exceptions to that rule. MISRA has several guidelines and when applicable, they contain references to standards or to the undefined, unspecified, and implementation-defined behavior. An example of this can be seen in Figure 3.
Figure 3: MISRA C references to undefined, unspecified, and implementation-defined behavior. (Source: LDRA)
The majority of the MISRA guidelines are “Decidable,” which means that a tool should be able to identify whether there is a violation or not. However, a few guidelines are “Undecidable,” meaning that is it not always possible for a tool to deduce whether there is a violation or not. An example of this is when an uninitialized variable is passed as an output parameter to a system function that should initialize it. However, unless the static analysis has access to the source code for the system function, then it is unable to know if that function uses the variable before it initializes it. If a simple MISRA checker is used, then it might not report this violation, possibly leading to a false-negative. Alternatively, if a MISRA checker is unsure then it could report the violation, possibly leading to a false-positive. What is best? Not knowing that there might be a problem? Or knowing exactly where to spend time ensuring that there definitely is not a problem? Surely it is preferable to have false-positives rather than false-negatives.
In April 2016, the MISRA Committee issued an amendment to MISRA C:2012 that added an additional 14 guidelines to help ensure that MISRA was applicable not just for safety-critical but also security-critical software. One of these guidelines was Directive 4.14, which, as can be seen in Figure 4, helps to prevent pitfalls due to undefined behavior.
Figure 4: MISRA and security considerations. (Source: LDRA)
Application Errors and Requirements Testing
Application bugs can only be found by testing that the product does what it is supposed to do, and that means having requirements. Avoiding application bugs requires both designing the right product, and designing the product right.
Designing the right product means establishing requirements up front and ensuring bidirectional traceability between the requirements and the source code so that every requirement has been implemented and every software function traces back to a requirement. Any missing or unnecessary functionality (that doesn’t meet a requirement) is also an application bug. Designing the product right is the process of confirming that the developed system code fulfills the project requirements, which can be achieved by performing requirements-based testing.
Figure 5 shows an example of bidirectional traceability. In this simple example, a single function has been selected, and upstream traceability is highlighted from the function to a low-level requirement, then to a high-level requirement, and finally to a system-level requirement.
Figure 5: Bidirectional traceability, with function selected. (Source: LDRA)
In Figure 6, a high-level requirement has been selected, and highlighting shows both upstream traceability to a system-level requirement and the downstream traceability to low-level requirements and on to source code functions.
click for larger image
Figure 6: Bidirectional traceability, with requirement selected. (Source: LDRA)
This ability to visualize traceability can lead to the detection of traceability issues (application bugs) early in the lifecycle.
Testing code functionality demands an awareness of what it is supposed to do, and that means having low-level requirements to state what each function does. Figure 7 shows an example of a low-level requirement, which, in this case, fully describes a single function.
Figure 7: Example low-level requirement. (Source: LDRA)
Test cases are derived from low-level requirements as illustrated in Table 1.
Table 1: Test cases derived from low-level requirements. (Source: LDRA)
Using a unit test tool, these test cases can then be executed on the host or the target to ensure that the code behaves in accordance with the requirements. Figure 8 shows that all the test cases have been regressed and passed.
Figure 8: Performing unit tests. (Source: LDRA)
When the test cases have run, then the structural coverage should be measured to ensure that all the code has been exercised. If the coverage is not 100 percent then it is possible that either more test cases are required or that there is superfluous code that should be removed.
With increasing software complexity, potential software errors also increase. Best-practice development techniques help prevent these errors from occurring. Best-practice development consists of using a state-of-the-art coding standard such as MISRA C:2012, measuring metrics on the code, tracing requirements, and implementing requirements-based testing. The extent to which these techniques are applied where there are no obligations to meet standards is clearly at the discretion of the development team. However, the standards champion these practices because experience says that they represent the most effective way of achieving quality, reliable, and robust software. And whether a product is safety-critical or not, that is surely an outcome that can only be beneficial to its development team.
|Mark Richardson has over 40 years of experience in the development of real-time embedded software in C, C++ and Java. He is currently working for LDRA, where he is a Lead Field Application Engineer working in close collaboration with LDRA’s numerous distributors. Prior to joining LDRA, Mark was the lead application engineer for IBM Rational Rhapsody and has over 12 years of experience using UML on embedded projects. He has lived in the UK, France and the USA, working on a variety of embedded projects. Mark currently lives in the UK.|