Testing code that uses an RTOS API

Embedded application software, which is designed to work under the control of a real time operating system, presents an interesting debugging and testing challenge. The code is most likely littered with RTOS system API (Application Program Interface) calls, which need to be verified, along with the logic applied to any response received. Ideally, the testing process would involve linking the application to the RTOS and debugging. However, this introduces a number of other unknowns and necessitates a target execution environment (which may or may not be the final hardware). It would be useful if this testing could simply be done on a host computer, as PCs are readily available. This article explores an approach to making progress in testing such code by running it natively and using a “test harness”.

The concepts discussed here arose after a talk about OS-aware debuggers at a conference and someone asked whether there is a good technique for unit testing of code for a multi-threaded application. They were considering an environment where a number of engineers were working on an embedded application (using Nucleus RTOS). Each guy was developing one or more tasks, which interact with one another and those written by other engineers. The questioner was wondering how these engineers could make some solid progress with testing and debugging ahead of building the complete system.

Unit Testing
Obviously, the quickest and easiest way to test out some code is to compile and run it on a desktop computer. This is essentially a zero-cost, easy to use debug environment. However, it does not help so much when the code being tested interacts with an RTOS via a number of API calls, as obviously the code will not even link with these calls unresolved.

A trivial early step is to comment out API calls or create dummy stubs that just allow the link to complete. This might enable basic program logic to be checked, but really does not allow too much progress, unless a program is largely comprised of complex algorithms. What is needed are API calls that respond sensibly. Normally, the only way to get “intelligent” responses from API calls is to actually link with an RTOS and run the code on a target system or to use a complex host-based simulation environment. The idea that came out of the discussion was for an RTOS test harness.

The Test Harness Idea
This test harness would simply be a library of functions corresponding to all (or most) of the API calls of the RTOS in use. These functions would accept the correct type and number of parameters and a call would result in a “sensible” response. Some basic parameter checking may result in an error return, otherwise a “success” response is likely. Where full API functionality can easily be simulated (e.g. dynamic memory allocation), the API call can be even more intelligent and appear to respond just like the OS. A separate module, containing some global data structures, would enable the developer to tune the response of API calls so that they can test their code's handling of API call responses, including various failure scenarios.

This approach has clear limitations compared with running the code on a real RTOS, but would give the opportunity to check much more logic before introducing other complexities.

The questioner went away to consider the implementation of this test harness in his team. (And I was left wondering whether such a “product” would be of interest to a wider body of users. I would be very interested to learn whether this approach does have wide appeal.)

Implementing a Test Harness
A test harnessallows the logic around API calls to be thoroughly exercised in waysthat running with a real OS cannot. The implementation is, obviously,totally RTOS specific, but it does not require any knowledge of the RTOSinternal workings. It would be expected that the API documentationwould give enough information about each call to write the necessaryharness function. The approach is best illustrated by example.

If you were using Nucleus RTOS and wanted to allocate a dynamic memory block, you might make an API call like this:

NU_Allocate_Partition(pool, &return_pointer, suspend);

Where the parameters have the following meanings:

pool is a pointer to a memory partition pool (which has previously been created with a call to NU_Create_Memory_Pool ).

return_pointer is a pointer which will point to the allocated partition; a pointer to this variable is passed to the API function.

suspend indicates whether the task should be suspended (and, if so, how) if no partition is currently available.

The function verifies the first two parameters and returns:

NU_INVALID_POOL if the first parameter is not a valid memory pool descriptor; it musthave a value that corresponds to the return from an earlier call to NU_Create_Memory_Pool .

NU_INVALID_POINTER if return_pointer is NULL .

Ifthe test harness is set to allow this call to succeed, the functionallocates some memory and returns a pointer the location indicated by return_pointer and the function returns NU_SUCCESS .

If the test harness is set to cause the call to fail, the return value will be:

NU_NO_PARTITION if the call did not specify a suspend and there are no partitions available.

NU_TIMEOUT if a timeout on suspend was specified and no partition had become available.

NU_POOL_DELETED if the specified pool was deleted during the task suspend.

This enables almost every possible response to be simulated.

So,a test harness is not an alternative to a simulator. It enables, inconjunction with a standard debugger, detailed logic testing andexercising that would not be possible (or, at least, be difficult) with areal RTOS or simulator.

Using the Host OS
Afterusing the test hardness approach, a next step is to have a library offunctions that map onto the RTOS API, which use the underlying host OS(Windows or Linux) to simulate the RTOS services. This can work verywell and many commercial RTOS products have such a tool available. Thecode runs very fast – often faster than on a real target! The real timeprofile is not correct, because the instruction mix on a Pentium isdifferent from, say, an ARM, but that does not stop such a tool beingvery useful for debugging complete (or near complete) multi-threadedapplications without the need for target hardware.

Target Testing
Eventually,it becomes necessary to bite the bullet and test on an actual target.The software is then built for the target device instead of for thehost. But even at this stage, there may be two options: debugging mayproceed on a virtual platform, which enables code execution on adetailed hardware simulation of the target; or, of course, real targethardware may be deployed.

Colin Walls hasover thirty years experience in the electronics industry, largelydedicated to embedded software. A frequent presenter at conferences andseminars and author of numerous technical articles and two books onembedded software, Colin is an embedded software technologist withMentor Embedded [the Mentor Graphics Embedded Software Division], and isbased in the UK. His regular blog is located at: http://blogs.mentor.com/colinwalls and he may be reached by email at colin_walls@mentor.com


Join over 2,000 technical professionals and embedded systems hardware, software, and firmware developers at ESC BostonMay 6-7, 2015, and learn about the latest techniques and tips forreducing time, cost, and complexity in the development process.

Passes for the ESC Boston 2015 Technical Conferenceare available at the conference's official site, with discountedadvance pricing until May 1, 2015. Make sure to follow updates about ESCBoston's other talks, programs, and announcements via the Destination ESC blog on Embedded.com and social media accounts Twitter, Facebook, LinkedIn, and Google+.

The Embedded Systems Conference, EE Times, and Embedded.com are owned by UBM Canon.

6 thoughts on “Testing code that uses an RTOS API

  1. “What you are calling a “test harness” is just a re-invention of the well-known testing construct: mock object.nnThere are many existing libraries for implementing mock objects and functions and giving the test suite control over what the mock returns

    Log in to Reply
  2. “We develop and test FreeRTOS on a PC by simply having a Windows port for the RTOS, much like any other target port, except the context switching is done by suspending and resuming Windows threads rather than pushing and popping registers. It offers a ver

    Log in to Reply
  3. “I've always tried to abstract my code from any particular RTOS or host environment. That way substituting a few functions can get the code ported either to a new target or into a PC for testing.nnThis has allowed me to write large bodies of code that ca

    Log in to Reply
  4. “It's nice to see any discussion of unit testing for embedded software. Yes, mocks! This can be challenging to do in C, but a tool I've found to help ease the pain is Ceedling:nnhttps://github.com/ThrowTheSwitch/Ceedlingn”

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.