Performance Testing
Because embedded systems often have real-time deadlines, we must concern ourselves with testing for performance, not just functionality. Performance testing determines whether the required result was generated within a certain amount of time. In many cases, we are interested in the worst-case execution time, although in some cases we may want to verify the best-case or averagecase execution time.

Performance analysis is very important here. Performance analysis can tell us what path causes the worst-case (or other case) execution time. From there we can determine the inputs required and the expected outputs. Real-time code with deadlines must always terminate—we know that the code must finish for any possible set of inputs. Functional testing can help us determine whether the program actually terminates.

How do we measure program performance? Functional testing can be performed on any computer in most cases so long as we can compile the code on that platform. Performance testing is not easy. We may be able to use a simulator to measure performance. However, CPU simulators vary widely in the amount of timing information they provide.

Performance can also be measured directly on the hardware. We can use a timer in the system to count execution time. The program resets the timer at the start and checks the timer's value at the end of execution. We can also use logic analyzers as long as the start and end points of the program are visible on the bus.

To obtain truly accurate performance measurements, we must run the program in the environment in which it will operate. For example, the CPU used must have the same size cache as the target platform will provide. The program should also be run along with other programs that operate concurrently. When several programs (or programs and interrupt drivers) operate concurrently, they can interfere with each other in the cache causing severe performance problems.

Conclusion
The program is a very fundamental unit of embedded systems design and it usually contains tightly interacting code. Because we care about more than just functionality, we need to understand how programs are created. Because today's compilers do not take directions such as "compile this to run in less than 1 microsecon," we have to be able to optimize programs ourselves for speed, power and space.

Our earlier understanding of computer architecture is critical to our ability to perform these optimizations. We also need to test programs to make sure they do what we want. Some of our testing technicques can also be useful in exercising the programs for performance optimization.

To read Part 1, go to  "Program design and analysis."
To read Part 2 , go to "Models of program, assemblers and linkers."
To read Part 3, go to  "Basic Compilation Techniques"
To read Part 4, go to  "The creation of procedures"
To read Part 5, go to  "Register allocation and scheduling"
To read Part 6, go to  "Analysis and optimization of execution time"
To read Part 7, go to  "Trace-Driven Performance Analysis"
To read Part 8, go to  "Analysis and optimization of energy and power."

Used with the permission of the publisher, Newnes/Elsevier, this series of nine articles is based on copyrighted material from "Computers as Components: Principles of Embedded Computer System Design" by Wayne Wolf. The book can be purchased on line.

Wayne Wolf  is currently the Georgia Research Alliance Eminent Scholar holding the Rhesa "Ray" S. Farmer, Jr., Distinguished Chair in Embedded Computer Systems at Georgia Tech's School of Electrical and Computer Engineering (ECE). Previously a professor of electrical engineering at Princeton University, he worked at AT&T Bell Laboratories. He has served as editor in chief of the ACM Transactions on Embedded Computing and of Design Automation for Embedded Systems.

References:
[Aho86] Alfred VAho, et. al. "Compilers: Principles, Techniques and Tools. Addison Wesley,1986.
[Hor96]  Joseph  R. Horgan  and Aditya Mathur, "Software testing and reliability" Chapter 13 in Hand book of Software Reliability Engineering," IEEE Computer Society Press/ McGraw Hill 1996.
[How82] W.E. Howden, "Weak mutation testing and the completeness of test cases," IEEE Transactions on Software Engineering.
[McC76] T.J. McCabe, "A complexity measure," IEEE Transactions on Software Engineering.