Software performance engineering for embedded systems: Part 2 – The importance of performance measurements

Robert Oshana, Freescale Semiconductor

September 30, 2012

Robert Oshana, Freescale Semiconductor

Step 5: Conduct an economic analysis of the project based on this plan

Lloyd Williams has mapped SPE into the RUP process as follows (4):

Inception Phase The primary objective of the inception phase is to scope the system adequately as a basis for validating initial costing and budgets. From a SPE perspective, high level risks that may impact system performance are identified and described in this phase.

Elaboration Phase In this phase, the main objective is to mitigate the key risk items identified by analysis up to the end of this phase. In this phase the problem domain analysis is made and the architecture of the project gets its basic form. This is where the critical business processes are decomposed to critical use cases. The type of requirements that relate to SPE are the non-functional requirements (NFR) that are not limited to use cases.

The primary difference between functional and non-functional requirements is that functional requirements define what the system should do, and can be expressed as "The embedded software shall (monitor, control, etc.), while non-functional requirements define when and/or how well the system should perform the functions, and can be expressed as "The embedded software shall be (fast, reliable, scalable, etc.).

One way to formulate a set of NFR’s for an embedded system is to use the acronym “SCRUPLED”:

• Security, Licensing, Installation– access privileges, security requirements, installation and licensing requirements
• Copyright, legal notices and other items – required corporate representations and legal protections
• Reliability – defects, Mean Time Between Failures, availability
• Usability – ease of use requirements, presentation design guidelines, UE standards, accessibility standards, training standards, sheets, help systems
• Performance – quantitative performance requirements
• Localization and Internationalization – foreign language operating systems, localization enablement, specific localizations
• Essential standards – industry, regulatory, and other externally imposed standards
• Design Constraints – other constraints on the system or development technologies.

Mandated programming languages and standards, platforms, common components - Initial models are created that describe the overall system load over a specified time period, defining how many of each type of key transaction (networking packets, video frames, etc.) will be executed per unit of time.

Construction phase The primary objective of the construction phase is to build the software system. The main focus is on the development of components and other features of the system. This is where the majority of the coding takes place. Several construction iterations may be developed in order to divide the use cases into manageable segments that produce demonstrable prototypes. SPE adds some activities to this phase. Performance tool-related activities are completed. For example, specifying a profiling tool for component development and unit testing is necessary. Automated frameworks are needed to drive the components under development and measure performance.

Transition phase This is where we transit the system from development into production. Activities include training the end users and maintainers, and beta testing the system to validate it against the end users' expectations. The product is also checked against the quality level set in the Inception phase. From an SPE perspective, this is when we configure operating systems, the network, and any message queuing software and other optimizations identified in the performance test plan. It's important to ensure that all necessary performance monitoring software is developed, deployed, and configured.


Embedded computer performance is characterized by the amount of useful work accomplished by a computer system compared to the time and resources used. Depending on the context, good computer performance may involve one or more of the following:

• Short response time for a given piece of work
• High throughput (rate of processing work)
• Low utilization of computing resources
• High availability of the computing system or application

It can be difficult to design a system that provides both low latency and high performance. However, real-world systems (such as Media, eNodeB, etc.) need both. For example, see Figure 14. This eNodeB system must be able to handle two basic NFR’s: low latency, 1 ms periodic interrupt used for scheduling important calls through the system; and maximum data throughout of 100 Mbps downlink and 50 Mbps uplink for supporting key customer use cases such as data transfer for web surfing and texting.



Click on image to enlarge.

This is a case where the designer needs to tune the system for the right balance of latency and performance. This would include the following basic decisions:
Partitioning the application between hardware cores and hardware acceleration. Embedded systems usually consist of processors that execute domain-specific programs. Much of their functionality is implemented in software, which is running on one or multiple generic processors. High performance functions may be implemented in hardware. Typical examples include TV sets, cellular phones, eNodeB basestations and printers. Most of these systems are running multimedia and/or telecom applications, like video and audio decoders.

Figure 15 shows a table summarizing the key performance intensive functions required for an eNodeB application, which of those functions are utilizing hardware acceleration, the allocated cycle budget, and the core loading percentage.



Click on image to enlarge.

Figure 15: An Example of using processor cores and accelerators to partition an application onto a SoC processor

Partitioning the software application across the programmable cores in order to achieve the NFRs required for the application. Figure 14 above shows a diagram of how the real-time tasks are allocated to one of the two available cores and the non real-time functions are allocated to the other processing core.

Designing the proper software architecture to support the NFRs. Figure 16 shows additional software support for performing zero copy transfer of required packets for eNodeB processing around the Linux software stack, avoiding the extra overhead required for unnecessarily going up and down the Linux stack.

Click on image to enlarge.

Figure 16: Bypassing the Linux software stack using a “fastpath” software bypass technology

If SPE has been properly applied at each iteration and phase of the project, this should be sufficient to enable the system to achieve the required performance goals. If there are use cases that cannot be tuned into compliance, then it will be necessary to consider portions of the system to be re-factored or, in the worse case, re-partitioned between hardware and software. In some cases the problem can be resolved with additional hardware, but adding more hardware leads quickly to diminishing returns, as Amdahl’s Law demonstrates (Figure 17).



Click on image to enlarge.
Figure 17: Amdahl’s Law dictates that more hardware may not necessarily improve performance linearly

SPE must be managed throughout the lifecycle. Crawl charts showing actual and target performance targets, like that shown in Figure 18, can be used to manage and report on performance status as the project executes. This can be a transparent way of communicating performance to key stakeholders and also a way to measure how well the SPE process is working. Software release iterations are possible with SPE and these iterations can also be tracked using crawl charts showing performance goals for each of the software iterations.

Click on image to enlarge.

Figure 18: A Performance “crawl chart” showing performance improvements over time

Click on image to enlarge.

Figure 19: A Performance crawl chart showing increasing performance improvements supporting an incremental release process

***

Next in Part 3: Collecting data and using it effectively
Read Part 1: Software performance engineering for embedded systems: Part 1 - What is SPE?

Rob Oshana, author of the soon to be published “Software engineering for embedded systems,” by Elsevier, is director of software R&D, Networking Systems Group, Freescale Semiconductor.

References

"A Maturity Model for Application Performance Management Process Evolution, A model for evolving organization’s application performance management process", By Shyam Kumar Doddavula, Nidhi Timari, and Amit Gawande

"Five Steps to Solving Software Performance Problems", Lloyd G. Williams, Ph.D.Connie U. Smith, Ph.D. June, 2002

"Software Performance Engineering", in UML for Real: Design of Embedded Real-Time Systems, Luciano Lavagno, Grant Martin, Bran Selic ed., Kluwer, 2003.

"Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software", Lloyd G. Williams, Ph.D., Connie U. Smith, Ph.D.

Used with permission from Morgan Kaufmann, a division of Elsevier, Copyright 2012. For more information about Software engineering for embedded systems and other similar books, visit www.elsevierdirect.com