Evaluating the performance of multi-core processors- Part 1
Multi-core Performance Benchmarks
Good benchmarks for evaluating embedded single-core processor performance have been available for years; however, benchmarks for embedded multi-core processor performance are scarcer. Some popular benchmark programs commonly used to evaluate multi-core embedded processor performance are:
SPEC CPU2006 (and CPU2000) rate
EEMBC MultiBench software
BDTI Benchmark Suites
SPEC OMP
SPEC CPU2006 evaluates multi-core processor performance when executed under what is termed " rate " mode. SPEC CPU2006 rate tests multi-core performance by executing multiple copies of the same benchmark simultaneously and determining the throughput.
The number of copies executed is determined by the tester, but is typically set equal to the number of processor cores in the system. SPEC CPU2006 rate provides a relatively straightforward method of evaluating multi-core processor performance, however, can be criticized for not representing typical workloads " how many users would execute multiple copies of the same application with the same data set simultaneously?
The EEMBC MultiBench benchmark software is a set of multi-context benchmarks based upon embedded market segment applications. Much of the work is in the creation of a platform independent harness, called the Multi-Instance-Test Harness (MITH), which can be ported to different embedded systems, some of which may not support a standard thread model such as POSIX threads or execution on a COTS OS. The group has targeted applications from several sources including creating parallel versions of the single-core EEMBC benchmarks.
 |
| Figure 3.6 : MITH contexts and work items |
The key benefit of MITH is that it allows the easy creation of a wide variety of workloads from predefi ned work items. As depicted in Figure 3.6 above a workload can be composed of the following testing strategies:
1. A single instance of a work item " similar to the SPEC CPU2006 rate benchmark.
2. Multiple instances of the same work item, where each instance executes a different data set " provides an environment similar to embedded applications such as a multiple channel VoIP application.
3. Multiple work items can be executed in parallel " emulates a complex system supporting a practically unlimited number of work items. An example of such a system would be a video conferencing application that is simultaneously running MPEG encode and decode algorithms, networking (TCP), screen updates ( jpeg), and even background music (MP3).
4. Instances of multi-threaded work items " some of the algorithms are built to take advantage of concurrency to speed up processing of a single data set. For example, H.264 encoding could employ each core to process separate frames.
As mentioned earlier, the BDTI Benchmark Suites are designed to be applicable to a very wide range of processors, including single-core and multi-core processors, as well as many-core processors [7] and even FPGAs. Taking advantage of this fl exibility, BDTI 's benchmarks have been implemented on a number of multi-core processors beginning in 2003.
In particular, the BDTI Communications Benchmark (OFDM) - and the BDTI Video Encoder and Decoder Benchmarks - are amenable to multi-core processor implementations. Consistent with its philosophy of broad benchmark applicability, BDTI does not require the use of threads or any other particular implementation approach. It is up to the benchmark implementer to decide the best approach for implementing BDTI 's benchmark suites on a given target processor.
SPEC OMP [8] was released in June 2001 and is based upon applications in SPEC CPU2000 that were parallelized using OpenMP directives. The benchmark assesses the performance of SMP systems and contains two data sets, SPEC OMPM2001 and SPEC OMPL2001.
SPEC OMPM2001 uses data sets that are smaller than those in SPEC OMPL2001 and is appropriate for evaluating the performance of small-scale multicore processors and multiprocessors. SPEC OMP is not recommended for evaluating embedded application performance; the application focus of SPEC OMP is the high performance computing (HPC) market segment and floating point performance.
Only 2 out of the 11 applications in OMPM2001 are written in C; the rest are written in Fortran. OpenMP fi ts a number of embedded application areas; however, I cannot recommend SPEC OMP as a method of benchmarking multi-core processors that will be employed in an embedded project.
Loading comments... Write a comment