The Embedded Microprocessor Benchmark Consortium (EEMBC) has developed a suite of benchmarks tailored for 8 and 16bit embedded microcontrollers.
Until now the EEMBC benchmarking standards were best suited for DSPs and 32 or 64bit devices. The consortium's latest microcontroller benchmarks were created by adapting a number of existing EEMBC benchmark kernels aimed at automotive and industrial applications. This suite also includes a new task-based benchmark suite that performs a variety of operations including memory transfers, multiplication, data transfer with a UART, and shift operations typical of encryption algorithms. The latest microcontroller suite was developed by the EEMBC Certification Labs (ECL) and a committee composed of representatives from EEMBC members including Hitachi, IBM, Microchip, Mitsubishi Electric, NEC Electronics, and Toshiba.
“As the Project Manager for EEMBC's Technical Advisory Group (TechTAG), and on behalf of EEMBC, I can tell you that we had the active participation of the leading 8 and 16bit vendors in creating this suite,” said ECL chairman and CTO Alan R. Weiss. “The process was fair and open, and has resulted in a very useful tool to help customers select the right part for their applications.”
The industrial/automotive algorithms adapted for this suite include bit manipulation, memory access, response to remote request, pointer chasing, pulse-width modulation, road speed calculation, and tooth-to-spark. EEMBC allows two scoring methods. For 'out-of-the-box' evaluations, a compiler is specified, but the original EEMBC source code remains unmodified. For 'optimized' evaluations, the EEMBC source code is hand-tuned to take advantage of certain architectural features, peripherals or hardware accelerators, special libraries, or assembly language. The automotive/industrial algorithms fit this model; however the task-based benchmark takes a different tack. The task-based benchmark is provided to EEMBC members as a detailed specification that must be implemented by each member.
The task-based algorithm is specified as a flow chart. Vendors are free to implement the task-based routines however they wish, in C or assembly, as long as the intermediate and final results are met during execution. These routines show the effectiveness and the efficiency in handling interrupts as well as basic data manipulation and processing. Overall, the 8 and 16bit benchmarks help engineers determine the efficiency of a microcontroller architecture.
A disclosure document is available to detail the specifics of the benchmarked platform and provides the engineer with more complete data such as the processor bus speed or compiler switch settings.
As with all devices tested under the EEMBC benchmark process, the Consortium requires that benchmark test results derived from the new microcontroller suite be repeated and certified by the EEMBC Certification Laboratories (ECL) before being made public. The benchmarking process begins when an EEMBC member ports the benchmarks and Test Harness (or contracts with the EEMBC Certification Labs (ECL) to do the porting). The member then runs the benchmarks according to the rules of EEMBC. Next, the member fills out a Disclosure Workbook and sends it, along with other important data, to ECL for certification. ECL certifies the benchmark scores and the Disclosure Workbook and then must get signed permission to release the benchmark scores.
Version 1.0 EEMBC benchmarks targetted five application areas: telecommunications, consumer, networking, office automation, and automotive/industrial. The consortium is continuing to develop the benchmarks for additional applications that are relevant to market demands. It is considering a variety of application areas for its Version 2.0 benchmarks.
EEMBC, the Embedded Microprocessor Benchmark Consortium, was formed in 1997 to develop meaningful performance benchmarks for processors and compilers in embedded applications.
Through the combined efforts of its members – more than 45 of the world's leading semiconductor, intellectual property, compiler, and RTOS companies – EEMBC benchmarks have become an industry standard for evaluating the capabilities of embedded processors and compilers according to objective, clearly-defined, application-based criteria. Since releasing its first certified benchmark scores in April 2000, EEMBC scores are said to have effectively replaced the obsolete Dhrystone mips, especially in situations where real engineering value is important. In a departure from the one-size-fits-all Dhrystone ratings, EEMBC benchmarks reflect real-world applications and the demands that processors encounter in these environments. The result is a collection of 'algorithms' organized into benchmark suites targeting telecommunications, networking, automotive/industrial, consumer, and office equipment products.
EEMBC's certification rules represent another break with the past. For a processor's scores to be published, the EEMBC Certification Laboratories (ECL) must execute benchmarks run by the manufacturer. ECL certification ensures that scores are repeatable, obtained fairly, and according to EEMBC's rules.
Latest scores from EEMBC
The 170MHz Carmel DSP, tested against EEMBC's Telecomm benchmark suite, scored 4.8 Telemarks. Infineon used its Infineon IC3 compiler to produce EEMBC out-of-the-box benchmark scores. It used its IC3 compiler to produce EEMBC out-of-the-box benchmark scores.A 96MHz Infineon Tricore TC11IB, tested against EEMBC's automotive/industrial benchmark suite, scored 33 Automarks. In this case it used Tasking's Tricore V1.4r1 compiler.
Tested against EEMBC's consumer benchmark suite, the ARCtangent-A4 software simulator received a consolidated Consumermark out-of-the-box score of 0.0558/MHz and a 'full-fury' optimized score of 1.0621/MHz. The consumer benchmark suite evaluates microprocessor performance in JPEG compression and decompression, gray space filtering, RGB-to -CMYK conversion, and RGB-to-YIQ conversion.
Tested against EEMBC's telecomm benchmarks, the LSI402ZX achieved an out-of-the-box Telemark score of 3.3 using the LSI Logic ZSP SDK 3.2 compiler. Under optimized conditions, the device received an aggregate Telemark score of 40.3.
Tested against 46 EEMBC benchmark 'kernels', each representing a different workload and stressing a different processor capability, the MPC7455 was tested and certified across all five application areas targeted by the EEMBC benchmarks.