Automotive chip companies talk about system-on-chips designed for Advanced Driving Assistance Systems all the time.
But how can the rest of us — reporters, analysts and most important, carmakers — tell one ADAS SoC from another?
Truth is, we can’t. The absence of scientific tools and benchmarks leaves little choice but to take the vendor’s word for it. Or we rely on such imperfect measures as trillion operations per second (TOPS) to compare Intel/Mobileye’s EyeQ5 with Nvidia’s Xavier, which is probably a bum steer.
About a month ago, EEMBC, an industry consortium that develops benchmarks for embedded hardware, rolled out “ADASMark,” an autonomous driving benchmark suite, which is now available for licensing.
The new tool suite, according to EEMBC, is designed to help tier ones and carmakers to optimize their use of compute resources ranging from CPU to GPU and hardware accelerators when they design their own ADAS systems.
Mike Demler, a senior analyst at The Linley Group, welcomed ADASMark, noting, “It’s good to see that this is not just an abstract performance metric, but they used real workloads.” Demler said that participation from AU-Zone Technologies — a Calgary-based engineering design services company — and chip vendors such as NXP Semiconductors and Texas Instruments made EEMBC's test more meaningful than, for example, Baidu’s generic DeepBench.
It’s all about frameworks
EE Times caught up with Peter Torelli, EEMBC president and CTO, to ask about challenges automakers face as they set out to design highly automated vehicles.
There’s no question that more and more automotive embedded systems deploy multiple cores. However, as Torelli pointed out, “there are still very few frameworks that can utilize their asymmetric compute resources.” He added, “Without a framework, every instance of the compiled benchmark would vary dramatically depending on the hardware, and make comparisons across platforms extremely difficult. Frameworks facilitate portability with very little modification.”
Consider the ADASMark Pipeline below, he said.
Torelli said: “The baseline performance of this system might be using the same CPU for all stages in the pipeline. But what if a developer wanted to swap in a custom neural-net chip for the last stage? Or perhaps use a dedicated DSP for the color space conversion?”
This is where a framework comes in.
“Without a framework the developer would need to insert code to interface between the benchmark and the compute device (NN, DSP or GPU). This is time consuming, complicated, and error prone, and can easily disrupt the intent of the benchmark (or corrupt the results).”
A framework makes this retargeting of compute devices much easier, Torelli explained.
EEMBC initially examined options available on the market today. “AMP and OpenAMP attempt to address this, but they are specifications for symmetric multicore, and they don't really help us here,” said Torelli. “We also looked at OpenCV and OpenVX, but support was spotty among the landscape of manufacturers.”
That’s how EEMBC came to develop ADASMark based on a new framework with a more relevant workload.
Focus on imaging pipeline
Key features of the ADASMark Benchmark Suite, according to EEMBC, “include an OpenCL 1.2 Embedded Profile API to ensure consistency between compute implementations; application flows created by a series of micro-benchmarks that measure and report performance for SoCs handling computer vision, autonomous driving, and mobile imaging tasks; and a traffic sign recognition CNN inference engine created by Au-Zone Technologies.”
Because ADAS requires compute-intensive object-detection and visual classification capabilities, ADASMark’s focus is on the imaging pipeline. It looks to use “real-world workloads that represent highly parallel applications, such as surround view stitching, contour detection, and convolutional neural-net (CNN) traffic sign classification,” EEMBC explained.