SAN FRANCISCO — As artificial intelligence (AI) capability moves from the cloud to edge, it is inevitable that chipmakers will find ways to implement AI functions like neural-network processing and voice recognition in smaller, more efficient, and cost-effective devices.
The big, expensive AI accelerators that perform tasks back in the data center aren’t going to cut it for edge node devices. Battle lines are being drawn among various devices — including CPUs, GPUs, FPGAs, DSPs, and even microcontrollers — to implement AI at the edge with the required footprint, price point, and power efficiency for given applications.
To that end, a pair of intriguing architectures created specifically for implementing AI at the edge are being introduced at the Linley Processor Conference on Tuesday by Cadence Design Systems and Flex Logix Technologies. Both focus on bringing AI functionality into edge node devices with an emphasis on reducing the memory footprint.
“Not everything is going to be in the cloud,” said Rich Wawrzyniak, a senior principal analyst at Semico Research. “Endpoint devices that have AI are going to be the norm.”
According to Jim McGregor, principal analyst at Tirias Research, most of the solutions that will hit the market in the immediate future are still likely to be “hybrid in some fashion, doing most of the processing at the edge but leveraging the cloud as necessary unless they are only listening for select words or sounds.”
“We do see a push for more voice processing at the edge,” said Kevin Krewell, also a primary analyst at Tirias. “There’s privacy issues sending all voice data to the cloud. Processing at the edge can reduce latency responses. In addition, there’s more processing capability at the edge.”
McGregor said that using a DSP is the most efficient way of doing processing at the edge. “However, with that said, I have seen Alexa running on MCUs like the STM32 from STMicroelectronics.”
While the Cadence Tensilica HiFi 5 DSP is focused specifically on implementing voice recognition and neural-network–based processing, Flex Logix’s NMAX architecture is designed for more complex neural inferencing. Both claim significant advantages in cost, performance, and power consumption versus competitors.
According to Wawrzyniak, the new architectures represent an escalation of IP suppliers introducing IP specifically for AI. He added that most of the IP that is currently being used in AI is the same type of IP that is used for other types of general-purpose SoCs. “As things get more refined, people are going to come out with products that are specifically optimized for AI.”