What do you get when you cross AI with the IoT? The artificial intelligence of things (AIoT) is the simple answer, but you also get a huge new application area for microcontrollers, enabled by advances in neural network techniques that mean machine learning is no longer limited to the world of supercomputers. These days, smartphone application processors can (and do) perform AI inference for image processing, recommendation engines, and other complex features.
Bringing this kind of capability to the humble microcontroller represents a huge opportunity. Imagine a hearing aid that can use AI to filter background noise from conversations, smart-home appliances that can recognize the user’s face and switch to their personalized settings, and AI-enabled sensor nodes that can run for years on the tiniest of batteries. Processing the data at the endpoint offers latency, security, and privacy advantages that can’t be ignored.
Used in tandem, Arm’s Cortex-M55 and Ethos-U55 have enough processing power for applications such as gesture recognition, biometrics and speech recognition (Image: Arm)
However, achieving meaningful machine learning with microcontroller-level devices is not an easy task. Memory, a key criterion for AI calculations, is often severely limited, for example. But data science is advancing quickly to reduce model size, and device and IP vendors are responding by developing tools and incorporating features tailored for the demands of modern machine learning.
TinyML takes off
As a sign of this sector’s rapid growth, the TinyML Summit, a new industry event held in February in Silicon Valley, is going from strength to strength. The first summit, held last year, had 11 sponsoring companies; this year’s event had 27, and slots sold out much earlier, according to the organizers. Attendance at TinyML’s global monthly meet-ups for designers has grown dramatically, organizers said.
“We see a new world with trillions of intelligent devices enabled by TinyML technologies that sense, analyze, and autonomously act together to create a healthier and more sustainable environment for all,” said Qualcomm Senior Director Evgeni Gousev, co-chair of the TinyML Committee, in his opening remarks at a recent conference.
Gousev attributed this growth to the development of more energy-efficient hardware and algorithms, combined with more mature software tools. Corporate and venture-capital investment is increasing, as are startup and M&A activity, he noted.
Eta Compute’s ECM3532 uses an Arm Cortex-M3 core plus an NXP CoolFlux DSP core. The machine learning workload can be handled by either, or both (Image: Eta Compute)
Today, the TinyML Committee believes that the tech has been validated and that initial products using machine learning in microcontrollers should hit the market in two to three years. “Killer apps” are thought to be three to five years away.
A big part of the tech validation came last spring when Google demonstrated a version of its TensorFlow framework for microcontrollers for the first time. TensorFlow Lite for Microcontrollers is designed to run on devices with only kilobytes of memory (the core runtime fits in 16 KB on an Arm Cortex-M3; with enough operators to run a speech keyword-detection model, it takes up a total of 22 KB). It supports inference but not training.
The big microcontroller makers, of course, are watching developments in the TinyML community with interest. As research enables neural network models to get smaller, the opportunities get bigger. Most have some kind of support for machine-learning applications. For example, STMicroelectronics has an extension pack, STM32Cube.AI, that enables mapping and running neural networks on its STM32 family of Arm Cortex-M–based microcontrollers.
Renesas Electronics’ e-AI development environment allows AI inference to be implemented on microcontrollers. It effectively translates the model into a form that is usable in the company’s e2 studio, compatible with C/C++ projects.
NXP Semiconductors said it has customers using its lower-end Kinetis and LPC MCUs for machine-learning applications. The company is embracing AI with hardware and software solutions, albeit primarily oriented around its bigger application processors and crossover processors (between application processors and microcontrollers).
Most of the established companies in the microcontroller space have one thing in common: Arm. The embedded-processor–core giant dominates the microcontroller market with its Cortex-M series. The company recently announced the brand new Cortex-M55 core, which is designed specifically for machine-learning applications, especially when used in combination with Arm’s Ethos-U55 AI accelerator. Both are designed for resource-constrained environments. But how can startups and smaller companies seek to compete with the big players in this market?
“Not by building Arm-based SoCs, because [the dominant players] do that really well,” laughed XMOS CEO Mark Lippett. “The only way to compete against those guys is by having an architectural edge … [that means] the intrinsic capabilities of the Xcore in terms of performance, but also the flexibility.”
XMOS’s Xcore.ai, its newly released crossover processor for voice interfaces, will not compete directly with microcontrollers, but the sentiment still holds true. Any company making an Arm-based SoC to compete with the big guys better have something pretty special in its secret sauce.