Why edge AI is a no-brainer - Embedded.com

Why edge AI is a no-brainer

In 2020, Deloitte predicts that more than 750 million edge AI chips — full chips or parts of chips that perform or accelerate machine learning tasks on-device, rather than in a remote data center — will be sold, representing US$2.6 billion in revenue. Furthermore, the edge AI chip market will grow much more quickly than the overall chip market. By 2024, we expect unit sales of edge AI chips to exceed 1.5 billion, possibly by a great deal. That represents compound annual unit sales growth of at least 20%, more than double the longer-term forecast of 9% CAGR for the overall semiconductor industry.

Figure 1: Locations in which intelligence can be embedded (Image: Deloitte Insights)

These edge AI chips will likely find their way into an increasing number of consumer devices, such as high-end smartphones, tablets, smart speakers, and wearables. They will also be used in multiple enterprise markets: robots, cameras, sensors, and other devices for the internet of things. The consumer market for edge AI chips is much larger than the enterprise market, but it is likely to grow more slowly, with a CAGR of 18% expected between 2020 and 2024. The enterprise edge AI chip market is growing much faster, with a predicted CAGR of 50% over the same time frame.

Figure 2: The edge AI chip market (Image: Deloitte Insights)

Nevertheless, this year, the consumer device market will likely represent more than 90% of the edge AI chip market, both in terms of the numbers sold and their dollar value. The vast majority of these edge AI chips will go into high-end smartphones, which account for more than 70% of all consumer edge AI chips currently in use. Indeed, not just in 2020 but for the next few years, AI chip growth will be driven principally by smartphones. We believe that more than a third of the 1.56 billion-unit smartphone market this year may contain edge AI chips.

Because of the extremely processor-intensive requirements, AI computations have almost all been performed remotely in data centers, on enterprise core appliances, or on telecom edge processors — not locally on devices. Edge AI chips are changing all that. They are physically smaller, relatively inexpensive, use much less power, and generate much less heat, making it possible to integrate them into handheld devices as well as non-consumer devices such as robots. By enabling these devices to perform processor-intensive AI computations locally, edge AI chips reduce or eliminate the need to send large amounts of data to a remote location, thereby delivering benefits in usability, speed, and data security and privacy.

Keeping the processing on the device is better in terms of privacy and security; personal information that never leaves a phone cannot be intercepted or misused. And when the edge AI chip is on the phone, it can do all these things even when not connected to a network.

Of course, not all AI computations have to take place locally. For some applications — for instance, when there is simply too much data for a device’s edge AI chip to handle — sending data to be processed by a remote AI array may be adequate or even preferred. In fact, most of the time, AI will be done in a hybrid fashion: some portion on the device and some in the cloud. The preferred mix in any given situation will vary depending on exactly what kind of AI processing needs to be done.

The economics of edge AI in smartphones

Smartphones aren’t the only devices that use edge AI chips; other device categories — tablets, wearables, smart speakers — contain them as well. In the short term, these non-smartphone devices will likely have much less of an impact on edge AI chip sales than smartphones, either because the market is not growing (as for tablets) or because it is too small to make a material difference (for instance, smart speakers and wearables combined are expected to sell a mere 125 million units in 2020). Many wearables and smart speakers depend on edge AI chips, however, so penetration is already high.

Currently, only the most expensive smartphones — those in the top third of the price distribution — are likely to use edge AI chips. But putting an AI chip in a smartphone doesn’t have to be price-prohibitive for the consumer.

It’s possible to arrive at a fairly sound estimate of a smartphone’s edge AI chip content. To date, images of phone processors in Samsung, Apple, and Huawei show the naked silicon die with all its features visible, allowing identification of which portions of the chips are used for which functions. A die shot of the chip for Samsung’s Exynos 9820 shows that about 5% of the total chip area is dedicated to AI processors. Samsung’s cost for the entire SoC application processor is estimated at US$70.50, which is the phone’s second-most expensive component (after the display), representing about 17% of the device’s total bill of materials. Assuming that the AI portion costs the same as the rest of the components on a die-area basis, the Exynos’s edge AI neural
processing unit (NPU) represents roughly 5% of the chip’s total cost. That translates to about US$3.50 each.

Figure 3: A die shot of the chip for Samsung’s Exynos 9820 shows that about 5% of the total chip area is dedicated to AI processors. (Image: ChipRebel; Annotation: AnandTech)

Similarly, Apple’s A12 Bionic chip dedicates about 7% of the die area to machine learning. At an estimated US$72 for the whole processor, that percentage suggests a cost of US$5.10 for the edge AI portion. The Huawei Kirin 970 chip, estimated to cost the manufacturer US$52.50, dedicates 2.1% of the die to the NPU, suggesting a cost of US$1.10. (Die area is not the only way to measure what percentage of a chip’s total cost goes toward AI, however. According to Huawei, the Kirin 970’s NPU has 150 million transistors, representing 2.7% of the chip’s total of 5.5 billion transistors. That would suggest a slightly higher NPU cost of US$1.42).

Figure 4: Apple’s A12 Bionic chip dedicates about 7% of the die area to machine learning. (Image: TechInsights/AnandTech)

Although the cited cost range is wide, it’s reasonable to assume that NPUs cost an average of US$3.50 per chip. Multiplied by half a billion smartphones (not to mention tablets, speakers, and wearables), that makes for a large market, despite the low price per chip. At an average cost of US$3.50 to the manufacturer, and a probable minimum of US$1, adding a dedicated edge AI NPU to smartphone processing chips starts looking like a no-brainer. Assuming normal markup, adding US$1 to the manufacturing cost translates into only US$2 more for the end customer. That means that NPUs and their attendant benefits — a better camera, offline voice assistance, and so on — can be put into even a US$250 smartphone for less than a 1% price increase.

Sourcing AI chips: In-house or third party?

Companies that manufacture smartphones and other devices vary in their approaches to obtaining edge AI chips, with the decision driven by factors such as phone model and, in some cases, geography. Some buy application processor/modem chips from third-party providers, such as Qualcomm and MediaTek, which together captured roughly 60% of the smartphone SoC market in 2018.

Both Qualcomm and MediaTek offer a range of SoCs at various prices; while not all of them include an edge AI chip, the higher-end offerings (including Qualcomm’s Snapdragon 845 and 855 and MediaTek’s Helio P60) usually do. At the other end of the scale, Apple does not use external AP chips at all: It designs and uses its own SoC processors, such as the A11, A12, and A13 Bionic chips, all of which have edge AI.

Other device makers, such as Samsung and Huawei, use a hybrid strategy, buying some SoCs from merchant market silicon suppliers and using their own chips (such as Samsung’s Exynos 9820 and Huawei’s Kirin 970/980) for the rest.

>> Continue reading page two of this article originally published on our sister site, EE Times Europe.

Duncan Stewart is with Deloitte’s Center for Technology, Media and Telecommunications.
Jeff Loucks is with Deloitte’s Center for Technology, Media and Telecommunications.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.