AI-capable cameras drive smart city trial - Embedded.com

AI-capable cameras drive smart city trial

An AI-powered smart city trial in Rome, using Sony image sensors with AI capability as part of a “Smart Tip” system, is intended to help reduce gridlock and pollution from traffic, cut pedestrian road accidents and minimize crowding on the city’s buses.

An AI-powered smart city trial in Rome, using Sony image sensors with AI capability as part of a “Smart Tip” system, is intended to help reduce gridlock and pollution from traffic, cut pedestrian road accidents and minimize crowding on the city’s buses.

The trial is intended to tackle three “pain points” common to most cities: traffic volume and congestion, particularly that related to looking for a parking space; safety of pedestrians when crossing the road; and optimising public transport availability for the pandemic era and beyond.

Three proof-of-concept Smart Tip systems have been deployed in the Eternal City so far, atop traffic lights.


The Smart Tip system, featuring two Sony IMX500 AI-capable camera modules, sits atop a traffic light (Source: Sony Semiconductor Europe)

The Smart Tips each feature two Sony IMX500 image sensors. These sensors, launched a year ago, notably feature AI processing capability that allows images to be processed at the edge of the network, without the images leaving the Smart Tip.

The IMX500 module has a 12.3 megapixel sensor alongside a logic chip featuring a Sony DSP design dedicated to AI processing, plus memory to store the AI model. The ability to process images within the device (without sending data to the cloud) is critical to the operation of smart city AI systems since it allows extremely fast latency. It also ensures consumer privacy; images do not leave the device, instead, information is extracted via AI and converted to metadata. This metadata might include the amount of people boarding or alighting a bus at a bus stop, or the presence of pedestrians crossing the road. (The IMX500 has the capability to output whole images or regions of interest/partial images if required, but communicating metadata only was selected for this trial).

“There is no image coming out of this sensor, it’s not a camera, it’s a camera used as a sensor – no image goes out,” said Antonio Avitabile, managing director of corporate alliance and investment at Sony. “What goes out is just a “yes, vehicle” or “no vehicle” or a number of people. The idea here is to serve the purpose, and do the job, but without watching people.”

Processing the images at the edge also avoids having to upload scores of 12-megapixel images to the cloud; the metadata alone is much better suited to transmission by the Smart Tip’s cellular modem.

click for full size image

The three applications being trialed by this project (Source: Sony Semiconductor Europe)

Three applications

“Traffic congestion is a very big issue in large cities,” Avitabile said. “A significant portion is pretty much correlated to drivers seeking an available parking space.”

The Rome smart city AI system aims to keep track of where there are available parking spaces in real-time. GPS locations of available spaces are sent to the cloud via the system’s cellular modem. Locations can then be communicated to drivers through a mobile phone app. The idea is to avoid drivers having to drive around for extended periods of time looking for a space, reducing congestion and pollution at the same time.

Pedestrian safety is another of the key pain points for any city; thousands of pedestrians are killed on European roads every year. The average distance between pedestrians and vehicles is a key metric used to measure pedestrian safety. The trial is aiming to deliver a quantitative analysis of this. The Smart Tips will detect pedestrians crossing the road, then activate visual or audible signals for vehicles when pedestrians are crossing.


Traffic outside Rome’s Colosseum (Credit: Boris Stroujko)

“Any time a pedestrian is identified crossing the road, we have some kind of lighting signal on the crossing that will alert the driver that someone is actually crossing,” Avitabile said. “We believe that will be a deterrent to generate fewer injuries and deaths at pedestrian crossings.”

Public transport is the third application the trial will tackle. Avitabile’s example, smart bus stops, would be able to check how many people were waiting at each bus stop and use this information to control the amount of people on each bus (particularly important during the ongoing pandemic). Buses could also avoid stopping at stops where there are no passengers waiting. The system can also monitor how many people get on and off the bus at each stop. This information can be used to better optimise bus routes and timing.

Backend system

Sony’s Smart Tip system includes a service platform running in the cloud. This platform will handle provisioning and enrolment of the Smart Tip devices. This platform also handles training of the AI models that will be used to detect cars and people, and production of models that can fit on the IMX500 device.

Scalability of the systems is ensured by minimising their bandwidth requirements (by sending only metadata as described above). The Smart Tips themselves are identical across the three use cases; only the AI model software on each is different. The AI model can be uploaded or updated over the air to improve efficacy or accommodate different applications going forward.

>> This article was originally published on our sister site, EE Times Europe.


Related Contents:

For more Embedded, subscribe to Embedded’s weekly email newsletter.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.