The implementation of gesture control has been growing especially in high-end vehicles and consumer electronics products. However, gesture-based interaction can enhance numerous other types of equipment, including kitchen appliances, thermostats, smart-home and smart-lighting controls, laptops, AR/VR headsets, tablets, and smartphones. There is also the added dimension of hygiene, where the ability to interpret touchless gestures in user interfaces can help prevent infections from spreading through equipment like vending and ticketing machines, elevator controls, and interactive signage.
To enable this more widespread adoption of gesture control, STMicroelectronics has launched a solution comprising free engineering software to be used with ST’s VL53L5CX FlightSense time-of-flight (ToF) multizone ranging sensor. Gesture recognition with ToF sensors can enable sophisticated interactions with a wide variety of devices.
Available in the form of companion software for its VL53L5CX multi-zone ToF ranging sensor, ST said it is now making gesture detection simpler and more affordable for mass-market applications. Together, the sensor and software calculate in real-time the X/Y/Z coordinates of the hand, enabling hand tracking and thus recognition of gestures like tapping, swiping, level control, and more.
Conventional gesture-recognition systems typically use more expensive and intrusive camera-based machine vision. ST said its solution lets designers build systems that benefit from greater user privacy and lower power consumption, and work in the dark without needing external illumination, unlike vision-based solutions. The lightweight gesture algorithm can run on a low-power microcontroller and demands minimal system resources, so is easily integrated in an existing application.
The software includes a complete development ecosystem with ready-to-use example code and intuitive GUI for reduced design times. The STSW-IMG035 software package is specially designed for the VL53L5CX multi-zone direct time-of-flight (dToF) ranging sensor and can be used with all STM32 microcontrollers based on Cortex M0+ and M4-based microcontrollers. This is ST’s latest-generation ToF sensor, providing 64 zones with high-accuracy ranging up to 400cm with a wide, square-edged 63°-diagonal field of view.
The VL53L5CX gesture recognition algorithms can detect multiple motions: tap, double tap, left and right swipe, as well as level control. The package includes: the GUI executable, example code for the NUCLEO-F401RE board, libraries for supported Cortex cores, and an easy-to-read user manual. To run the GUI, a VL53L5CX Nucleo expansion board (X-NUCLEO-53L5A1) is required with a NUCLEO-F401RE Nucleo board. These two boards are also available as a kit under P-NUCLEO-53L5A1. Breakout boards (VL53L5CX-SATEL) connected to the STM32 Nucleo board can also be used.
“Current trends are driving so-called edge devices to become more affordable and energy efficient, as well as more advanced and powerful,” said Eric Aussedat, ST’s executive VP, imaging sub-group general manager. “ST is setting the pace with technologies such as our FlightSense imaging product portfolio, which now adds free and ready-to-use software that helps integrate gesture sensing for touchless control in all kinds of consumer and industrial equipment.”
A key advantage of ST’s gesture solution is that it provides full privacy thanks to the VL53L5CX computing distance information. The performances are independent of target reflectance. The solution works even with gloves or in low light conditions, unlike camera-based solutions.
The gesture recognition mode can be combined with the distance measurement mode of the VL53L5CX sensor, allowing user detection and system activation use-cases. The small size of the sensor makes it easy to integrate, and it can be hidden behind a dark cover window.
Gesture recognition can be used to enhance numerous applications and end-products. It can be incorporated into home appliance devices like coffee machines, cooking plates or ovens. In public places, it could control vending machines, and elevator call buttons. Personal electronics can also benefit from gesture recognition, especially in smart speakers, laptops, tablets, or AR/VR headsets.
The VL53L5CX sensor is in production now, in a 6.4mm x 3.0mm x 1.5mm 16-pin optical LGA package, priced from $3.90 for orders of 1000 pieces.
- Time of flight system design: Depth sensing architecture
- Time of flight system design: System overview