Gesture recognition--first step toward 3D UIs?

Dong-Ik Ko and Gaurav Agarwal, Texas Instruments

December 2, 2011

Dong-Ik Ko and Gaurav Agarwal, Texas Instruments

Technology processing steps
For many applications, both a 2D and 3D camera system will be needed to properly enable the technology. Figure 2 shows the basic data path of these systems. Getting the data from the sensors and into the vision analytics is not as simple as it seems from the data path. Specifically, TOF sensors require up to 16 times the bandwidth of 2D sensors, causing a big input/output (I/O) problem. Another bottleneck occurs in the processing from the raw 3D data to a 3D point cloud. Having the right combination of software and hardware to address these issues is critical for gesture recognition and 3D success. Today, this data path is realized in DSP/GPP combination processors along with discrete analog components and software libraries.


Click on image to enlarge.

Challenges for 3d-vision embedded systems
Input challenges: As discussed, input bandwidth constraints are a considerable challenge for 3D-vision embedded systems. Additionally, there is no standardization for the input interface. Designers can choose to work with different options, including serial and parallel interfaces for 2D sensor and general purpose external memory interfaces. Until a standard input interface is developed with the best possible bandwidth, designers will have to work with what is available.

Two different processor architectures: The 3D depth map processing of Figure 2 can be divided into two categories: vision specific, data-centric processing and application upper-level processing. Vision specific, data-centric processing requires a processor architecture that can perform single instruction, multiple data (SIMD), fast floating-point multiplication and addition, and fast search algorithms. A DSP is a perfect candidate for quickly and reliably performing this type of processing. For application upper-level-processing, high-level operating systems (OSes) and stacks can provide the necessary feature set that the upper layer of any application needs.

Based on the requirements for both processor architectures, a system-on-chip (SoC) that provides a GPP+ DSP+ SIMD processor with a high data rate I/O is a good fit for 3D vision processing, providing the necessary data and application upper level processing.

Lack of standard middleware: The world of middleware for 3D vision processing is a combination of many different pieces pulled together from multiple sources, including open source (for example, OpenCV) as well as proprietary commercial sources. Commercial libraries are targeted for body tracking applications, which is a specific application of 3D vision. No standardized middleware interface has been developed yet for all the different 3D vision applications.

Anything cool after "z"?
While no one questions the "cool" factor of 3D vision, researchers are already looking into new ways to see beyond, through, and inside people and objects. Using multi-path light analysis, researchers around the world are looking for ways to see around corners or objects. Transparence research will yield systems that are able to see through objects and materials. And with emotion detection systems, applications will be able to see inside the human mind to detect whether the person is lying.

The possibilities are endless when it comes to 3D vision and gesture recognition technologies. But the research will be for nothing if the hardware and middleware needed to support these exciting new technologies are not there. Moving forward, SoCs that provide a GPP+DSP +SIMD architecture will be able to deliver the right mix of processing performance with peripheral support and the necessary bandwidth to enable this exciting technology and its applications.

Dong-Ik Ko is a technical lead in 3D vision business unit at Texas Instruments. He has more than 18 years experience in industry and academy research on embedded system design and optimization methods. He has a master of science in electrical engineering from Korea University and a Ph.D. in computer engineering from University of Maryland College Park.

Gaurav Agarwal a business development manager at Texas Instruments, where he identifies growth areas for the DaVinci digital media processor business. He holds a bachelor of technology in electrical engineering from the Indian Institute of Technology, Kanpur and a master of science in electronic engineering from University of Maryland.

This article provided courtesy of Embedded.com and Embedded Systems Design Magazine. Sign up for subscriptions and newsletters. Copyright © 2011 UBM--All rights reserved.