Event-based vision toolkit debuts - Embedded.com

Event-based vision toolkit debuts

Event-based vision is a growing field of research, and applications are starting to emerge. To accelerate the pace of innovation in machine vision systems, Paris-based Prophesee has launched its Metavision Intelligence Suite, a software toolkit that provides algorithms, code samples, and ready-to-use applications.

Inspired by human vision, Prophesee develops both neuromorphic sensors and machine learning algorithms whose function is inspired by the eye and brain. The company advocates an event-based vision approach to sensing and processing that selects the vitality of the scene and overlooks the irrelevant. This compares with traditional frame-based vision approaches.

In just one year, the French startup has come a long way. In September 2019, Prophesee rolled out what it claimed was the first event-based vision sensor, in an industry-standard package: its third-generation Metavision sensor. And after raising €25 million ($28 million) in funding, it jointly developed with Sony an event-based vision sensor where the pixel chip and the logic chip are stacked together. The logic incorporates signal processing circuits designed to detect changes in luminance based on an asynchronous delta modulation method. Today, Prophesee is launching its Metavision Intelligence Suite to accelerate the development of event-based solutions.

“We see our roles as being an enabler for the entire ecosystem needed to proliferate event-based vision more broadly,” said Lucas Verre, CEO of Prophesee. The development tools and technologies available through this offering will help jump start product developers regardless of where they are on the adoption or learning curve.”

The Metavision Intelligence Suite consists of 62 algorithms, 54 code samples, and 11-ready-to-use applications. And that’s the “initial offering,” Verre indicated. Still it provides a wide range of capabilities to allow engineers to experiment and iterate design concepts that are optimized for their specific needs.


(Source: Prophesee)

With its software suite, Prophesee expects engineers to explore and implement event-based applications in markets such as industrial automation, IoT, surveillance, mobile, medical, and automotive. “Plug-and-play” algorithms include high-speed counting, vibration monitoring, spatter monitoring, object tracking, optical flow, ultra-slow-motion, and machine learning.


Prophesee’s Metavision Intelligence Suite for the continuous vibration monitoring of machines

5 years

Under development for five years, the software development kit is the result of a collaborative effort internally and externally. “While Prophesee has more than 100 engineers and developers working on our solution, the Metavision Intelligence Suite was greatly aided by our work with hundreds of developers in a range of industries, from automotive to industrial automation to consumer,” said Verre. “Our experience with companies like Renault, Bosch, Intel, Thales, Safran, IBM, and many more has helped build out the technology and highly efficient tools to implement use cases that can benefit from event-based vision.”

Along the way, the French startup also worked with industrial camera makers like Imago or Century Arks, who have just announced the launch of a camera based on Metavision’s event-based technology and compatible with the Metavision Intelligence Suite. Prophesee is also engaged with various distribution partners such as Fujitsu, Macnica, Restar, and Okaya.

Discover, prototype, develop

The three components of the suite Player, Designer, and SDK provide engineers and software developers with the means to quickly iterate and develop their own applications and products, the company said. Each component is aimed at a specific stage of the design process.

The three modules of the suite work independently, and users can pick and choose according to their needs whether they are exploring event-based vision or already building machine vision products. “These components are modular and built to reflect both different levels of technological competencies as well as customer journey. They are designed to address a range of user types, their skills and needs,” said Verre. Who may be interested? Everyone from engineers to computer vision specialists to developers, to system integrators to camera manufacturers, he continued.

More specifically, Metavision Player is a good place to start, whether users own an evaluation kit or not. It features a graphical user interface allowing anyone to visualize and record data streamed by Prophesee-compatible event-based vision systems. Users can also read provided event datasets to deepen your understanding of event-based vision.

From there, Metavision Designer is aimed at engineers with basic Python knowledge and allows them to build algorithms, prototypes, and test applications quickly. It consists of a large set of libraries, Python APIs, and code examples built for quick and efficient integration and testing.

Finally, Metavision SDK is for developers who are developing high-performance event-based solutions. Algorithms are available via C++ API, ready to go to production.

Obviously, not starting from scratch can have immediate benefits for customers as fewer resources need to be involved. “For the development of the suite itself, we have spent more than five years and had 25 engineers and computer vision specialists working on proof of concepts, testing, and developing new applications,” Verre said. For customers to create their own specific application by themselves, he pursued, the investment would be significant, “probably many months of development and testing”. With this out-of-the-box solution, Prophesee expects its customers to not only save time and effort, but also feel confident that it has been tested and works with its Metavision event-based vision platform. “It’s hard to put a specific number or time frame on the benefits, as each coding project or application is unique, but it’s clearly a more efficient approach than trying to develop on your own and it frees up resources to focus on your real added-value and on differentiation for your system,” Verre commented.

Prophesee’s Metavision Intelligence Suite is available in a time-unlimited free trial version and a professional version. More precisely, the company is offering the suite in a free evaluation version so that people can try it out and see what aspects of it are best suited to their needs. And once they are ready to use in a production project, they can upgrade to the professional version, which includes access to source code, advanced add-ons, extensive support, and commercial rights, Verre specified. This requires a commercial licence, which is similar to any software engagement and works on a subscription model.

Prophesee said the support offering ranges from extensive documentation, online help and resources, as well as support packages and hands-on training workshops.

>> This article was originally published on our sister site, EE Times Europe.

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.