A three-year European Union (EU) funded project has concluded with the delivery of a comprehensive reference platform for vision-based embedded system designers. This will enable those engineers to balance low power, low latency, high performance and real-time image processing design constraints.
The Tulipp (Towards Ubiquitous Low-power Image Processing Platforms) Consortium, which received €4 million (approximately US $4.5 million) from the EU’s Horizon 2020 research and innovation program, began in January 2016 to develop high-performance, energy-efficient systems for complex, vision-based image processing applications.
The resulting reference platform includes a full development kit, comprising an FPGA-based embedded, multicore computing board, parallel real-time operating system and development tool chain with guidelines, coupled with ‘real world’ use cases focusing on medical x-ray imaging, driver assistance and autonomous drones with obstacle avoidance. The complete Tulipp ecosystem was demonstrated earlier in the year to vision-based system designers in a series of hands-on tutorials.
Philippe Millet of Thales, who was also Tulipp’s project co-ordinator, said, “By taking a diverse range of application domains as the basis for defining a common reference processing platform that captures the commonality of real-time, high-performance image processing and vision applications, it has successfully addressed the fundamental challenges facing today’s embedded vision-based system designers.”
Developed by Sundance Multiprocessor Technology, the Tulipp processing platform is 40mm x 50mm and is compliant with the PC/104 embedded processor board standard. The hardware platform utilizes the multicore Xilinx Zynq Ultrascale+ MPSoC which contains, along with the Xilinx FinFET+ FPGA, an ARM Cortex-A53 quad-core CPU, an ARM Mali-400 MP2 graphics processing unit (GPU), and a real-time processing unit (RPU) containing a dual-core ARM Cortex-R5 32-bit real-time processor based on the ARM-v7R architecture. A separate expansion module (VITA57.1 FMC) allows application-specific boards with different input and output interfaces to be created while keeping the interfaces with the processing module consistent.
Medical X-ray image before (left) and after (right) processing to remove sensor noise. (Image: Tulipp)
Coupled with the Tulipp hardware platform, is a parallel, low latency embedded real-time operating system developed by Hipperos specifically to managecomplex multi-threaded embedded applications i n a predictable manner.
Additionally, in order to help designers understand the impact of their functional mapping and scheduling choices on the available resources, the Tulipp reference platform has been extended with performance analysis and power measurement features developed by Norges Teknisk-Naturvitenskapelige Universitet (NTNU) and Technische Universität Dresden (TUD) and implemented in the Tulipp STHEM toolchain.
The insights of the Tulipp Consortium have been captured in a set of guidelines, consisting of practical advice, best practice approaches and recommended implementation methods, to help vision-based system designers select the optimal implementation strategy for their own applications. This will become a TULIPP book to be published by Springer by the end of 2019 and supported by endorsements from the growing ecosystem of developers that are currently testing the concept.