Image capture and video-based applications have come a long way in the past decade. Image quality has made major leaps with the transition from HDTV to 4k ultra HD (and beyond), the introduction of high dynamic range, and the broadening of the spectral range.
The purpose of this project is to further improve image capture systems for high-end security and broadcast, while lowering cost and opening the path for new functionality and ease of use. The core of MANTIS vision is the development of a multi-modal, multi µ-camera system connected to an artificial intelligence (AI) capable edge processing box. To this extent:
- A new µ-camera architecture will be designed supporting single- and dual mode capture: combining visual with either Near Infra Red (NIR) or Long Wave Infra Red (LWIR).
- The edge processing box will support existing image processing pipelines for cost-effective use cases. It will also include new deep learning (DL) based, real-time algorithms for new applications like DL-based image fusion, and real-time free view point rendering and 3D reconstruction.
- For the system, new image sensors will be developed for NIR, LWIR, and the visible spectrum to further improve image quality, and to research integration of intelligence on the image sensor.
The above described system will support surveillance applications (high-end security, personal vision systems) and live television productions, both part of digital life.
In terms of utilization:
- For high-end security systems, the project will deliver new solutions that allow users to see more, farther and longer at the same size, weight, power, and cost, without increasing the complexity for the end user.
- For broadcast, project results will help content creators for live television productions support the evolution of mobile technology to transform the way people around the world consume media content. Project results will furthermore help drive the expansion in overall media consumption. The new micro camera system that will be developed will broaden the product portfolios and enable the path into new AI applications like bird flight views in a stadium environment.
Some partners (Adimec, IDEAS, AMS, TNO and Grass Valley) will collaborate to evaluate multi-camera solutions in the LWIR, NIR and visible range for surveillance applications. Image Sensor Components will be developed and improved by AMS, Grass Valley, Delft Technical University and IDEAS. Developments will cover new image sensors for visual, NIR and LWIR.