Image Blending in a High Frame Rate FPGA-based Multi-Camera System

Panoptic is a custom spherical light field camera used as a polydioptric system where imagers are distributed over a hemispherical surface, each having its own vision of the surroundings and a distinct focal plane. The spherical light field camera records light information from any direction around its center. This paper revises previously developed Nearest Neighbor and Linear blending techniques. Novel Gaussian blending and Restricted Gaussian blending techniques for vision reconstruction of a virtual observer located inside the spherical geometry are presented. These new blending techniques improve the output quality of the reconstructed image with respect to the ordinary stitching techniques and simpler image blending algorithms. A comparison of the developed blending algorithms is also given in this paper. A hardware architecture based on Field Programmable Gate Arrays (FPGA) enabling the real-time implementation of the blending algorithms is presented, along with the imaging results and resource utilization comparison. A recorded omnidirectional video is attached as a supplementary material

Large Field of View Multi-Camera Endoscopic Apparatus with Omni-Directional Illumination

A multi-camera hemispherical very wide field of view imaging apparatus with omnidirectional illumination capability comprises a cylindrical body (4, 4.a, 4.b), a hemispherical mechanical frame (2) arranged on one end of the cylindrical body (4, 4.a, 4.b), a plurality of imaging channels (3), each imaging channel (3) comprising at least an image sensor and related optics with a fixed focus appropriate for endoscopic imaging, the plurality of imaging channels (3) being distributed over the hemispherical mechanical frame (2), a light source arranged centre-down at a back part of the plurality of imaging channels (3) and inside or at the end of the cylindrical body (4, 4.a, 4.b). Each imaging channel (3) comprises a plurality of lightning channels (1) around their centre, each of the plurality of lightning channels (1) comprising at least one microfiber light guide having a determined angle of curvature arranged to transmit the light from the light source. The imaging apparatus further comprises a control and processing circuit (5) comprising a camera control unit (6), an illumination control unit (7), an illumination unit (8), a sample and capture unit (9), an image processing unit (10) and an output interface (11) to a PC. The camera control unit (6) is configured to power each of the plurality of imaging channels (3) and make automatic gain compensation for each imaging channel (3), the illumination control unit (7) is configured for automatic intensity dimming, the sample and capture unit (9) is an interface circuit for correct sampling, extraction and capturing frames of individual imaging channels (3), the image processing unit (10) is configured for constructing a spherical panoramic image by applying a determined algorithm, and the output interface (11) is arranged to output the spherical panoramic image to a system configured to visualize it

Large field of view multi-camera endoscopic apparatus with omni-directional illumination

A multi-camera hemispherical very wide field of view imaging apparatus with omnidirectional illumination capability comprises a cylindrical body (4, 4.a, 4.b), a hemispherical mechanical frame (2) arranged on one end of the cylindrical body (4, 4.a, 4.b), a plurality of imaging channels (3), each imaging channel (3) comprising at least an image sensor and related optics with a fixed focus appropriate for endoscopic imaging, the plurality of imaging channels (3) being distributed over the hemispherical mechanical frame (2), a light source arranged centre-down at a back part of the plurality of imaging channels (3) and inside or at the end of the cylindrical body (4, 4.a, 4.b). Each imaging channel (3) comprises a plurality of lightning channels (1) around their centre, each of the plurality of lightning channels (1) comprising at least one microfiber light guide having a determined angle of curvature arranged to transmit the light from the light source. The imaging apparatus further comprises a control and processing circuit (5) comprising a camera control unit (6), an illumination control unit (7), an illumination unit (8), a sample and capture unit (9), an image processing unit (10) and an output interface (11) to a PC. The camera control unit (6) is configured to power each of the plurality of imaging channels (3) and make automatic gain compensation for each imaging channel (3), the illumination control unit (7) is configured for automatic intensity dimming, the sample and capture unit (9) is an interface circuit for correct sampling, extraction and capturing frames of individual imaging channels (3), the image processing unit (10) is configured for constructing a spherical panoramic image by applying a determined algorithm, and the output interface (11) is arranged to output the spherical panoramic image to a system configured to visualize it

Design and implementation of real-time multi-sensor vision systems

This book discusses the design of multi-camera systems and their application to fields such as the virtual reality, gaming, film industry, medicine, automotive industry, drones, etc.The authors cover the basics of image formation, algorithms for stitching a panoramic image from multiple cameras, and multiple real-time hardware system architectures, in order to have panoramic videos. Several specific applications of multi-camera systems are presented, such as depth estimation, high dynamic range imaging, and medical imaging

Hemispherical Multiple Camera System for High Resolution Omni-Directional Light Field Imaging

Inspiration from light field imaging concepts has led to the construction of multi-aperture imaging systems. Using multiple cameras as individual apertures is a topic of high relevance in the light field imaging domain. The need for wide field-of-view (FOV) and high-resolution video for applications in areas of surveillance, robotics and automotive systems has driven the idea of omni-directional vision. Recently, the Panoptic camera concept has been presented that mimics the eyes of flying insects using multiple imagers. The Panoptic camera utilizes a novel methodology for constructing a spherically arranged wide FOV plenoptic imaging system whereas omni-directional image quality is limited by low resolution sensors. In this paper, a very-high resolution light field imaging and recording system inspired from the Panoptic approach is presented. Major challenges consisting of managing the huge amount of data as well as maintaining a scalable system are addressed. The proposed system is capable of recording omni-directional video at 30 fps with a resolution exceeding 9000 by 2400 pixels. The system is capable of capturing the surrounding light field in a FOV. This important feature opens the door to various post processing techniques such as quality-enhanced 3D cinematography, very high resolution depth map estimation and high dynamic-range applications which are beyond standard stitching and panorama generation

A New Omni-Directional Multi-Camera System for High Resolution Surveillance

Omni-directional high resolution surveillance has a wide application range in defense and security fields. The GigaEye-1 system, with 44 single cameras and 22 FPGAs, is capable of recording omni-directional video in a 360º×100º FOV at 9.5 fps with a resolution over (17,000x5,000) pixels (85MP). Real-time video capturing capability is also verified at 30 fps for a resolution over (9,000x2,400) pixels (22MP). The important capacity of GigaEye-1 opens the door to various post-processing techniques in surveillance domain such as large perimeter object tracking, very-high resolution depth map estimation and high dynamic-range imaging which are beyond standard stitching and panorama generation methods

FireBird: PowerPC e200 Based SoC for High Temperature Operation

PowerPC Architecture microcontrollers are commonly used in embedded applications. In this work we present FireBird, the first PowerPC based SoC for reliable operation beyond 200C. It is a significant challenge to design SoCs for reliable operation at high temperatures, due to increased static leakage current, reduced carrier mobility, and increased electromigration. To alleviate the consequences of high temperatures, this paper proposes to customize a PowerPC e200 based SoC by using a dynamically reconfigurable clock frequency, exhaustive clock gating, and electromigration-resistant power supply rings. A 20x9mm2 chip implementing this design has been fabricated in 0.35m CMOS technology. The custom testing procedure showed the expected maximum operating frequency reduction from 43MHz at room-temperature to 33MHz at 200C, which illustrates the importance of an adaptable clock frequency under temperature gradients. At 200C, the maximum power dissipation at 3.3V supply voltage was 1.75W and the idle state static leakage current was 0.2A. Silicon measurements proved that this design outclasses PowerPC based SoCs available in the high-temperature microcontrollers market which are not operational at temperatures above 125C