Multidimensional Optical Sensing and Imaging Systems (MOSIS): From Macro to Micro Scales

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.B. Javidi wishes to acknowledge support by the National Science Foundation (NSF) under Grant NSF/IIS-1422179, and DARPA and US Army under contract number W911NF-13-1-0485. The work of P. Latorre Carmona, A. Martínez-Uso, J. M. Sotoca and F. Pla was supported by the Spanish Ministry of Economy under the project ESP2013-48458-C4-3-P, and by MICINN under the project MTM2013-48371-C2-2-PDGI, by Generalitat Valenciana under the project PROMETEO-II/2014/062, and by Universitat Jaume I through project P11B2014-09. The work of M. Martínez-Corral and G. Saavedra was supported by the Spanish Ministry of Economy and Competitiveness under the grant DPI2015-66458-C2-1R, and by the Generalitat Valenciana, Spain under the project PROMETEOII/2014/072

Three-dimensional imaging with multiple degrees of freedom using data fusion

This paper presents an overview of research work and some novel strategies and results on using data fusion in 3-D imaging when using multiple information sources. We examine a variety of approaches and applications such as 3-D imaging integrated with polarimetric and multispectral imaging, low levels of photon flux for photon-counting 3-D imaging, and image fusion in both multiwavelength 3-D digital holography and 3-D integral imaging. Results demonstrate the benefits data fusion provides for different purposes, including visualization enhancement under different conditions, and 3-D reconstruction quality improvement

An overview of head tracking integral imaging three-dimensional display using smart pseudoscopic-to-orthoscopic conversion

We overview a previously reported head tracking integral imaging three-dimensional (3D) display to extend viewing angle accommodated to a viewer's position without the crosstalk phenomenon. A head detection system is applied to obtain the head position and rotation of a viewer, and a new set of elemental images is then computed using the smart pseudoscopic-to-orthoscopic conversion (SPOC) method for head tracking 3D display. Experimental results validate the proposed method for high quality 3D display with large viewing angle

Information theoretic approach for assessing image fidelity in photon-counting arrays

The method of photon-counting integral imaging has been introduced recently for three-dimensional object sensing, visualization, recognition and classification of scenes under photon-starved conditions. This paper presents an information-theoretic model for the photon-counting imaging (PCI) method, thereby providing a rigorous foundation for the merits of PCI in terms of image fidelity. This, in turn, can facilitate our understanding of the demonstrated success of photon-counting integral imaging in compressive imaging and classification. The mutual information between the source and photon-counted images is derived in a Markov random field setting and normalized by the source-image’s entropy, yielding a fidelity metric that is between zero and unity, which respectively corresponds to complete loss of information and full preservation of information. Calculations suggest that the PCI fidelity metric increases with spatial correlation in source image, from which we infer that the PCI method is particularly effective for source images with high spatial correlation; the metric also increases with the reduction in photon-number uncertainty. As an application to the theory, an image-classification problem is considered showing a congruous relationship between the fidelity metric and classifier’s performance

Three Dimensional Visualization of Long Range Scenes by Photon Counting Mid-Wave Infrared Integral Imaging

Integral Imaging under photon counting conditions has found different three-dimensional (3D) imaging applications, including 3D image reconstruction and recognition. In this letter, we present the application of the maximum likelihood (ML) estimation method for visualization of 3D scenes in photon starved environments using Mid-Wave Infrared 3D data of real scenes acquired at distances ranging from 50m to more than 2km. To the best of our knowledge, this is the first report on Mid-Wave Infrared 3D photon counting integral imaging of distant scenes

Analysis of the depth of field of integral imaging displays based on wave optics

In this paper, we analyze the depth of field (DOF) of integral imaging displays based on wave optics. With considering the diffraction effect, we analyze the intensity distribution of light with multiple microlenses and derive a DOF calculation formula for integral imaging display system. We study the variations of DOF values with different system parameters. Experimental results are provided to verify the accuracy of the theoretical analysis. The analyses and experimental results presented in this paper could be beneficial for better understanding and designing of integral imaging displays

Polarimetric 3D integral imaging in photon-starved conditions

We develop a method for obtaining 3D polarimetric integral images from elemental images recorded in low light illumination conditions. Since photon-counting images are very sparse, calculation of the Stokes parameters and the degree of polarization should be handled carefully. In our approach, polarimetric 3D integral images are generated using the Maximum Likelihood Estimation and subsequently reconstructed by means of a Total Variation Denoising filter. In this way, polarimetric results are comparable to those obtained in conventional illumination conditions. We also show that polarimetric information retrieved from photon starved images can be used in 3D object recognition problems. To the best of our knowledge, this is the first report on 3D polarimetric photon counting integral imaging

Enhanced field-of-view integral imaging display using multi-Köhler illumination

A common drawback in 3D integral imaging displays is the appearance of pseudoimages beyond the viewing angle. These pseudoimages appear when the light rays coming from each elemental image are not passing through the corresponding microlens, and a set of barriers must be used to avoid this flipping effect. We present a pure optical arrangement based on Köhler illumination to generate these barriers thus avoiding the pseudoimages. The proposed system does not use additional lenses to project the elemental images, so no optical aberrations are introduced. As an added benefit, Köhler illumination provides a higher contrast 3D display.This work was supported in part by the Plan Nacional I + D + I, under Grants DPI2012-32994 and ENE2013-48565-C2-2-P, Ministerio de Economía y Competitividad (Spain), and also by the Generalitat Valenciana (Spain) under Grant PROMETEOII/2014/072. The work of Ángel Tolosa was supported by the Generalitat Valenciana under IVACE Grant PROMECE 2014

Multispectral integral imaging acquisition and processing using a monochrome camera and a liquid crystal tunable filter

This paper presents an acquisition system and a procedure to capture 3D scenes in different spectral bands. The acquisition system is formed by a monochrome camera, and a Liquid Crystal Tunable Filter (LCTF) that allows to acquire images at different spectral bands in the [480, 680]nm wavelength interval. The Synthetic Aperture Integral Imaging acquisition technique is used to obtain the elemental images for each wavelength. These elemental images are used to computationally obtain the reconstruction planes of the 3D scene at different depth planes. The 3D profile of the acquired scene is also obtained using a minimization of the variance of the contribution of the elemental images at each image pixel. Experimental results show the viability to recover the 3D multispectral information of the scene. Integration of 3D and multispectral information could have important benefits in different areas, including skin cancer detection, remote sensing and pattern recognition, among others.This work was supported in part by the Spanish Ministry of Science and Innovation under the projects ALFI3D TIN 2009-14103-C 03-01, Consolider Ingenio 2010CSD 2007- 00018, EODIX AYA 2008-05965-C04-04/ESP, and FIS2009-9135, and also by the Generalitat Valen- ciana through the projects PROMETEO/2010/028 and PROMETEO2009-077