Validation of a light field camera for investigation on complex plasma

Ekoplasma is going to be the fourth-generation laboratory for the investigation of complex plasmas under microgravity conditions on the International Space Station (ISS). For the first time, particle motion is aimed to be observed three-dimensional by a light field camera under microgravity. The main features of a light field camera were explored. And a three-dimensional calibration target for the validation process was designed and approved. Overall, due to its design this target provides the same capabilities for every imaging system. It therefore is a universal device that will help to develop and classify an imaging system applicable for three dimensional plasma research on the ISS

Multidimensional Optical Sensing and Imaging Systems (MOSIS): From Macro to Micro Scales

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.B. Javidi wishes to acknowledge support by the National Science Foundation (NSF) under Grant NSF/IIS-1422179, and DARPA and US Army under contract number W911NF-13-1-0485. The work of P. Latorre Carmona, A. Martínez-Uso, J. M. Sotoca and F. Pla was supported by the Spanish Ministry of Economy under the project ESP2013-48458-C4-3-P, and by MICINN under the project MTM2013-48371-C2-2-PDGI, by Generalitat Valenciana under the project PROMETEO-II/2014/062, and by Universitat Jaume I through project P11B2014-09. The work of M. Martínez-Corral and G. Saavedra was supported by the Spanish Ministry of Economy and Competitiveness under the grant DPI2015-66458-C2-1R, and by the Generalitat Valenciana, Spain under the project PROMETEOII/2014/072

Three-dimensional imaging with multiple degrees of freedom using data fusion

This paper presents an overview of research work and some novel strategies and results on using data fusion in 3-D imaging when using multiple information sources. We examine a variety of approaches and applications such as 3-D imaging integrated with polarimetric and multispectral imaging, low levels of photon flux for photon-counting 3-D imaging, and image fusion in both multiwavelength 3-D digital holography and 3-D integral imaging. Results demonstrate the benefits data fusion provides for different purposes, including visualization enhancement under different conditions, and 3-D reconstruction quality improvement

Progressive Refinement Imaging

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. // Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory‐efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. // We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras

3D panoramic imaging for virtual environment construction

The project is concerned with the development of algorithms for the creation of photo-realistic 3D virtual environments, overcoming problems in mosaicing, colour and lighting changes, correspondence search speed and correspondence errors due to lack of surface texture. A number of related new algorithms have been investigated for image stitching, content based colour correction and efficient 3D surface reconstruction. All of the investigations were undertaken by using multiple views from normal digital cameras, web cameras and a ”one-shot” panoramic system. In the process of 3D reconstruction a new interest points based mosaicing method, a new interest points based colour correction method, a new hybrid feature and area based correspondence constraint and a new structured light based 3D reconstruction method have been investigated. The major contributions and results can be summarised as follows: • A new interest point based image stitching method has been proposed and investigated. The robustness of interest points has been tested and evaluated. Interest points have been proved robust to changes in lighting, viewpoint, rotation and scale. • A new interest point based method for colour correction has been proposed and investigated. The results of linear and linear plus affine colour transforms have proved more accurate than traditional diagonal transforms in accurately matching colours in panoramic images. • A new structured light based method for correspondence point based 3D reconstruction has been proposed and investigated. The method has been proved to increase the accuracy of the correspondence search for areas with low texture. Correspondence speed has also been increased with a new hybrid feature and area based correspondence search constraint. • Based on the investigation, a software framework has been developed for image based 3D virtual environment construction. The GUI includes abilities for importing images, colour correction, mosaicing, 3D surface reconstruction, texture recovery and visualisation. • 11 research papers have been published.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Localisation and tracking of stationary users for extended reality

In this thesis, we investigate the topics of localisation and tracking in the context of Extended Reality. In many on-site or outdoor Augmented Reality (AR) applications, users are standing or sitting in one place and performing mostly rotational movements, i.e. stationary. This type of stationary motion also occurs in Virtual Reality (VR) applications such as panorama capture by moving a camera in a circle. Both applications require us to track the motion of a camera in potentially very large and open environments. State-of-the-art methods such as Structure-from-Motion (SfM), and Simultaneous Localisation and Mapping (SLAM), tend to rely on scene reconstruction from significant translational motion in order to compute camera positions. This can often lead to failure in application scenarios such as tracking for seated sport spectators, or stereo panorama capture where the translational movement is small compared to the scale of the environment. To begin with, we investigate the topic of localisation as it is key to providing global context for many stationary applications. To achieve this, we capture our own datasets in a variety of large open spaces including two sports stadia. We then develop and investigate these techniques in the context of these sports stadia using a variety of state-of-the-art localisation approaches. We cover geometry-based methods to handle dynamic aspects of a stadium environment, as well as appearance-based methods, and compare them to a state-of-the-art SfM system to identify the most applicable methods for server-based and on-device localisation. Recent work in SfM has shown that the type of stationary motion that we target can be reliably estimated by applying spherical constraints to the pose estimation. In this thesis, we extend these concepts into a real-time keyframe-based SLAM system for the purposes of AR, and develop a unique data structure for simplifying keyframe selection. We show that our constrained approach can track more robustly in these challenging stationary scenarios compared to state-of-the-art SLAM through both synthetic and real-data tests. In the application of capturing stereo panoramas for VR, this thesis demonstrates the unsuitability of standard SfM techniques for reconstructing these circular videos. We apply and extend recent research in spherically constrained SfM to creating stereo panoramas and compare this with state-of-the-art general SfM in a technical evaluation. With a user study, we show that the motion requirements of our SfM approach are similar to the natural motion of users, and that a constrained SfM approach is sufficient for providing stereoscopic effects when viewing the panoramas in VR

Vers un modèle unifié pour l'affichage autostéréoscopique d'images

Dans un premier chapitre, nous décrivons un modèle de formation d’image affichée sur un écran en revenant sur les concepts que sont la lumière, la géométrie et l’optique. Nous détaillons ensuite les différentes techniques d’affichage stéréoscopique utilisées à l’heure actuelle, en parcourant la stéréoscopie, l’autostéréoscopie et plus particulièrement le principe d’imagerie intégrale. Le deuxième chapitre introduit un nouveau modèle de formation d’image stéréoscopique. Ce dernier nous permet d’observer deux images d’une paire stéréoscopique soumises à des éventuelles transformations et à l’effet d’une ou de plusieurs optiques particulières, pour reproduire la perception de trois dimensions. Nous abordons l’aspect unificateur de ce modèle. En effet il permet de décrire et d’expliquer de nombreuses techniques d’affichage stéréoscopique existantes. Enfin, dans un troisième chapitre nous discutons d’une méthode particulière de création de paires d’images stéréoscopiques à l’aide d’un champ lumineux

Calibrage et modélisation d’un système de stéréovision hybride et panoramique

Dans cette thèse nos contributions à la résolution de deux problématiques rencontrées en vision numérique et en photogrammétrie, qui sont le calibrage de caméras et la stéréovision, sont présentées. Ces deux problèmes font l’objet de très nombreuses recherches depuis plusieurs années. Les techniques de calibrage existantes diffèrent beaucoup suivant le type de caméras à calibrer (classique ou panoramique, à focale fixe ou à focale variable, ..). Notre première contribution est un banc de calibrage, à l’aide des éléments d’optique diffractive, qui permet de calibrer avec une bonne précision une très grande partie des caméras existantes. Un modèle simple et précis qui décrit la projection de la grille formée sur l’image et une méthode de calibrage pour chaque type de caméras est proposé. La technique est très robuste et les résultats pour l’ensemble des caméras calibrées sont optimaux. Avec la multiplication des types de caméras et la diversité des modèles de projections, un modèle de formation d'image générique semble très intéressant. Notre deuxième contribution est un modèle de projection unifié pour plusieurs caméras classiques et panoramiques. Dans ce modèle, toute caméra est modélisée par une projection rectiligne et des splines cubiques composées permettant de représenter toutes sortes de distorsions. Cette approche permet de modéliser géométriquement les systèmes de stéréovision mixtes ou panoramiques et de convertir une image panoramique en une image classique. Par conséquent, le problème de stéréovision mixte ou panoramique est transformé en un problème de stéréovision conventionnelle. Mots clés : calibrage, vision panoramique, distorsion, fisheye, zoom, panomorphe, géométrie épipolaire, reconstruction tridimensionnelle, stéréovision hybride, stéréovision panoramique.This thesis aims to present our contributions to the resolution of two problems encountered in the field of computer vision and photogrammetry, which are camera calibration and stereovision. These two problems have been extensively studied in the last years. Different camera calibration techniques have been developed in the literature depending on the type of camera (classical or panoramic, with zoom lens or fixed lens..). Our first contribution is a compact and accurate calibration setup, based on diffractive optical elements, which is suitable for different kind of cameras. The technique is very robust and optimal results were achieved for different types of cameras. With the multiplication of camera types and the diversity of the projection models, a generic model has become very interesting. Our second contribution is a generic model, which is suitable for conventional and panoramic cameras. In this model, composed cubic splines functions provide more realistic model of both radial and tangential distortions. Such an approach allows to model either hybrid or panoramic stereovision system and to convert panoramic image to classical image. Consequently, the processing challenges of a hybrid stereovision system or a panoramic stereovision system are turned into simple classical stereovision problems. Keywords: Calibration, panoramic vision, distortions, fisheye, zoom, panomorph, epipolar geometry, three-dimensional reconstruction, hybrid stereovision, panoramic stereovision