INTERMEDIATE VIEW RECONSTRUCTION FOR MULTISCOPIC 3D DISPLAY

This thesis focuses on Intermediate View Reconstruction (IVR) which generates additional images from the available stereo images. The main application of IVR is to generate the content of multiscopic 3D displays, and it can be applied to generate different viewpoints to Free-viewpoint TV (FTV). Although IVR is considered a good approach to generate additional images, there are some problems with the reconstruction process, such as detecting and handling the occlusion areas, preserving the discontinuity at edges, and reducing image artifices through formation of the texture of the intermediate image. The occlusion area is defined as the visibility of such an area in one image and its disappearance in the other one. Solving IVR problems is considered a significant challenge for researchers. In this thesis, several novel algorithms have been specifically designed to solve IVR challenges by employing them in a highly robust intermediate view reconstruction algorithm. Computer simulation and experimental results confirm the importance of occluded areas in IVR. Therefore, we propose a novel occlusion detection algorithm and another novel algorithm to Inpaint those areas. Then, these proposed algorithms are employed in a novel occlusion-aware intermediate view reconstruction that finds an intermediate image with a given disparity between two input images. This novelty is addressed by adding occlusion awareness to the reconstruction algorithm and proposing three quality improvement techniques to reduce image artifices: filling the re-sampling holes, removing ghost contours, and handling the disocclusion area. We compared the proposed algorithms to the previously well-known algorithms on each field qualitatively and quantitatively. The obtained results show that our algorithms are superior to the previous well-known algorithms. The performance of the proposed reconstruction algorithm is tested under 13 real images and 13 synthetic images. Moreover, analysis of a human-trial experiment conducted with 21 participants confirmed that the reconstructed images from our proposed algorithm have very high quality compared with the reconstructed images from the other existing algorithms

Stereoskopische Korrespondenzbestimmung mit impliziter Detektion von Okklusionen

Der Einsatz binokularer Sehsysteme eröffnet sowohl in der Natur als auch in der Technik die Möglichkeit zum räumlichen Sehen.Das Grundprinzip bildet hierbei eine passive Triangulation, deren Ausgangspunkte die korrespondierenden Positionen darstellen, auf die ein Raumpunkt in die Stereobilder projiziert wird. Das zentrale Problem besteht bei dieser Technik darin, die korrespondierenden Bildpunkte eindeutig einander zuzuordnen. Dieses sogenannte Korrespondenzproblem ist einerseits aufgrund mehrerer ähnlicher Strukturen in der betrachteten Szene oft stark mehrdeutig und besitzt andererseits nicht immer eine Lösung, da Bereiche in der Szeneauftreten können, die nur aus einer der beiden Perspektiven zusehen sind. Weiterhin wird eine eindeutige Zuordnung korrespondierender Bildbereiche durch interokuläre Differenzen wie perspektivische Verzerrungen, Beleuchtungsunterschiede und Rauschprozesse zusätzlich erschwert. In der vorliegenden Arbeit werden die einzelnen Komponenten eines Gesamtsystems vorgestellt, die zur stereoskopischen Rekonstruktion der räumlichen Struktur einer Szene erforderlich sind. Den Schwerpunkt der Arbeit bildet ein Selbstorganisationsprozeß, der in Verbindung mit weiteren Verfahrensschritten eine eindeutige Zuordnung korrespondierender Bildpunkte erlaubt. Darüber hinaus werden hierbei einseitig sichtbare Bildbereiche, die eine wesentliche Fehlerursache in der Stereoskopie darstellen, detektiert und vom Zuordnungsprozeß ausgeschlossen.Stereo vision is a passive method used to recover the depth information of a scene, which is lost during the projection of a point in the 3D-scene onto the 2D image plane. In stereo vision, in which two or more views of a scene are used, the depth information can be reconstructed from the different positions in the images to which a physical point in the 3D-scene is projected. The displacement of the corresponding positions in the image planes is called disparity. The central problem in stereo vision, known as the correspondence problem, is to find corresponding points or features in the images. This task can be an ambiguous one due to several similar structures or periodic elements in the images. Furthermore, there may be occluded regions in the scene, which can be seen only by one camera. In these regions there is no solution for the correspondence problem. Interocular differences such as perspective distortions, differences in illumination and camera noise make it even more difficult to solve the correspondence problem. The main focus of this work is a new stereo matching algorithm, in which the matching of occluded areas is suppressed by a self-organizing process. In the first step the images are filtered by a set of oriented Gabor filters. A complex valued correlation-based similarity measurement, which is applied to the responses of the Gabor filters, is used in the second step to initialize a self-organizing process. In this self-organizing network, which is described by coupled, non-linear evolution equations, the continuity and the uniqueness constraints are established. Occlusions are detected implicitly without a computationally intensive bidirectional matching strategy.von Dipl.-Ing. Ralph Trapp aus Winterberg. Referent: Prof. Dr. rer. nat Georg Hartmann, Korreferent: Prof. Dr.-Ing. Ulrich RückertTag der Verteidigung: 15.09.1998Universität Paderborn, Univ., Dissertation, 199

Algorithms for VLSI stereo vision circuits applied to autonomous robots

Since the inception of Robotics, visual information has been incorporated in order to allow the robots to perform tasks that require an interaction with their environment, particularly when it is a changing environment. Depth perception is a most useful information for a mobile robot to navigate in its environment and interact with its surroundings. Among the different methods capable of measuring the distance to the objects in the scene, stereo vision is the most advantageous for a small, mobile robot with limited energy and computational power. Stereoscopy implies a low power consumption because it uses passive sensors and it does not require the robot to move. Furthermore, it is more robust, because it does not require a complex optic system with moving elements. On the other hand, stereo vision is computationally intensive. Objects in the scene have to be detected and matched across images. Biological sensory systems are based on simple computational elements that process information in parallel and communicate among them. Analog VLSI chips are an ideal substrate to mimic the massive parallelism and collective computation present in biological nervous systems. For mobile robotics they have the added advantage of low power consumption and high computational power, thus freeing the CPU for other tasks. This dissertation discusses two stereoscopic methods that are based on simple, parallel cal- culations requiring communication only among neighboring processing units (local communication). Algorithms with these properties are easy to implement in analog VLSI and they are also very convenient for digital systems. The first algorithm is phase-based. Disparity, i.e., the spatial shift between left and right images, is recovered as a phase shift in the spatial-frequency domain. Gábor functions are used to recover the frequency spectrum of the image because of their optimum joint spatial and spatial-frequency properties. The Gábor-based algorithm is discussed and tested on a Khepera miniature mobile robot. Two further approximations are introduced to ease the analog VLSI and digital implementations. The second stereoscopic algorithm is difference-based. Disparity is recovered by a simple calculation using the image differences and their spatial derivatives. The algorithm is simulated on a digital system and an analog VLSI implementation is proposed and discussed. The thesis concludes with the description of some tools used in this research project. A stereo vision system has been developed for the Webots mobile robotics simulator, to simplify the testing of different stereo algorithms. Similarly, two stereo vision turrets have been built for the Khepera robot