Performance Analysis on Stereo Matching Algorithms Based on Local and Global Methods for 3D Images Application

Stereo matching is one of the methods in computer vision and image processing. There have numerous algorithms that have been found associated between disparity maps and ground truth data. Stereo Matching Algorithms were applied to obtain high accuracy of the depth as well as reducing the computational cost of the stereo image or video. The smoother the disparity depth map, the better results of triangulation can be achieved. The selection of an appropriate set of stereo data is very important because these stereo pairs have different characteristics. This paper discussed the performance analysis on stereo matching algorithm through Peak Signal to Noise Ratio (PSNR in dB), Structural Similarity (SSIM), the effect of window size and execution time for different type of techniques such as Sum Absolute Differences (SAD), Sum Square Differences (SSD), Normalized Cross Correlation (NCC), Block Matching (BM), Global Error Energy Minimization by Smoothing Functions, Adapting BP and Dynamic Programming (DP). The dataset of stereo images that used for the experimental purpose is obtained from Middlebury Stereo Datasets

Fast GPU Accelerated Stereo Correspondence for Embedded Surveillance Camera Systems

Many surveillance applications could benefit from the use of stereo cam- eras for depth perception. While state-of-the-art methods provide high quality scene depth information, many of the methods are very time consuming and not suitable for real-time usage in limited embedded systems. This study was conducted to examine stereo correlation methods to find a suitable algorithm for real-time or near real-time depth perception through disparity maps in a stereo video surveillance camera with an embedded GPU. Moreover, novel refinements and alternations was investigated to further improve performance and quality. Quality tests were conducted in Octave while GPU suitability and performance tests were done in C++ with the OpenGL ES 2.0 library. The result is a local stereo correlation method using Normalized Cross Correlation together with sparse support windows and a suggested improvement for pixel-wise matching confidence. Applying sparse support windows increased frame rate by 35% with minimal quality penalty as compared to using full support windows

Color Stereo Vision Using Hierarchical Block Matching and Active Color Illumination

Stereo is a well-known technique for obtaining depth information from digital images. Nevertheless, this technique still suffers from a lack in accuracy and/or long computation time needed to match stereo images. A new hierarchical algorithm using an image pyramid for obtaining dense depth maps from color stereo images is presented. We show that matching results of high quality are obtained when using the new hierarchical chromatic Block Matching algorithm. Most stereo matching algorithms can not compute correct dense depth maps in homogenous image regions. This paper shows that using an active color illumination will considerably improve the quality of the matching results. We present results for synthetic and for real images. 1. Introduction During the past few years the development of stereo algorithms has been a subject of considerable research activity. The key problem in stereo is how to find the corresponding points in the left and in the right image, referred to as the corresp..

Design und Implementierung eines Systems zur schnellen Rekonstruktion dreidimensionaler Modelle aus Stereobildern

Im Rahmen dieser Arbeit wurde ein aus Hard- und Software bestehendes System zur schnellen Rekonstruktion dreidimensionaler Oberflächen entwickelt. Ausgehend von einer geplanten Anwendung, die mit existierenden Systemen nicht realisierbar war, wurde zunächst festgestellt wo die Stärken und Schwächen der betrachteten Systeme lagen, darauf basierend ein geeignetes Verfahren gewählt und die zu lösenden Teilaufgaben identifiziert. Die Arbeit konzentriert sich auf die Entwicklung möglichst allgemein verwendbarer Kernalgorithmen, ohne dabei die geplanten Anwendungen aus den Augen zu verlieren. Insbesondere wurde auf modulares Design geachtet, so daß sich die einzelnen Bausteine leicht für beliebige Anwendungen verwenden lassen, die eine Funktionalität zur 3D-Rekonstruktion benötigen

Efficient Techniques for High Resolution Stereo

The purpose of stereo is extracting 3-dimensional (3D) information from 2-dimensional (2D) images, which is a fundamental problem in computer vision. In general, given a known imaging geometry the position of any 3D point observed by two or more different views can be recovered by triangulation, so 3D reconstruction task relies on figuring out the pixel’s correspondence between the reference and matching images. In general computational complexity of stereo algorithms is proportional to the image resolution (the total number of pixels) and the search space (the number of depth candidates). Hence, high resolution stereo tasks are not tractable for many existing stereo algorithms whose computational costs (including the processing time and the storage space) increase drastically with higher image resolution. The aim of this dissertation is to explore techniques aimed at improving the efficiency of high resolution stereo without any accuracy loss. The efficiency of stereo is the first focus of this dissertation. We utilize the implicit smoothness property of the local image patches and propose a general framework to reduce the search space of stereo. The accumulated matching costs (measured by the pixel similarity) are investigated to estimate the representative depths of the local patch. Then, a statistical analysis model for the search space reduction based on sequential probability ratio test is provided, and an optimal sampling scheme is proposed to find a complete and compact candidate depth set according to the structure of local regions. By integrating our optimal sampling schemes as a pre-processing stage, the performance of most existing stereo algorithms can be significantly improved. The accuracy of stereo algorithms is the second focus. We present a plane-based approach for the local geometry estimation combining with a parallel structure propagation algorithm, which outperforms most state-of-the-art stereo algorithms. To obtain precise local structures, we also address the problem of utilizing surface normals, and provide a framework to integrate color and normal information for high quality scene reconstruction.Doctor of Philosoph

Depth Estimation from a Single Holoscopic 3D Image and Image Up-sampling with Deep-learning

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London3D depth information is widely utilized in industries such as security, autonomous vehicles, robotics, 3D printing, AR/VR entertainment, cinematography and medical science. However, state-of-the-art imaging and 3D depth-sensing technologies are rather complicated or expensive and still lack scalability and interoperability. The research identified, entails the development of an innovative technique for reliable and efficient 3D depth estimation that deliver better accuracy. The proposed (1) multilayer Holoscopic 3D encoding technique reduces the computational cost of extracting viewpoint images from complex structured Holoscopic 3D data by 95%, by using labelled multilayer elemental images. It also addresses misplacement of elemental image pixels due to lens distortion error. The multilayer Holoscopic 3D encoding computing efficiency leads to the implementation of real-time 3D depth-dependent applications. Also, (2) an innovative approach of a deep learning-based single image super-resolution framework is developed and evaluated. It identified that learning-based image up-sampling techniques could be used regardless of inadequate 3D training data, as 2D training data can yield the same results. (3) The research is extended further by implementation of an H3D depth disparity -based framework, where a Holoscopic content adaptation technique for extracting semi-segmented stereo viewpoint image is introduced, and the design of a smart 3D depth mapping technique is proposed. Particularly, it provides a somewhat accurate 3D depth estimation from H3D images in near real-time. Holoscopic 3D image has thousands of perspective elemental images from omnidirectional viewpoint images and (4) a novel 3D depth estimation technique is developed to estimates 3D depth information directly from a single Holoscopic 3D image without the loss of any angular information and the introduction of unwanted artefacts. The proposed 3D depth measurement techniques are computationally efficient and robust with high accuracy; these can be incorporated in real-time applications of autonomous vehicles, security and AR/VR for real-time interaction

Development of an augmented reality guided computer assisted orthopaedic surgery system

Previously held under moratorium from 1st December 2016 until 1st December 2021.This body of work documents the developed of a proof of concept augmented reality guided computer assisted orthopaedic surgery system – ARgCAOS. After initial investigation a visible-spectrum single camera tool-mounted tracking system based upon fiducial planar markers was implemented. The use of visible-spectrum cameras, as opposed to the infra-red cameras typically used by surgical tracking systems, allowed the captured image to be streamed to a display in an intelligible fashion. The tracking information defined the location of physical objects relative to the camera. Therefore, this information allowed virtual models to be overlaid onto the camera image. This produced a convincing augmented experience, whereby the virtual objects appeared to be within the physical world, moving with both the camera and markers as expected of physical objects. Analysis of the first generation system identified both accuracy and graphical inadequacies, prompting the development of a second generation system. This too was based upon a tool-mounted fiducial marker system, and improved performance to near-millimetre probing accuracy. A resection system was incorporated into the system, and utilising the tracking information controlled resection was performed, producing sub-millimetre accuracies. Several complications resulted from the tool-mounted approach. Therefore, a third generation system was developed. This final generation deployed a stereoscopic visible-spectrum camera system affixed to a head-mounted display worn by the user. The system allowed the augmentation of the natural view of the user, providing convincing and immersive three dimensional augmented guidance, with probing and resection accuracies of 0.55±0.04 and 0.34±0.04 mm, respectively.This body of work documents the developed of a proof of concept augmented reality guided computer assisted orthopaedic surgery system – ARgCAOS. After initial investigation a visible-spectrum single camera tool-mounted tracking system based upon fiducial planar markers was implemented. The use of visible-spectrum cameras, as opposed to the infra-red cameras typically used by surgical tracking systems, allowed the captured image to be streamed to a display in an intelligible fashion. The tracking information defined the location of physical objects relative to the camera. Therefore, this information allowed virtual models to be overlaid onto the camera image. This produced a convincing augmented experience, whereby the virtual objects appeared to be within the physical world, moving with both the camera and markers as expected of physical objects. Analysis of the first generation system identified both accuracy and graphical inadequacies, prompting the development of a second generation system. This too was based upon a tool-mounted fiducial marker system, and improved performance to near-millimetre probing accuracy. A resection system was incorporated into the system, and utilising the tracking information controlled resection was performed, producing sub-millimetre accuracies. Several complications resulted from the tool-mounted approach. Therefore, a third generation system was developed. This final generation deployed a stereoscopic visible-spectrum camera system affixed to a head-mounted display worn by the user. The system allowed the augmentation of the natural view of the user, providing convincing and immersive three dimensional augmented guidance, with probing and resection accuracies of 0.55±0.04 and 0.34±0.04 mm, respectively