Disparity refinement process based on RANSAC plane fitting for machine vision applications

This paper presents a new disparity map refinement process for stereo matching algorithm and the refinement stage that will be implemented by partitioning the place or mask image and re-projected to the preliminary disparity images. This process is to refine the noise and sparse of initial disparity map from weakly textured. The plane fitting algorithm is using Random Sample Consensus. Two well-known stereo matching algorithms have been tested on this framework with different filtering techniques applied at disparity refinement stage. The framework is evaluated on three Middlebury datasets. The experimental results show that the proposed framework produces better-quality and more accurate than normal flow state-of-the-art stereo matching algorithms. The performance evaluations are based on standard image quality metrics i.e. structural similarity index measure, peak signal-to-noise ratio and mean square error.Keywords: computer vision; disparity refinement; image segmentation; RANSAC; stereo.

Disparity Map Generation from Illumination Variant Stereo Images Using Efficient Hierarchical Dynamic Programming

A novel hierarchical stereo matching algorithm is presented which gives disparity map as output from illumination variant stereo pair. Illumination difference between two stereo images can lead to undesirable output. Stereo image pair often experience illumination variations due to many factors like real and practical situation, spatially and temporally separated camera positions, environmental illumination fluctuation, and the change in the strength or position of the light sources. Window matching and dynamic programming techniques are employed for disparity map estimation. Good quality disparity map is obtained with the optimized path. Homomorphic filtering is used as a preprocessing step to lessen illumination variation between the stereo images. Anisotropic diffusion is used to refine disparity map to give high quality disparity map as a final output. The robust performance of the proposed approach is suitable for real life circumstances where there will be always illumination variation between the images. The matching is carried out in a sequence of images representing the same scene, however in different resolutions. The hierarchical approach adopted decreases the computation time of the stereo matching problem. This algorithm can be helpful in applications like robot navigation, extraction of information from aerial surveys, 3D scene reconstruction, and military and security applications. Similarity measure SAD is often sensitive to illumination variation. It produces unacceptable disparity map results for illumination variant left and right images. Experimental results show that our proposed algorithm produces quality disparity maps for both wide range of illumination variant and invariant stereo image pair

Literature Survey On Stereo Vision Disparity Map Algorithms

This paper presents a literature survey on existing disparity map algorithms. It focuses on four main stages of processing as proposed by Scharstein and Szeliski in a taxonomy and evaluation of dense two-frame stereo correspondence algorithms performed in 2002. To assist future researchers in developing their own stereo matching algorithms, a summary of the existing algorithms developed for every stage of processing is also provided. The survey also notes the implementation of previous software-based and hardware-based algorithms. Generally, the main processing module for a software-based implementation uses only a central processing unit. By contrast, a hardware-based implementation requires one or more additional processors for its processing module, such as graphical processing unit or a field programmable gate array. This literature survey also presents a method of qualitative measurement that is widely used by researchers in the area of stereo vision disparity mappings

Técnicas de visión estereoscópica para determinar la estructura tridimensional de la escena

En este trabajo se realiza un estudio sobre la efectividad de una serie de métodos de correspondencia estereoscópica. La correspondencia estereoscópica constituye uno de los pasos esenciales dentro de la visión estereoscópica en los sistemas robotizados, de ahí su importancia. El objetivo se centra en el estudio de la viabilidad de los mismos de cara a su implementación en sistemas estereoscópicos que han de operar en entornos de exterior y bajo condiciones del entorno adversas. La motivación del trabajo proviene de la necesidad derivada de una serie de proyectos de investigación dentro de las actividades del grupo ISCAR. En este trabajo se han realizado diversas pruebas experimentales orientadas a la identificación de los métodos más prometedores en el ámbito de la correspondencia estereoscópica con la finalidad indicada. Se han estudiado varias técnicas existentes en la literatura y se han establecido las pautas a seguir en el futuro a tenor de los resultados obtenidos para su implementación en sistemas reales. [ABSTRACT] In this work we have studied several stereovision matching approaches with the aim of testing its effectiveness. The main step in robotized systems, equipped with stereovision, is the correspondence, here is its relevance. The goal of this work is focused on the study of the viability of such methods with the aim that they can be implemented in stereoscopic vision-based systems working in adverse outdoor environmental conditions. This work is motivated because the ISCAR group is currently working in several research projects where the stereovision is a crucial system. In this work several experimental tests have been carried out oriented toward the identification of the most promising correspondence methods with the above expressed goal. Several existing approaches in the literature have been studied and, as a result, some guidelines have been established based on the results reported, so that the research is oriented toward future implementations in real systems

Stereoscopic depth estimation for online vision systems

A lot of work has been done in the area of machine stereo vision, but a severe drawback of today's algorithms is that they either achieve high accuracy and robustness by sacrificing real-time speed or they are real-time capable but with major deficiencies in quality. In order to tackle this problem this thesis presents two new methods which exhibit a very good balance between computational effort and depth accuracy. First, the summed normalized cross-correlation is proposed which constitutes a new cost function for block-matching stereo processing. In contrast to most standard cost functions it hardly suffers from the fattening effect while being computationally very efficient. Second, the direct surface fitting, a new algorithm for fitting parametric surface models to stereo images, is introduced. This algorithm is inspired by the homography-constrained gradient descent methods but in contrast to these allows also for the estimation of non-planar surfaces. Experimental evaluations demonstrate that both newly introduced algorithms are competitive to state-of-the-art in terms of accuracy while having a much lower computational time.Die visuelle Wahrnehmung des Menschen wird in hohem Maße vom stereoskopischenSehen beeinflusst. Die dreidimensionale Wahrnehmung entsteht dabei durch dieleicht unterschiedlichen Blickwinkel der beiden Augen. Es ist eine nahe liegendeAnnahmen, dass maschinelle Sehsysteme ebenfalls von einem vergleichbaren Sinnprofitieren können. Obwohl es bereits zahlreiche Arbeiten auf dem Gebiet desmaschinellen stereoskopischen Sehen gibt, erfüllen die heutigen Algorithmenentweder nicht die Anforderungen für eine effiziente Berechnung oder aber siehaben nur eine geringe Genauigkeit und Robustheit. Das Ziel dieser Doktorarbeit ist die Entwicklung von echtzeit- undrealweltfähigen stereoskopischen Algorithmen. Insbesondere soll die Berechnungder Algorithmen leichtgewichtig genug sein, um auf mobilen Plattformeneingesetzt werden zu können. Dazu werden im Rahmen dieser Arbeit zwei neueMethoden vorgestellt, welche sich durch eine gute Balance zwischenGeschwindigkeit und Genauigkeit auszeichnen. Als erstes wird die "Summed Normalized Cross-Correlation" (SNCC) vorgestellt,eine neue Kostenfunktion für blockvergleichende, stereoskopischeTiefenschätzung. Im Unterschied zu den meisten anderen Kostenfunktionen ist SNCCnicht anfällig für den qualitätsmindernden "Fattening"-Effekt, kann abertrotzdem sehr effizient berechnet werden. Die Auswertung der Genauigkeit aufStandard Benchmark-Tests zeigt, dass mit SNCC die Genauigkeit von lokaler,blockvergleichsbasierter, stereoskopischer Berechnung nahe an die Genauigkeitvon global optimierenden Methoden basierend auf "Graph Cut" oder "BeliefPropagation" heran kommt. Die zweite vorgestellte Methode ist das "Direct Surface Fitting", ein neuerAlgorithmus zum Schätzen parametrischer Oberflächenmodelle an Hand vonStereobildern. Dieser Algorithmus ist inspiriert vom Homographie-beschränktenGradientenabstieg, welcher häufig dazu benutzt wird um die Lage von planarenOberflächen im Raum zu Schätzen. Durch die Ersetzung des Gradientenabstiegs mitder direkten Suchmethodik von Hooke und Jeeves wird die planare Schätzung aufbeliebige parametrische Oberflächenmodelle und beliebige Kostenfunktionenerweitert. Ein Vergleich auf Standard Benchmark-Tests zeigt, dass "DirectSurface Fitting" eine vergleichbare Genauigkeit wie Methoden aus dem Stand derTechnik hat, im Gegensatz zu diesen aber höhere Robustheit in anspruchsvollenSituationen besitzt. Um die Realwelttauglichkeit und Effizienz der vorgestellten Methoden zuuntermauern wurden diese in ein Automobil- und in ein Robotersystemintegriert. Die mit diesen mobilen Systemen durchgeführten Experimentedemonstrieren die hohe Robustheit und Stabilität der eingeführten Methoden

Numerical characterisation of contaminant transport and distribution in airliner cabins

The in-flight air quality of commercial airliners has drawn increasing attentions with the rapid annual-growth of air travellers. Contaminants inside airliner cabins could be released from multiple sources (coughing, sneezing, ozone production, etc.) and would suspend inside the cabin as particulate matters (PM). The focus of this project is to comprehensively and effectively assess the transport characteristics of particulate contaminants in airliner cabins and the corresponding infection risks of passengers. Computational Fluid Dynamics (CFD) has been proven as a cost-efficient approach to analyse and optimise air quality in airliner cabin environments. However, the holistic simulation of airliner cabin is still absent in existing literature due to the extreme complexity of the cabin environment induced by the multi-scale, multi-coupling and non-linear transport characteristics. Theoretically, existing studies mostly relied on the Lagrangian model to depict particle transport in occupied airliner cabins due to its mechanistic coupling of air-particle interactions. Studies were mostly limited in reduced cabin section due to the factor that the Lagrangian tracking model is very time consuming and costly induced by the individual-tracking strategy. Numerically, due to the extreme complexity of the cabin environment, computational thermal manikin (CTM) models were arbitrarily simplified, causing the deficiency of passenger body features, and the passengers’ thermal effect was also eliminated. This could provide misleading information, particularly in passengers’ breathing zones, when evaluating the infection risks associated with particulate exposure. This research, however, further evaluated these overlooked factors associated with in-depth investigations and optimisation of theoretical and numerical models. By integrating mechanistic multi-phase flow models, novel manikin simplification approach and 3D dynamic characterisation of contaminant transport, a systematic and cost-efficient platform was thereby developed for comprehensive assessment of air quality and particulate contaminant transport in airliner cabins. The main body of this thesis was composed of nine chapters. In the first two chapters, research background and comprehensive literature review were summarised in conjunction with the highlighted research gaps found in the existing literature, followed by the research methodology in Chapter 3. The main research contributions were demonstrated from chapters 4 to 8. Chapter 4 evaluated the importance of passengers’ thermal effect in densely occupied cabin environment, which were mostly overlooked in existing studies due to the complexity of cabin environment. In Chapter 5, three mathematical models (Lagrangian, drift-flux and newly proposed Eulerian-Eulerian (E-E) models) were tested and compared in terms of the reliability and efficiency. A particle source in cell (PSI-C) method based program was developed using Matlab to convert particle trajectories into concentrations. The computational thermal manikins (CTMs) were optimised and simplified using various approaches in Chapter 6. The degree/level of applied simplification approaches were found uncontrollable. As a solution, a novel and quantifiable simplification approach based on the mesh decimating algorithm was developed and tested under cabin environment in Chapter 7. Chapter 8 demonstrated a systematic assessment of contaminants transport and infection risks in a large scale cabin environment. All the major components tested in the previous chapters were integrated to achieve comprehensiveness. Unsteady flow behaviour at the aisle region of the cabin was noticed in this study and verified by the collaborator’s experimental measurement. The PSI-C based program was further optimised using Mathematica to provide smooth concentration distributions. A quantifiable approach to assess infection risks was also proposed in this chapter. All the aforementioned contributions are concluded and highlighted in Chapter 9, followed by a list of all published publications during the PhD candidature period. This research contributed to the following outcomes: (a) A novel and quantifiable manikin simplification algorithm was developed to reduce the computational costs without sacrificing accuracy. (b) Comprehensive descriptions of inter-phase mechanisms were achieved in a cost-efficient way using E-E model to realise fast predictions of the PM concentration in airliner cabins. (c) An unique technique to convert particle trajectories to concentration was developed and optimised based on the PSI-C method. (d) A quantifiable approach to assess the infection risks in the airliner cabins was proposed. (e) A systematic platform was developed to holistically assess the infection risks in airliner cabin environment. The outcomes of this research laid an important and solid foundation for air quality optimisation and health risks assessment in other densely occupied spaces (high-speed rail, metro, etc.)

Correspondencia estereoscópica en imágenes obtenidas con proyección omnidireccional para entornos forestales

Los sistemas de visión estereoscópica se han venido utilizando de forma manual desde hace varias décadas para captar información tridimensional del entorno en diferentes aplicaciones. Con el desarrollo experimentado en los últimos años por las técnicas de procesamiento computacional de imágenes, la visión estereoscópica se viene incorporando cada vez más a sistemas automáticos de diferente naturaleza. El problema central en la automatización de un sistema de visión estereoscópica es la determinación de la correspondencia entre píxeles del par de imágenes estereoscópicas que proceden del mismo punto de la escena tridimensional. El trabajo de investigación desarrollado en esta tesis consiste en el diseño de una estrategia global para dar solución al problema de la correspondencia estereoscópica para un tipo característico de imágenes omnidireccionales procedentes de entornos forestales. Las imágenes son obtenidas mediante un sistema óptico basado en las denominadas lentes de ojo de pez. Este trabajo tiene su origen en el interés suscitado por el Centro de Investigación Forestal (CIFOR) del Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria (INIA) para automatizar el proceso de extracción de información mediante el dispositivo de medición con número de patente MU-200501738. El interés se centra en obtener dicha información de los troncos de los árboles a partir de imágenes estereoscópicas. Con las medidas obtenidas, los técnicos realizan inventarios forestales que incluyen estudios sobre el volumen de madera, la densidad de árboles, la evolución o crecimiento de éstos, entre otros. La contribución principal de este trabajo consiste en la propuesta de una estrategia que combina los dos procesos esenciales en visión estereoscópica artificial como son la segmentación y correspondencia de ciertas estructuras existentes en las dos imágenes del par estereoscópico. La estrategia se diseña para dos tipos de imágenes procedentes de sendos entornos forestales. El primero de dichos entornos se refiere a pinares de pino silvestre (Pinus sylvestris L.) donde las imágenes se han obtenido en días soleados y por tanto con una alta variabilidad de los niveles de intensidad debido a las zonas iluminadas. En el segundo entorno las imágenes proceden de bosques de roble rebollo (Quercus pyrenaica Willd.) cuya característica más relevante es que se obtienen bajo unas condiciones de iluminación relativamente escasas, días nublados o al amanecer o atardecer, pero suficiente como para producir alto contraste entre los troncos y el cielo. .Debido a las características tan diferentes de ambos entornos, tanto en lo relativo a la iluminación como a la naturaleza de los propios árboles y las texturas que les rodean, los procesos de segmentación y correspondencia se diseñan atendiendo al tipo concreto de entorno forestal. Hecho éste, que marca la tendencia de la futura investigación cuando se analicen otros entornos forestales. En el caso de los bosques de pino, el proceso de segmentación se plantea desde el punto de vista del aislamiento de los troncos mediante la exclusión de las texturas que les rodean (hojas de los pinos, suelo, cielo). Por ello, se proponen técnicas específicas de identificación de texturas para las hojas y de clasificación para el resto. En este último caso se combinan dos técnicas de clasificación clásicas como son el método de Agrupamiento Borroso y el estimador paramétrico Bayesiano. El proceso de correspondencia se plantea en dos fases. En primer lugar se identifican los píxeles homólogos en sendas imágenes del par estereoscópico mediante la adaptación a este problema de las siguientes técnicas procedentes de la teoría general de la decisión: Integral Fuzzy de Choquet, Integral Fuzzy de Sugeno, Teoría Dempster-Shafer y Toma de Decisiones Multicriterio Fuzzy. En segundo lugar, los resultados relativos a la correspondencia obtenidos mediante esas técnicas se procesan para conseguir mejorarlos mediante la adaptación de sendos paradigmas: los Mapas Cognitivos Fuzzy y la Red Neuronal de Hopfield. Para el segundo entorno de bosques de roble, la segmentación se plantea como un proceso de identificación de los troncos de los árboles utilizando técnicas específicas de procesamiento de imágenes, en concreto técnicas de extracción y etiquetado de regiones. Para cada región se obtiene un conjunto de atributos o propiedades que la caracterizan, y el proceso de correspondencia establece las regiones homólogas de las dos imágenes del par estereoscópico mediante medidas de similitud entre los atributos de las regiones. La estrategia propuesta, basada en los procesos de segmentación y correspondencia, se compara favorablemente desde la perspectiva de la automatización del proceso y se plantea para su aplicación a cualquier tipo de entorno forestal, si bien con las pertinentes adaptaciones y modificaciones inherentes a los procesos de segmentación y correspondencia en función de la naturaleza del entorno forestal analizado. [ABSTRACT] Stereoscopic vision systems have been used manually for decades to capture three-dimensional information of the environment in different applications. With the growth experienced in recent years by the techniques of computer image processing, stereoscopic vision has been increasingly incorporating automated systems of different nature. The central problem in the automation of a stereoscopic vision system is the determination of the correspondence between pixels of the pair of stereoscopic images that come from the same point in three-dimensional scene. The research undertaken in this thesis comprises the design of a global strategy to solve the stereoscopic correspondence problem for a specific kind of omnidirectional image from forest environments. The images are obtained through an optical system based on the lens known as fisheye. This work stems from the interest generated by the Forest Research Centre (CIFOR) part of the National Institute for Agriculture and Food Research and Technology (INIA) to automate the process of extracting information through the measurement mechanism with patent number MU-200501738. The focus is on obtaining this information from tree trunks using stereoscopic images. The technicians carry out forest inventories which include studies on wood volume and tree density as well as the evolution and growth of the trees with the measurements obtained. This paper’s main contribution is the proposal for a strategy that combines the two essential processes involved in artificial stereo vision: segmentation and correspondence of certain structures in the dual images of the stereoscopic pair. The strategy is designed for two types of images from two forest environments. The first of these refers to Scots pine forests (Pinus sylvestris L.) where images were obtained on sunny days and therefore exhibit highly variable intensity levels due to the illuminated areas. In the second of these, the images come from Rebollo oak forests (Quercus pyrenaica Willd.), the main characteristic of which is that they are obtained under relatively low light conditions, on cloudy days or at dawn or dusk, but with sufficient light to produce high contrast between the trees and sky. Due to the very different characteristics of each environment - both in terms of light and the nature of trees themselves and textures that surround them - the segmentation and correspondence processes are designed specifically according to the specific type of forest environment. This sets the trend for future research when analyzing other forest environments. In the case of pine forests, the segmentation process is approached from the point of view of isolating the trunks by excluding the textures that surround them (pine needles, the ground, the sky). For this reason, we propose the use of the specific techniques of texture identification for the pine needles and of classification for the rest. The latter case combines two classic classification techniques: Fuzzy Clustering and the Bayesian Parametric estimator. The matching process is set out in two phases. The first identifies the homogeneous pixels in separate stereo pair images, by means of the adaptation of the following techniques from general decision theory to this problem: Choquet’s Fuzzy Integral, Sugeno’s Fuzzy Integral, Dempster-Shafer Theory and Fuzzy Multicriteria Decision Making. Second, the results relating to correspondence obtained by these techniques are enhanced through the adaptation of two separate paradigms, namely: Fuzzy Cognitive Maps and the Hopfield Neural Network. Regarding the second type of forest analyzed, i.e. oak, the segmentation process s designed in order to identify the tree trunks by applying specific techniques in image processing, relating to the extraction and labelling of regions, so that each region is given a set of attributes or properties that characterizes it. The matching process establishes the equivalent regions from the two stereo pair images using measures of similarity among the attributes of the regions. The proposed strategy based on segmentation and correspondence processes can be favourably compared from the perspective of the automation of the process and we suggest it can be applied to any type of forest environment, with the appropriate adaptations inherent to the segmentation and correspondence processes in accordance with the nature of the forest environment analyzed