Automating Active Stereo Vision Calibration Process with Cobots

Collaborative robots help the academia and industry to accelerate the work by introducing a new concept of cooperation between human and robot. In this paper, a calibration process for an active stereo vision rig has been automated to accelerate the task and improve the quality of the calibration. As illustrated in this paper by using Baxter Robot, the calibration process has been done faster by three times in comparison to the manual calibration that depends on the human. The quality of the calibration was improved by 120% when the Baxter robot was used

Active stereo platform: online epipolar geometry update

This paper presents a novel method to update a variable epipolar geometry platform directly from the motor encoder based on mapping the motor encoder angle to the image space angle, avoiding the use of feature detection algorithms. First, an offline calibration is performed to establish a relationship between the image space and the hardware space. Second, a transformation matrix is generated using the results from this mapping. The transformation matrix uses the updated epipolar geometry of the platform to rectify the images for further processing. The system has an overall error in the projection of ± 5 pixels, which drops to ± 1.24 pixels when the verge angle increases beyond 10°. The platform used in this project has 3° of freedom to control the verge angle and the size of the baseline

The Role of Fixation and Visual Attention in Object Recognition

This research project is a study of the role of fixation and visual attention in object recognition. In this project, we build an active vision system which can recognize a target object in a cluttered scene efficiently and reliably. Our system integrates visual cues like color and stereo to perform figure/ground separation, yielding candidate regions on which to focus attention. Within each image region, we use stereo to extract features that lie within a narrow disparity range about the fixation position. These selected features are then used as input to an alignment-style recognition system. We show that visual attention and fixation significantly reduce the complexity and the false identifications in model-based recognition using Alignment methods. We also demonstrate that stereo can be used effectively as a figure/ground separator without the need for accurate camera calibration

Biometric fusion methods for adaptive face recognition in computer vision

PhD ThesisFace recognition is a biometric method that uses different techniques to identify the individuals based on the facial information received from digital image data. The system of face recognition is widely used for security purposes, which has challenging problems. The solutions to some of the most important challenges are proposed in this study. The aim of this thesis is to investigate face recognition across pose problem based on the image parameters of camera calibration. In this thesis, three novel methods have been derived to address the challenges of face recognition and offer solutions to infer the camera parameters from images using a geomtric approach based on perspective projection. The following techniques were used: camera calibration CMT and Face Quadtree Decomposition (FQD), in order to develop the face camera measurement technique (FCMT) for human facial recognition. Facial information from a feature extraction and identity-matching algorithm has been created. The success and efficacy of the proposed algorithm are analysed in terms of robustness to noise, the accuracy of distance measurement, and face recognition. To overcome the intrinsic and extrinsic parameters of camera calibration parameters, a novel technique has been developed based on perspective projection, which uses different geometrical shapes to calibrate the camera. The parameters used in novel measurement technique CMT that enables the system to infer the real distance for regular and irregular objects from the 2-D images. The proposed system of CMT feeds into FQD to measure the distance between the facial points. Quadtree decomposition enhances the representation of edges and other singularities along curves of the face, and thus improves directional features from face detection across face pose. The proposed FCMT system is the new combination of CMT and FQD to recognise the faces in the various pose. The theoretical foundation of the proposed solutions has been thoroughly developed and discussed in detail. The results show that the proposed algorithms outperform existing algorithms in face recognition, with a 2.5% improvement in main error recognition rate compared with recent studies

Técnicas de visión estereoscópica para determinar la estructura tridimensional de la escena

En este trabajo se realiza un estudio sobre la efectividad de una serie de métodos de correspondencia estereoscópica. La correspondencia estereoscópica constituye uno de los pasos esenciales dentro de la visión estereoscópica en los sistemas robotizados, de ahí su importancia. El objetivo se centra en el estudio de la viabilidad de los mismos de cara a su implementación en sistemas estereoscópicos que han de operar en entornos de exterior y bajo condiciones del entorno adversas. La motivación del trabajo proviene de la necesidad derivada de una serie de proyectos de investigación dentro de las actividades del grupo ISCAR. En este trabajo se han realizado diversas pruebas experimentales orientadas a la identificación de los métodos más prometedores en el ámbito de la correspondencia estereoscópica con la finalidad indicada. Se han estudiado varias técnicas existentes en la literatura y se han establecido las pautas a seguir en el futuro a tenor de los resultados obtenidos para su implementación en sistemas reales. [ABSTRACT] In this work we have studied several stereovision matching approaches with the aim of testing its effectiveness. The main step in robotized systems, equipped with stereovision, is the correspondence, here is its relevance. The goal of this work is focused on the study of the viability of such methods with the aim that they can be implemented in stereoscopic vision-based systems working in adverse outdoor environmental conditions. This work is motivated because the ISCAR group is currently working in several research projects where the stereovision is a crucial system. In this work several experimental tests have been carried out oriented toward the identification of the most promising correspondence methods with the above expressed goal. Several existing approaches in the literature have been studied and, as a result, some guidelines have been established based on the results reported, so that the research is oriented toward future implementations in real systems

Depth recovery and parameter analysis using single-lens prism based stereovision system

Ph.DDOCTOR OF PHILOSOPH

Correspondencia estereoscópica en imágenes obtenidas con proyección omnidireccional para entornos forestales

Los sistemas de visión estereoscópica se han venido utilizando de forma manual desde hace varias décadas para captar información tridimensional del entorno en diferentes aplicaciones. Con el desarrollo experimentado en los últimos años por las técnicas de procesamiento computacional de imágenes, la visión estereoscópica se viene incorporando cada vez más a sistemas automáticos de diferente naturaleza. El problema central en la automatización de un sistema de visión estereoscópica es la determinación de la correspondencia entre píxeles del par de imágenes estereoscópicas que proceden del mismo punto de la escena tridimensional. El trabajo de investigación desarrollado en esta tesis consiste en el diseño de una estrategia global para dar solución al problema de la correspondencia estereoscópica para un tipo característico de imágenes omnidireccionales procedentes de entornos forestales. Las imágenes son obtenidas mediante un sistema óptico basado en las denominadas lentes de ojo de pez. Este trabajo tiene su origen en el interés suscitado por el Centro de Investigación Forestal (CIFOR) del Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria (INIA) para automatizar el proceso de extracción de información mediante el dispositivo de medición con número de patente MU-200501738. El interés se centra en obtener dicha información de los troncos de los árboles a partir de imágenes estereoscópicas. Con las medidas obtenidas, los técnicos realizan inventarios forestales que incluyen estudios sobre el volumen de madera, la densidad de árboles, la evolución o crecimiento de éstos, entre otros. La contribución principal de este trabajo consiste en la propuesta de una estrategia que combina los dos procesos esenciales en visión estereoscópica artificial como son la segmentación y correspondencia de ciertas estructuras existentes en las dos imágenes del par estereoscópico. La estrategia se diseña para dos tipos de imágenes procedentes de sendos entornos forestales. El primero de dichos entornos se refiere a pinares de pino silvestre (Pinus sylvestris L.) donde las imágenes se han obtenido en días soleados y por tanto con una alta variabilidad de los niveles de intensidad debido a las zonas iluminadas. En el segundo entorno las imágenes proceden de bosques de roble rebollo (Quercus pyrenaica Willd.) cuya característica más relevante es que se obtienen bajo unas condiciones de iluminación relativamente escasas, días nublados o al amanecer o atardecer, pero suficiente como para producir alto contraste entre los troncos y el cielo. .Debido a las características tan diferentes de ambos entornos, tanto en lo relativo a la iluminación como a la naturaleza de los propios árboles y las texturas que les rodean, los procesos de segmentación y correspondencia se diseñan atendiendo al tipo concreto de entorno forestal. Hecho éste, que marca la tendencia de la futura investigación cuando se analicen otros entornos forestales. En el caso de los bosques de pino, el proceso de segmentación se plantea desde el punto de vista del aislamiento de los troncos mediante la exclusión de las texturas que les rodean (hojas de los pinos, suelo, cielo). Por ello, se proponen técnicas específicas de identificación de texturas para las hojas y de clasificación para el resto. En este último caso se combinan dos técnicas de clasificación clásicas como son el método de Agrupamiento Borroso y el estimador paramétrico Bayesiano. El proceso de correspondencia se plantea en dos fases. En primer lugar se identifican los píxeles homólogos en sendas imágenes del par estereoscópico mediante la adaptación a este problema de las siguientes técnicas procedentes de la teoría general de la decisión: Integral Fuzzy de Choquet, Integral Fuzzy de Sugeno, Teoría Dempster-Shafer y Toma de Decisiones Multicriterio Fuzzy. En segundo lugar, los resultados relativos a la correspondencia obtenidos mediante esas técnicas se procesan para conseguir mejorarlos mediante la adaptación de sendos paradigmas: los Mapas Cognitivos Fuzzy y la Red Neuronal de Hopfield. Para el segundo entorno de bosques de roble, la segmentación se plantea como un proceso de identificación de los troncos de los árboles utilizando técnicas específicas de procesamiento de imágenes, en concreto técnicas de extracción y etiquetado de regiones. Para cada región se obtiene un conjunto de atributos o propiedades que la caracterizan, y el proceso de correspondencia establece las regiones homólogas de las dos imágenes del par estereoscópico mediante medidas de similitud entre los atributos de las regiones. La estrategia propuesta, basada en los procesos de segmentación y correspondencia, se compara favorablemente desde la perspectiva de la automatización del proceso y se plantea para su aplicación a cualquier tipo de entorno forestal, si bien con las pertinentes adaptaciones y modificaciones inherentes a los procesos de segmentación y correspondencia en función de la naturaleza del entorno forestal analizado. [ABSTRACT] Stereoscopic vision systems have been used manually for decades to capture three-dimensional information of the environment in different applications. With the growth experienced in recent years by the techniques of computer image processing, stereoscopic vision has been increasingly incorporating automated systems of different nature. The central problem in the automation of a stereoscopic vision system is the determination of the correspondence between pixels of the pair of stereoscopic images that come from the same point in three-dimensional scene. The research undertaken in this thesis comprises the design of a global strategy to solve the stereoscopic correspondence problem for a specific kind of omnidirectional image from forest environments. The images are obtained through an optical system based on the lens known as fisheye. This work stems from the interest generated by the Forest Research Centre (CIFOR) part of the National Institute for Agriculture and Food Research and Technology (INIA) to automate the process of extracting information through the measurement mechanism with patent number MU-200501738. The focus is on obtaining this information from tree trunks using stereoscopic images. The technicians carry out forest inventories which include studies on wood volume and tree density as well as the evolution and growth of the trees with the measurements obtained. This paper’s main contribution is the proposal for a strategy that combines the two essential processes involved in artificial stereo vision: segmentation and correspondence of certain structures in the dual images of the stereoscopic pair. The strategy is designed for two types of images from two forest environments. The first of these refers to Scots pine forests (Pinus sylvestris L.) where images were obtained on sunny days and therefore exhibit highly variable intensity levels due to the illuminated areas. In the second of these, the images come from Rebollo oak forests (Quercus pyrenaica Willd.), the main characteristic of which is that they are obtained under relatively low light conditions, on cloudy days or at dawn or dusk, but with sufficient light to produce high contrast between the trees and sky. Due to the very different characteristics of each environment - both in terms of light and the nature of trees themselves and textures that surround them - the segmentation and correspondence processes are designed specifically according to the specific type of forest environment. This sets the trend for future research when analyzing other forest environments. In the case of pine forests, the segmentation process is approached from the point of view of isolating the trunks by excluding the textures that surround them (pine needles, the ground, the sky). For this reason, we propose the use of the specific techniques of texture identification for the pine needles and of classification for the rest. The latter case combines two classic classification techniques: Fuzzy Clustering and the Bayesian Parametric estimator. The matching process is set out in two phases. The first identifies the homogeneous pixels in separate stereo pair images, by means of the adaptation of the following techniques from general decision theory to this problem: Choquet’s Fuzzy Integral, Sugeno’s Fuzzy Integral, Dempster-Shafer Theory and Fuzzy Multicriteria Decision Making. Second, the results relating to correspondence obtained by these techniques are enhanced through the adaptation of two separate paradigms, namely: Fuzzy Cognitive Maps and the Hopfield Neural Network. Regarding the second type of forest analyzed, i.e. oak, the segmentation process s designed in order to identify the tree trunks by applying specific techniques in image processing, relating to the extraction and labelling of regions, so that each region is given a set of attributes or properties that characterizes it. The matching process establishes the equivalent regions from the two stereo pair images using measures of similarity among the attributes of the regions. The proposed strategy based on segmentation and correspondence processes can be favourably compared from the perspective of the automation of the process and we suggest it can be applied to any type of forest environment, with the appropriate adaptations inherent to the segmentation and correspondence processes in accordance with the nature of the forest environment analyzed

pp. 153-159. Stereo Ranging with Verging Cameras

[2] I. Overington and P. Greenway, “Practical first-difference edge detection with subpixel accuracy, ” Image Ksion Comput., vol. 5, no. 3