Estimating heading direction from monocular video sequences using biologically-based sensor

The determination of one’s movement through the environment (visual odometry or self-motion estimation) from monocular sources such as video is an important research problem because of its relevance to robotics and autonomous vehicles. The traditional computer vision approach to this problem tracks visual features across frames in order to obtain 2-D image motion estimates from which the camera motion can be derived. We present an alternative scheme which uses the properties of motion sensitive cells in the primate brain to derive the image motion and the camera heading vector. We tested heading estimation using a camera mounted on a linear translation table with the line of sight of the camera set at a range of angles relative to straight ahead (0◦ to 50◦ in 10◦ steps). The camera velocity was also varied (0.2, 0.4, 0.8, 1.2, 1.6 and 2.0 m/s). Our biologically-based method produced accurate heading estimates over a wide range of test angles and camera speeds. Our approach has the advantage of being a one-shot estimator and not requiring iterative search techniques for finding the heading

Small Fixed-wing Aerial Positioning Using Inter-vehicle Ranging Combined with Visual Odometry

There has been increasing interest in developing the ability for small unmanned aerial systems (SUAS) to be able to operate in environments where GPS is not available. This research considers the case of a larger aircraft loitering above a smaller GPS-denied SUAS. This larger aircraft is assumed to have greater resources which can overcome the GPS jamming and provide range information to the SUAS flying a mission below. This research demonstrates that using a ranging update combined with an aircraft motion model and visual odometry can greatly improve the accuracy of a SUASs estimated position in a GPS-denied environment

Single camera vision-only SLAM on a suburban road network

Simultaneous Localization And Mapping (SLAM) is one of the major challenges in mobile robotics. Probabilistic techniques using high-end range finding devices are well established in the field, but recent work has investigated vision only approaches. This paper presents a method for generating approximate rotational and translation velocity information from a single vehicle-mounted consumer camera, without the computationally expensive process of tracking landmarks. The method is tested by employing it to provide the odometric and visual information for the RatSLAM system while mapping a complex suburban road network. RatSLAM generates a coherent map of the environment during an 18 km long trip through suburban traffic at speeds of up to 60 km/hr. This result demonstrates the potential of ground based vision-only SLAM using low cost sensing and computational hardware

Vision Augmented State Estimation with Fault Tolerance

Obtaining accurate and reliable measurement data is really crucial for any vehicle system, especially if the system deals with maintaining safe operations of the vehicle. Conventional direct methods of obtaining such measurements by using sensors like GPS, INS, and Wheel Encoders are not reliable. The raw measurement obtained from these sensors is accompanied with bias and noise. Also, these sensors tend to underperform in certain conditions and faults, which adds to the inaccuracy and uncertainty of a system. Many different approaches exist, which intelligently fuse information from multiple sensors in order to produce reliable estimates. However, when used independently, these approaches suffer due to faults like plausibility faults, frequency faults, and error code faults, which can occur at the sensor level. This thesis therefore takes into account the development and implementation of an architecture which merges a planned fault tolerance approach to an intelligent state estimation process. The main contribution of the developed prototype is that it demonstrates the effectiveness of the developed architecture in mitigation of various faults that occur at the sensor level. This prototype lays the ground for the use of VO as an alternate source of measurement to the conventional sensors. Starting from the implementation of sensor data specific integrity checking to the implementation of Extended Kalman Filter, fault tolerance methods have been employed at different levels. A 3 DoF dynamic vehicle model has been used to enable the Extended Kalman Filter predictions. Finally, test cases were devised to validate the effectiveness of the prototype

Topological place recognition for life-long visual localization

Premio Extraordinario de Doctorado de la UAH en el año académico 2016-2017La navegación de vehículos inteligentes o robots móviles en períodos largos de tiempo ha experimentado un gran interés por parte de la comunidad investigadora en los últimos años. Los sistemas basados en cámaras se han extendido ampliamente en el pasado reciente gracias a las mejoras en sus características, precio y reducción de tamaño, añadidos a los progresos en técnicas de visión artificial. Por ello, la localización basada en visión es una aspecto clave para desarrollar una navegación autónoma robusta en situaciones a largo plazo. Teniendo en cuenta esto, la identificación de localizaciones por medio de técnicas de reconocimiento de lugar topológicas puede ser complementaria a otros enfoques como son las soluciones basadas en el Global Positioning System (GPS), o incluso suplementaria cuando la señal GPS no está disponible.El estado del arte en reconocimiento de lugar topológico ha mostrado un funcionamiento satisfactorio en el corto plazo. Sin embargo, la localización visual a largo plazo es problemática debido a los grandes cambios de apariencia que un lugar sufre como consecuencia de elementos dinámicos, la iluminación o la climatología, entre otros. El objetivo de esta tesis es enfrentarse a las dificultades de llevar a cabo una localización topológica eficiente y robusta a lo largo del tiempo. En consecuencia, se van a contribuir dos nuevos enfoques basados en reconocimiento visual de lugar para resolver los diferentes problemas asociados a una localización visual a largo plazo. Por un lado, un método de reconocimiento de lugar visual basado en descriptores binarios es propuesto. La innovación de este enfoque reside en la descripción global de secuencias de imágenes como códigos binarios, que son extraídos mediante un descriptor basado en la técnica denominada Local Difference Binary (LDB). Los descriptores son eficientemente asociados usando la distancia de Hamming y un método de búsqueda conocido como Approximate Nearest Neighbors (ANN). Además, una técnica de iluminación invariante es aplicada para mejorar el funcionamiento en condiciones luminosas cambiantes. El empleo de la descripción binaria previamente introducida proporciona una reducción de los costes computacionales y de memoria.Por otro lado, también se presenta un método de reconocimiento de lugar visual basado en deep learning, en el cual los descriptores aplicados son procesados por una Convolutional Neural Network (CNN). Este es un concepto recientemente popularizado en visión artificial que ha obtenido resultados impresionantes en problemas de clasificación de imagen. La novedad de nuestro enfoque reside en la fusión de la información de imagen de múltiples capas convolucionales a varios niveles y granularidades. Además, los datos redundantes de los descriptores basados en CNNs son comprimidos en un número reducido de bits para una localización más eficiente. El descriptor final es condensado aplicando técnicas de compresión y binarización para realizar una asociación usando de nuevo la distancia de Hamming. En términos generales, los métodos centrados en CNNs mejoran la precisión generando representaciones visuales de las localizaciones más detalladas, pero son más costosos en términos de computación.Ambos enfoques de reconocimiento de lugar visual son extensamente evaluados sobre varios datasets públicos. Estas pruebas arrojan una precisión satisfactoria en situaciones a largo plazo, como es corroborado por los resultados mostrados, que comparan nuestros métodos contra los principales algoritmos del estado del arte, mostrando mejores resultados para todos los casos.Además, también se ha analizado la aplicabilidad de nuestro reconocimiento de lugar topológico en diferentes problemas de localización. Estas aplicaciones incluyen la detección de cierres de lazo basada en los lugares reconocidos o la corrección de la deriva acumulada en odometría visual usando la información proporcionada por los cierres de lazo. Asimismo, también se consideran las aplicaciones de la detección de cambios geométricos a lo largo de las estaciones del año, que son esenciales para las actualizaciones de los mapas en sistemas de conducción autónomos centrados en una operación a largo plazo. Todas estas contribuciones son discutidas al final de la tesis, incluyendo varias conclusiones sobre el trabajo presentado y líneas de investigación futuras

Sistema de odometría visual para localización monocular

En este documento se pretende resolver el problema de posicionamiento del robot móvil a través de la técnica de odometría visual monocular, la cual, como su propio nombre indica, utiliza una única cámara como sensor. Las cámaras son sensores de coste relativamente bajo que además tienen la ventaja de aportar gran cantidad de información sobre el entorno que rodea al robot móvil. En este proyecto se diseña un algoritmo de odometría visual monocular, pensado para tiempo real y con frecuencia de captura de imágenes relativamente alta (30 fps). En el estado del arte se describen las diferentes técnicas existentes para abordar la odometría visual monocular. Discutiremos el modelo matemático utilizado para proyectar puntos tridimensionales del mundo real en el plano bidimensional de la imagen, los diversos medios para extraer información relevante de una imagen y la forma de emplear dicha información para obtener la trayectoria recorrida por el robot móvil. En el tercer apartado se hará hincapié en el método propuesto para resolver el problema de odometría visual monocular. Tras esto, se propone un cuarto apartado en el que se analizan los resultados arrojados por el algoritmo propuesto. Finalmente se proponen una serie de mejoras de cara a ser implantadas en un futuro, ya que este trabajo está basado en un área de estudio que actualmente se encuentra en investigación.Universidad de Sevilla. Grado en Ingeniería de las Tecnologías Industriale

Non-Parametric Learning for Monocular Visual Odometry

This thesis addresses the problem of incremental localization from visual information, a scenario commonly known as visual odometry. Current visual odometry algorithms are heavily dependent on camera calibration, using a pre-established geometric model to provide the transformation between input (optical flow estimates) and output (vehicle motion estimates) information. A novel approach to visual odometry is proposed in this thesis where the need for camera calibration, or even for a geometric model, is circumvented by the use of machine learning principles and techniques. A non-parametric Bayesian regression technique, the Gaussian Process (GP), is used to elect the most probable transformation function hypothesis from input to output, based on training data collected prior and during navigation. Other than eliminating the need for a geometric model and traditional camera calibration, this approach also allows for scale recovery even in a monocular configuration, and provides a natural treatment of uncertainties due to the probabilistic nature of GPs. Several extensions to the traditional GP framework are introduced and discussed in depth, and they constitute the core of the contributions of this thesis to the machine learning and robotics community. The proposed framework is tested in a wide variety of scenarios, ranging from urban and off-road ground vehicles to unconstrained 3D unmanned aircrafts. The results show a significant improvement over traditional visual odometry algorithms, and also surpass results obtained using other sensors, such as laser scanners and IMUs. The incorporation of these results to a SLAM scenario, using a Exact Sparse Information Filter (ESIF), is shown to decrease global uncertainty by exploiting revisited areas of the environment. Finally, a technique for the automatic segmentation of dynamic objects is presented, as a way to increase the robustness of image information and further improve visual odometry results

Non-Parametric Learning for Monocular Visual Odometry

This thesis addresses the problem of incremental localization from visual information, a scenario commonly known as visual odometry. Current visual odometry algorithms are heavily dependent on camera calibration, using a pre-established geometric model to provide the transformation between input (optical flow estimates) and output (vehicle motion estimates) information. A novel approach to visual odometry is proposed in this thesis where the need for camera calibration, or even for a geometric model, is circumvented by the use of machine learning principles and techniques. A non-parametric Bayesian regression technique, the Gaussian Process (GP), is used to elect the most probable transformation function hypothesis from input to output, based on training data collected prior and during navigation. Other than eliminating the need for a geometric model and traditional camera calibration, this approach also allows for scale recovery even in a monocular configuration, and provides a natural treatment of uncertainties due to the probabilistic nature of GPs. Several extensions to the traditional GP framework are introduced and discussed in depth, and they constitute the core of the contributions of this thesis to the machine learning and robotics community. The proposed framework is tested in a wide variety of scenarios, ranging from urban and off-road ground vehicles to unconstrained 3D unmanned aircrafts. The results show a significant improvement over traditional visual odometry algorithms, and also surpass results obtained using other sensors, such as laser scanners and IMUs. The incorporation of these results to a SLAM scenario, using a Exact Sparse Information Filter (ESIF), is shown to decrease global uncertainty by exploiting revisited areas of the environment. Finally, a technique for the automatic segmentation of dynamic objects is presented, as a way to increase the robustness of image information and further improve visual odometry results

The suitability of the dendritic cell algorithm for robotic security applications

The implementation and running of physical security systems is costly and potentially hazardous for those employed to patrol areas of interest. From a technial perspective, the physical security problem can be seen as minimising the probability that intruders and other anomalous events will occur unobserved. A robotic solution is proposed using an artificial immune system, traditionally applied to software security, to identify threats and hazards: the dendritic cell algorithm. It is demonstrated that the migration from the software world to the hardware world is achievable for this algorithm and key properties of the resulting system are explored empirically and theoretically. It is found that the algorithm has a hitherto unknown frequency-dependent component, making it ideal for filtering out sensor noise. Weaknesses of the algorithm are also discovered, by mathematically phrasing the signal processing phase as a collection of linear classifiers. It is concluded that traditional machine learning approaches are likely to outperform the implemented system in its current form. However, it is also observed that the algorithm’s inherent filtering characteristics make modification, rather than rejection, the most beneficial course of action. Hybridising the dendritic cell algorithm with more traditional machine learning techniques, through the introduction of a training phase and using a non-linear classification phase is suggested as a possible future direction

The suitability of the dendritic cell algorithm for robotic security applications

The implementation and running of physical security systems is costly and potentially hazardous for those employed to patrol areas of interest. From a technial perspective, the physical security problem can be seen as minimising the probability that intruders and other anomalous events will occur unobserved. A robotic solution is proposed using an artificial immune system, traditionally applied to software security, to identify threats and hazards: the dendritic cell algorithm. It is demonstrated that the migration from the software world to the hardware world is achievable for this algorithm and key properties of the resulting system are explored empirically and theoretically. It is found that the algorithm has a hitherto unknown frequency-dependent component, making it ideal for filtering out sensor noise. Weaknesses of the algorithm are also discovered, by mathematically phrasing the signal processing phase as a collection of linear classifiers. It is concluded that traditional machine learning approaches are likely to outperform the implemented system in its current form. However, it is also observed that the algorithm’s inherent filtering characteristics make modification, rather than rejection, the most beneficial course of action. Hybridising the dendritic cell algorithm with more traditional machine learning techniques, through the introduction of a training phase and using a non-linear classification phase is suggested as a possible future direction