Spherical Image Processing for Accurate Visual Odometry with Omnidirectional Cameras

International audienceDue to their omnidirectional view, the use of catadioptric cameras is of great interest for robot localization and visual servoing. For simplicity, most vision-based algorithms use image processing tools (e.g. image smoothing) that were designed for perspective cameras. This can be a good approximation when the camera displacement is small with respect to the distance to the observed environment. Otherwise, perspective image processing tools are unable to accurately handle the signal distortion that is induced by the specific geometry of omnidirectional cameras. In this paper, we propose an appropriate spherical image processing for increasing the accuracy of visual odometry estimation. The omnidirectional images are mapped onto a unit sphere and treated in the spherical spectral domain. The spherical image processing take into account the specific geometry of omnidirectional cameras. For example we can design, a more accurate and more repeatable Harris interest point detector. The interest points can be matched between two images with a large baseline in order to accurately estimate the camera motion. We demonstrate with a real experiment the accuracy of the visual odometry obtained using the spherical image processing and the improvement with respect to the use of a standard perspective image processing

Real-Time Multi-Fisheye Camera Self-Localization and Egomotion Estimation in Complex Indoor Environments

In this work a real-time capable multi-fisheye camera self-localization and egomotion estimation framework is developed. The thesis covers all aspects ranging from omnidirectional camera calibration to the development of a complete multi-fisheye camera SLAM system based on a generic multi-camera bundle adjustment method

Correspondenceless Structure from Motion

We present a novel approach for the estimation of 3D-motion directly from two images using the Radon transform. The feasibility of any camera motion is computed by integrating over all feature pairs that satisfy the epipolar constraint. This integration is equivalent to taking the inner product of a similarity function on feature pairs with a Dirac function embedding the epipolar constraint. The maxima in this five dimensional motion space will correspond to compatible rigid motions. The main novelty is in the realization that the Radon transform is a filtering operator: If we assume that the similarity and Dirac functions are defined on spheres and the epipolar constraint is a group action of rotations on spheres, then the Radon transform is a correlation integral. We propose a new algorithm to compute this integral from the spherical Fourier transform of the similarity and Dirac functions. Generating the similarity function now becomes a preprocessing step which reduces the complexity of the Radon computation by a factor equal to the number of feature pairs processed. The strength of the algorithm is in avoiding a commitment to correspondences, thus being robust to erroneous feature detection, outliers, and multiple motions

Global Shipping Container Monitoring Using Machine Learning with Multi-Sensor Hubs and Catadioptric Imaging

We describe a framework for global shipping container monitoring using machine learning with multi-sensor hubs and infrared catadioptric imaging. A wireless mesh radio satellite tag architecture provides connectivity anywhere in the world which is a significant improvement to legacy methods. We discuss the design and testing of a low-cost long-wave infrared catadioptric imaging device and multi-sensor hub combination as an intelligent edge computing system that, when equipped with physics-based machine learning algorithms, can interpret the scene inside a shipping container to make efficient use of expensive communications bandwidth. The histogram of oriented gradients and T-channel (HOG+) feature as introduced for human detection on low-resolution infrared catadioptric images is shown to be effective for various mirror shapes designed to give wide volume coverage with controlled distortion. Initial results for through-metal communication with ultrasonic guided waves show promise using the Dynamic Wavelet Fingerprint Technique (DWFT) to identify Lamb waves in a complicated ultrasonic signal

Gesture recognition with application in music arrangement

This thesis studies the interaction with music synthesis systems using hand gestures. Traditionally users of such systems were limited to input devices such as buttons, pedals, faders, and joysticks. The use of gestures allows the user to interact with the system in a more intuitive way. Without the constraint of input devices, the user can simultaneously control more elements within the music composition, thus increasing the level of the system's responsiveness to the musician's creative thoughts. A working system of this concept is implemented, employing computer vision and machine intelligence techniques to recognise the user's gestures.Dissertation (MSc)--University of Pretoria, 2006.Computer ScienceMScunrestricte

Airborne vision-based attitude estimation and localisation

Vision plays an integral part in a pilot's ability to navigate and control an aircraft. Therefore Visual Flight Rules have been developed around the pilot's ability to see the environment outside of the cockpit in order to control the attitude of the aircraft, to navigate and to avoid obstacles. The automation of these processes using a vision system could greatly increase the reliability and autonomy of unmanned aircraft and flight automation systems. This thesis investigates the development and implementation of a robust vision system which fuses inertial information with visual information in a probabilistic framework with the aim of aircraft navigation. The horizon appearance is a strong visual indicator of the attitude of the aircraft. This leads to the first research area of this thesis, visual horizon attitude determination. An image processing method was developed to provide high performance horizon detection and extraction from camera imagery. A number of horizon models were developed to link the detected horizon to the attitude of the aircraft with varying degrees of accuracy. The second area investigated in this thesis was visual localisation of the aircraft. A terrain-aided horizon model was developed to estimate the position, altitude as well as attitude of the aircraft. This gives rough positions estimates with highly accurate attitude information. The visual localisation accuracy was improved by incorporating ground feature-based map-aided navigation. Road intersections were detected using a developed image processing algorithm and then they were matched to a database to provide positional information. The developed vision system show comparable performance to other non-vision-based systems while removing the dependence on external systems for navigation. The vision system and techniques developed in this thesis helps to increase the autonomy of unmanned aircraft and flight automation systems for manned flight