Homography-Based Tracking Control for Mobile Robots

This work presents a control strategy that allows a follower robot to track a target vehicle moving along an unknown trajectory with unknown velocity. It uses only artificial vision to establish both the robot’s position and orientation relative to the target. The control system is proved to be asymptotically stable at the equilibrium point, which corresponds to the navigation objective. Experimental results with two robots, a leader and a follower, are included to show the performance of the proposed vision-based tracking control system

An edge detection framework conjoining with IMU data for assisting indoor navigation of visually impaired persons

Smartphone applications based on object detection techniques have recently been proposed to assist visually impaired persons with navigating indoor environments. In the smartphone, digital cameras are installed to detect objects which are important for navigation. Prior to detect the interested objects from images, edges on the objects have to be identified. Object edges are difficult to be detected accurately as the image is contaminated by strong image blur which is caused by camera movement. Although deblurring algorithms can be used to filter blur noise, they are computationally expensive and not suitable for real-time implementation. Also edge detection algorithms are mostly developed for stationary images without serious blur. In this paper, a modified sigmoid function (MSF) framework based on inertial measurement unit (IMU) is proposed to mitigate these problems. The IMU estimates blur levels to adapt the MSF which is computationally simple. When the camera is moving, the topological structure of the MSF is estimated continuously in order to improve effectiveness of edge detections. The performance of the MSF framework is evaluated by detecting object edges on video sequences associated with IMU data. The MSF framework is benchmarked against existing edge detection techniques and results show that it can obtain comparably lower errors. It is further shown that the computation time is significantly decreased compared to using techniques that deploy deblurring algorithms, thus making our proposed technique a strong candidate for reliable real-time navigation

An Image Understanding System for Detecting Indoor Features

The capability of identifying physical structures of an unknown environment is very important for vision based robot navigation and scene understanding. Among physical structures in indoor environments, corridor lines and doors are important visual landmarks for robot navigation since they show the topological structure in an indoor environment and establish connections among the different places or regions in the indoor environment. Furthermore, they provide clues for understanding the image. In this thesis, I present two algorithms to detect the vanishing point, corridor lines, and doors respectively using a single digital video camera. In both algorithms, we utilize a hypothesis generation and verification method to detect corridor and door structures using low level linear features. The proposed method consists of low, intermediate, and high level processing stages which correspond to the extraction of low level features, the formation of hypotheses, and verification of the hypotheses via seeking evidence actively. In particular, we extend this single-pass framework by employing a feedback strategy for more robust hypothesis generation and verification. We demonstrate the robustness of the proposed methods on a large number of real video images in a variety of corridor environments, with image acquisitions under different illumination and reflection conditions, with different moving speeds, and with different viewpoints of the camera. Experimental results performed on the corridor line detection algorithm validate that the method can detect corridor line locations in the presence of many spurious line features about one second. Experimental results carried on the door detection algorithm show that the system can detect visually important doors in an image with a very high accuracy rate when a robot navigates along a corridor environment

Low-Resolution Vision for Autonomous Mobile Robots

The goal of this research is to develop algorithms using low-resolution images to perceive and understand a typical indoor environment and thereby enable a mobile robot to autonomously navigate such an environment. We present techniques for three problems: autonomous exploration, corridor classification, and minimalistic geometric representation of an indoor environment for navigation. First, we present a technique for mobile robot exploration in unknown indoor environments using only a single forward-facing camera. Rather than processing all the data, the method intermittently examines only small 32X24 downsampled grayscale images. We show that for the task of indoor exploration the visual information is highly redundant, allowing successful navigation even using only a small fraction (0.02%) of the available data. The method keeps the robot centered in the corridor by estimating two state parameters: the orientation within the corridor and the distance to the end of the corridor. The orientation is determined by combining the results of five complementary measures, while the estimated distance to the end combines the results of three complementary measures. These measures, which are predominantly information-theoretic, are analyzed independently, and the combined system is tested in several unknown corridor buildings exhibiting a wide variety of appearances, showing the sufficiency of low-resolution visual information for mobile robot exploration. Because the algorithm discards such a large percentage (99.98%) of the information both spatially and temporally, processing occurs at an average of 1000 frames per second, or equivalently takes a small fraction of the CPU. Second, we present an algorithm using image entropy to detect and classify corridor junctions from low resolution images. Because entropy can be used to perceive depth, it can be used to detect an open corridor in a set of images recorded by turning a robot at a junction by 360 degrees. Our algorithm involves detecting peaks from continuously measured entropy values and determining the angular distance between the detected peaks to determine the type of junction that was recorded (either middle, L-junction, T-junction, dead-end, or cross junction). We show that the same algorithm can be used to detect open corridors from both monocular as well as omnidirectional images. Third, we propose a minimalistic corridor representation consisting of the orientation line (center) and the wall-floor boundaries (lateral limit). The representation is extracted from low-resolution images using a novel combination of information theoretic measures and gradient cues. Our study investigates the impact of image resolution upon the accuracy of extracting such a geometry, showing that centerline and wall-floor boundaries can be estimated with reasonable accuracy even in texture-poor environments with low-resolution images. In a database of 7 unique corridor sequences for orientation measurements, less than 2% additional error was observed as the resolution of the image decreased by 99.9%

Generalised correlation higher order neural networks, neural network operation and Levenberg-Marquardt training on field programmable gate arrays

Higher Order Neural Networks (HONNs) were introduced in the late 80's as a solution to the increasing complexity within Neural Networks (NNs). Similar to NNs HONNs excel at performing pattern recognition, classification, optimisation particularly for non-linear systems in varied applications such as communication channel equalisation, real time intelligent control, and intrusion detection. This research introduced new HONNs called the Generalised Correlation Higher Order Neural Networks which as an extension to the ordinary first order NNs and HONNs, based on interlinked arrays of correlators with known relationships, they provide the NN with a more extensive view by introducing interactions between the data as an input to the NN model. All studies included two data sets to generalise the applicability of the findings. The research investigated the performance of HONNs in the estimation of short term returns of two financial data sets, the FTSE 100 and NASDAQ. The new models were compared against several financial models and ordinary NNs. Two new HONNs, the Correlation HONN (C-HONN) and the Horizontal HONN (Horiz-HONN) outperformed all other models tested in terms of the Akaike Information Criterion (AIC). The new work also investigated HONNs for camera calibration and image mapping. HONNs were compared against NNs and standard analytical methods in terms of mapping performance for three cases; 3D-to-2D mapping, a hybrid model combining HONNs with an analytical model, and 2D-to-3D inverse mapping. This study considered 2 types of data, planar data and co-planar (cube) data. To our knowledge this is the first study comparing HONNs against NNs and analytical models for camera calibration. HONNs were able to transform the reference grid onto the correct camera coordinate and vice versa, an aspect that the standard analytical model fails to perform with the type of data used. HONN 3D-to-2D mapping had calibration error lower than the parametric model by up to 24% for plane data and 43% for cube data. The hybrid model also had lower calibration error than the parametric model by 12% for plane data and 34% for cube data. However, the hybrid model did not outperform the fully non-parametric models. Using HONNs for inverse mapping from 2D-to-3D outperformed NNs by up to 47% in the case of cube data mapping. This thesis is also concerned with the operation and training of NNs in limited precision specifically on Field Programmable Gate Arrays (FPGAs). Our findings demonstrate the feasibility of on-line, real-time, low-latency training on limited precision electronic hardware such as Digital Signal Processors (DSPs) and FPGAs. This thesis also investigated the e�ffects of limited precision on the Back Propagation (BP) and Levenberg-Marquardt (LM) optimisation algorithms. Two new HONNs are compared against NNs for estimating the discrete XOR function and an optical waveguide sidewall roughness dataset in order to find the Minimum Precision for Lowest Error (MPLE) at which the training and operation are still possible. The new findings show that compared to NNs, HONNs require more precision to reach a similar performance level, and that the 2nd order LM algorithm requires at least 24 bits of precision. The final investigation implemented and demonstrated the LM algorithm on Field Programmable Gate Arrays (FPGAs) for the first time in our knowledge. It was used to train a Neural Network, and the estimation of camera calibration parameters. The LM algorithm approximated NN to model the XOR function in only 13 iterations from zero initial conditions with a speed-up in excess of 3 x 10^6 compared to an implementation in software. Camera calibration was also demonstrated on FPGAs; compared to the software implementation, the FPGA implementation led to an increase in the mean squared error and standard deviation of only 17.94% and 8.04% respectively, but the FPGA increased the calibration speed by a factor of 1:41 x 106