Fast Terrain Classification Using Variable-Length Representation for Autonomous Navigation

We propose a method for learning using a set of feature representations which retrieve different amounts of information at different costs. The goal is to create a more efficient terrain classification algorithm which can be used in real-time, onboard an autonomous vehicle. Instead of building a monolithic classifier with uniformly complex representation for each class, the main idea here is to actively consider the labels or misclassification cost while constructing the classifier. For example, some terrain classes might be easily separable from the rest, so very simple representation will be sufficient to learn and detect these classes. This is taken advantage of during learning, so the algorithm automatically builds a variable-length visual representation which varies according to the complexity of the classification task. This enables fast recognition of different terrain types during testing. We also show how to select a set of feature representations so that the desired terrain classification task is accomplished with high accuracy and is at the same time efficient. The proposed approach achieves a good trade-off between recognition performance and speedup on data collected by an autonomous robot

Some Objects Are More Equal Than Others: Measuring and Predicting Importance

We observe that everyday images contain dozens of objects, and that humans, in describing these images, give different priority to these objects. We argue that a goal of visual recognition is, therefore, not only to detect and classify objects but also to associate with each a level of priority which we call 'importance'. We propose a definition of importance and show how this may be estimated reliably from data harvested from human observers. We conclude by showing that a first-order estimate of importance may be computed from a number of simple image region measurements and does not require access to image meaning

A Comparison of RBF Neural Network Training Algorithms for Inertial Sensor Based Terrain Classification

This paper introduces a comparison of training algorithms of radial basis function (RBF) neural networks for classification purposes. RBF networks provide effective solutions in many science and engineering fields. They are especially popular in the pattern classification and signal processing areas. Several algorithms have been proposed for training RBF networks. The Artificial Bee Colony (ABC) algorithm is a new, very simple and robust population based optimization algorithm that is inspired by the intelligent behavior of honey bee swarms. The training performance of the ABC algorithm is compared with the Genetic algorithm, Kalman filtering algorithm and gradient descent algorithm. In the experiments, not only well known classification problems from the UCI repository such as the Iris, Wine and Glass datasets have been used, but also an experimental setup is designed and inertial sensor based terrain classification for autonomous ground vehicles was also achieved. Experimental results show that the use of the ABC algorithm results in better learning than those of others

Terrain Classification from Body-mounted Cameras during Human Locomotion

Abstract—This paper presents a novel algorithm for terrain type classification based on monocular video captured from the viewpoint of human locomotion. A texture-based algorithm is developed to classify the path ahead into multiple groups that can be used to support terrain classification. Gait is taken into account in two ways. Firstly, for key frame selection, when regions with homogeneous texture characteristics are updated, the fre-quency variations of the textured surface are analysed and used to adaptively define filter coefficients. Secondly, it is incorporated in the parameter estimation process where probabilities of path consistency are employed to improve terrain-type estimation. When tested with multiple classes that directly affect mobility a hard surface, a soft surface and an unwalkable area- our proposed method outperforms existing methods by up to 16%, and also provides improved robustness. Index Terms—texture, classification, recursive filter, terrain classification I

3D mapping and path planning from range data

This thesis reports research on mapping, terrain classification and path planning. These are classical problems in robotics, typically studied independently, and here we link such problems by framing them within a common proprioceptive modality, that of three-dimensional laser range scanning. The ultimate goal is to deliver navigation paths for challenging mobile robotics scenarios. For this reason we also deliver safe traversable regions from a previously computed globally consistent map. We first examine the problem of registering dense point clouds acquired at different instances in time. We contribute with a novel range registration mechanism for pairs of 3D range scans using point-to-point and point-to-line correspondences in a hierarchical correspondence search strategy. For the minimization we adopt a metric that takes into account not only the distance between corresponding points, but also the orientation of their relative reference frames. We also propose FaMSA, a fast technique for multi-scan point cloud alignment that takes advantage of the asserted point correspondences during sequential scan matching, using the point match history to speed up the computation of new scan matches. To properly propagate the model of the sensor noise and the scan matching, we employ first order error propagation, and to correct the error accumulation from local data alignment, we consider the probabilistic alignment of 3D point clouds using a delayed-state Extended Information Filter (EIF). In this thesis we adapt the Pose SLAM algorithm to the case of 3D range mapping, Pose SLAM is the variant of SLAM where only the robot trajectory is estimated and where sensor data is solely used to produce relative constraints between robot poses. These dense mapping techniques are tested in several scenarios acquired with our 3D sensors, producing impressively rich 3D environment models. The computed maps are then processed to identify traversable regions and to plan navigation sequences. In this thesis we present a pair of methods to attain high-level off-line classification of traversable areas, in which training data is acquired automatically from navigation sequences. Traversable features came from the robot footprint samples during manual robot motion, allowing us to capture terrain constrains not easy to model. Using only some of the traversed areas as positive training samples, our algorithms are tested in real scenarios to find the rest of the traversable terrain, and are compared with a naive parametric and some variants of the Support Vector Machine. Later, we contribute with a path planner that guarantees reachability at a desired robot pose with significantly lower computation time than competing alternatives. To search for the best path, our planner incrementally builds a tree using the A* algorithm, it includes a hybrid cost policy to efficiently expand the search tree, combining random sampling from the continuous space of kinematically feasible motion commands with a cost to goal metric that also takes into account the vehicle nonholonomic constraints. The planer also allows for node rewiring, and to speed up node search, our method includes heuristics that penalize node expansion near obstacles, and that limit the number of explored nodes. The method book-keeps visited cells in the configuration space, and disallows node expansion at those configurations in the first full iteration of the algorithm. We validate the proposed methods with experiments in extensive real scenarios from different very complex 3D outdoors environments, and compare it with other techniques such as the A*, RRT and RRT* algorithms.Esta tesis reporta investigación sobre el mapeo, clasificación de terreno y planificación de trayectorias. Estos son problemas clásicos en robótica los cuales generalmente se estudian de forma independiente, aquí se vinculan enmarcandolos con una modalidad propioceptiva común: un láser de rango 3D. El objetivo final es ofrecer trayectorias de navegación para escenarios complejos en el marco de la robótica móvil. Por esta razón también entregamos regiones transitables en un mapa global consistente calculado previamente. Primero examinamos el problema de registro de nubes de puntos adquiridas en diferentes instancias de tiempo. Contribuimos con un novedoso mecanismo de registro de pares de imagenes de rango 3D usando correspondencias punto a punto y punto a línea, en una estrategia de búsqueda de correspondencias jerárquica. Para la minimización optamos por una metrica que considera no sólo la distancia entre puntos, sino también la orientación de los marcos de referencia relativos. También proponemos FAMSA, una técnica para el registro rápido simultaneo de multiples nubes de puntos, la cual aprovecha las correspondencias de puntos obtenidas durante el registro secuencial, usando inicialmente la historia de correspondencias para acelerar el cálculo de las correspondecias en los nuevos registros de imagenes. Para propagar adecuadamente el modelo del ruido del sensor y del registro de imagenes, empleamos la propagación de error de primer orden, y para corregir el error acumulado del registro local, consideramos la alineación probabilística de nubes de puntos 3D utilizando un Filtro Extendido de Información de estados retrasados. En esta tesis adaptamos el algóritmo Pose SLAM para el caso de mapas con imagenes de rango 3D, Pose SLAM es la variante de SLAM donde solamente se estima la trayectoria del robot, usando los datos del sensor como restricciones relativas entre las poses robot. Estas técnicas de mapeo se prueban en varios escenarios adquiridos con nuestros sensores 3D produciendo modelos 3D impresionantes. Los mapas obtenidos se procesan para identificar regiones navegables y para planificar secuencias de navegación. Presentamos un par de métodos para lograr la clasificación de zonas transitables fuera de línea. Los datos de entrenamiento se adquieren de forma automática usando secuencias de navegación obtenidas manualmente. Las características transitables se captan de las huella de la trayectoria del robot, lo cual permite capturar restricciones del terreno difíciles de modelar. Con sólo algunas de las zonas transitables como muestras de entrenamiento positivo, nuestros algoritmos se prueban en escenarios reales para encontrar el resto del terreno transitable. Los algoritmos se comparan con algunas variantes de la máquina de soporte de vectores (SVM) y una parametrizacion ingenua. También, contribuimos con un planificador de trayectorias que garantiza llegar a una posicion deseada del robot en significante menor tiempo de cálculo a otras alternativas. Para buscar el mejor camino, nuestro planificador emplea un arbol de busqueda incremental basado en el algoritmo A*. Incluimos una póliza de coste híbrido para crecer de manera eficiente el árbol, combinando el muestro aleatorio del espacio continuo de comandos cinemáticos del robot con una métrica de coste al objetivo que también concidera las cinemática del robot. El planificador además permite reconectado de nodos, y, para acelerar la búsqueda de nodos, se incluye una heurística que penaliza la expansión de nodos cerca de los obstáculos, que limita el número de nodos explorados. El método conoce las céldas que ha visitado del espacio de configuraciones, evitando la expansión de nodos en configuraciones que han sido vistadas en la primera iteración completa del algoritmo. Los métodos propuestos se validán con amplios experimentos con escenarios reales en diferentes entornos exteriores, asi como su comparación con otras técnicas como los algoritmos A*, RRT y RRT*.Postprint (published version

Path Planning for incline terrain using Embodied Artificial Intelligence

Η Ενσώματη Τεχνητή Νοημοσύνη στοχεύει στο να καλύψει την ανάγκη για την αναπαράσταση ενός προβλήματος αναζήτησης, καθώς και την αναπαράσταση του τι συνιστά “καλή” λύση για το πρόβλημα αυτό σε μια έξυπνη μηχανή. Στην περίπτωση της παρούσας πτυχιακής, αυτή η έξυπνη μηχανή είναι ένα ρομπότ. Συνδυάζοντας την Τεχνητή Νοημοσύνη και την Ρομποτική μπορούμε να ορίσουμε πειράματα των οποίων ο χώρος αναζήτησης είναι ο φυσικός κόσμος και τα αποτελέσματα κάθε πράξης συνιστούν την αξιολόγηση της κάθε λύσης. Στο πλαίσιο της πτυχιακής μου είχα την ευκαιρία να πειραματιστώ με την ανάπτυξη αλγορίθμων Τεχνητής Νοημοσύνης οι οποίοι καθοδηγούν ένα μη επανδρωμένο όχημα εδάφους στην ανακάλυψη μιας λύσης ενός δύσκολου προβλήματος πλοήγησης σε εξωτερικό χώρο, όπως η διάσχιση ενός εδάφους με απότομη κλίση. Επιχείρησα να αντιμετωπίσω το πρόβλημα αυτό με τρεις διαφορετικές προσεγγίσεις, μία με αλγόριθμο Hill Climbing, μία με N-best αναζήτηση και μία με Εξελικτικό Αλγόριθμο, καθεμία με τα δικά της προτερήματα και τις δικές της αδυναμίες. Τελικά, δημιούργησα και αξιολόγησα επίδειξεις, τόσο σε προσομοιωμένα σενάρια όσο και σε ένα σενάριο στον πραγματικό κόσμο. Τα αποτελέσματα αυτών των επιδείξεων δείχνουν μία σαφή πρόοδο στην προσέγγιση του προαναφερθέντος προβλήματος από μία ρομποτική πλαρφόρμα.Embodied Artificial Intelligence aims to cover the need of a search problem’s representation, as well as the representation of what constitutes a “good” solution to this problem in a smart machine. In this thesis’ case, this smart machine is a robot. When we combine Artificial Intelligence and Robotics we can define experiments where the search space is the physical world and the results of each action constitute each solution’s evaluation. In my thesis’ context, I had the opportunity to experiment with the development of artificial intelligence algorithms that guide an unmanned ground vehicle to discover the solution of a tough outdoor navigation problem, like traversing a terrain region of steep incline. I attempted to face the problem with three different approaches. A Hill Climbing algorithm approach, a N-best search approach and an Evolutionary Algorithm approach, each one with its own strengths and weaknesses. In the end, I created and I evaluated demonstrations, both in simulated scenarios and in a real world scenario. The results of these demonstrations show a clear progress in the approach of the aforementioned problem, by the robotic platform

Modeling of Wheel-Soil Interaction for Small Ground Vehicles Operating on Granular Soil.

Unmanned ground vehicles continue to increase in importance for many industries, from planetary exploration to military defense. These vehicles require significantly fewer resources compared to manned vehicles while reducing risks to human life. Terramechanics can aid in the design and operation of small vehicles to help ensure they do not become immobilized due to limited traction or energy depletion. In this dissertation methods to improve terramechanics modeling for vehicle design and control of small unmanned ground vehicles (SUGVs) on granular soil are studied. Various techniques are developed to improve the computational speed and modeling capability for two terramechanics methods. In addition, a new terramechanics method is developed that incorporates both computational efficiency and modeling capability. First, two techniques for improving the computation performance of the semi-empirical Bekker terramechanics method are developed. The first technique stores Bekker calculations offline in lookup tables. The second technique approximates the stress distributions along the wheel-soil interface. These techniques drastically improve computation speed but do not address its empirical nature or assumption of steady-state operation. Next, the discrete element method (DEM) is modified and tuned to match soil test data, evaluated against the Bekker method, and used to determine the influence of rough terrain on SUGV performance. A velocity-dependent rolling resistance term is developed that reduced DEM simulation error for soil tests. DEM simulation shows that surface roughness can potentially have a significant impact on SUGV performance. DEM has many advantages compared to the Bekker method, including better locomotion prediction, however large computation costs limit its applicability for design and control. Finally, a surrogate DEM model (S-DEM) is developed to maintain the simulation accuracy and capabilities of DEM with reduced computation costs. This marks one of the first surrogate models developed for DEM, and the first known model developed for terramechanics. S-DEM stores wheel-soil interaction forces and soil velocities extracted from DEM simulations. S-DEM reproduces drawbar pull and driving torque for wheel locomotion on flat and rough terrain, though wheel sinkage error can be significant. Computational costs are reduced by three orders of magnitude, bringing the benefits of DEM modeling to vehicle design and control.PhDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/108811/1/wsmithw_1.pd

Automated taxiing for unmanned aircraft systems

Over the last few years, the concept of civil Unmanned Aircraft System(s) (UAS) has been realised, with small UASs commonly used in industries such as law enforcement, agriculture and mapping. With increased development in other areas, such as logistics and advertisement, the size and range of civil UAS is likely to grow. Taken to the logical conclusion, it is likely that large scale UAS will be operating in civil airspace within the next decade. Although the airborne operations of civil UAS have already gathered much research attention, work is also required to determine how UAS will function when on the ground. Motivated by the assumption that large UAS will share ground facilities with manned aircraft, this thesis describes the preliminary development of an Automated Taxiing System(ATS) for UAS operating at civil aerodromes. To allow the ATS to function on the majority of UAS without the need for additional hardware, a visual sensing approach has been chosen, with the majority of work focusing on monocular image processing techniques. The purpose of the computer vision system is to provide direct sensor data which can be used to validate the vehicle s position, in addition to detecting potential collision risks. As aerospace regulations require the most robust and reliable algorithms for control, any methods which are not fully definable or explainable will not be suitable for real-world use. Therefore, non-deterministic methods and algorithms with hidden components (such as Artificial Neural Network (ANN)) have not been used. Instead, the visual sensing is achieved through a semantic segmentation, with separate segmentation and classification stages. Segmentation is performed using superpixels and reachability clustering to divide the image into single content clusters. Each cluster is then classified using multiple types of image data, probabilistically fused within a Bayesian network. The data set for testing has been provided by BAE Systems, allowing the system to be trained and tested on real-world aerodrome data. The system has demonstrated good performance on this limited dataset, accurately detecting both collision risks and terrain features for use in navigation

Vision based environment perception system for next generation off-road ADAS : innovation report

Advanced Driver Assistance Systems (ADAS) aids the driver by providing information or automating the driving related tasks to improve driver comfort, reduce workload and improve safety. The vehicle senses its external environment using sensors, building a representation of the world used by the control systems. In on-road applications, the perception focuses on establishing the location of other road participants such as vehicles and pedestrians and identifying the road trajectory. Perception in the off-road environment is more complex, as the structure found in urban environments is absent. Off-road perception deals with the estimation of surface topography and surface type, which are the factors that will affect vehicle behaviour in unstructured environments. Off-road perception has seldom been explored in automotive context. For autonomous off-road driving, the perception solutions are primarily related to robotics and not directly applicable in the ADAS domain due to the different goals of unmanned autonomous systems, their complexity and the cost of employed sensors. Such applications consider only the impact of the terrain on the vehicle safety and progress but do not account for the driver comfort and assistance. This work addresses the problem of processing vision sensor data to extract the required information about the terrain. The main focus of this work is on the perception task with the constraints of automotive sensors and the requirements of the ADAS systems. By providing a semantic representation of the off-road environment including terrain attributes such as terrain type, description of the terrain topography and surface roughness, the perception system can cater for the requirements of the next generation of off-road ADAS proposed by Land Rover. Firstly, a novel and computationally efficient terrain recognition method was developed. The method facilitates recognition of low friction grass surfaces in real-time with high accuracy, by applying machine learning Support Vector Machine with illumination invariant normalised RGB colour descriptors. The proposed method was analysed and its performance was evaluated experimentally in off-road environments. Terrain recognition performance was evaluated on a variety of different surface types including grass, gravel and tarmac, showing high grass detection performance with accuracy of 97%. Secondly, a terrain geometry identification method was proposed which facilitates semantic representation of the terrain in terms of macro terrain features such as slopes, crest and ditches. The terrain geometry identification method processes 3D information reconstructed from stereo imagery and constructs a compact grid representation of the surface topography. This representation is further processed to extract object representation of slopes, ditches and crests. Thirdly, a novel method for surface roughness identification was proposed. The surface roughness descriptor is then further used to recommend a vehicle velocity, which will maintain passenger comfort. Surface roughness is described by the Power Spectral Density of the surface profile which correlates with the acceleration experienced by the vehicle. The surface roughness descriptor is then mapped onto vehicle speed recommendation so that the speed of the vehicle can be adapted in anticipation of the surface roughness. Terrain geometry and surface roughness identification performance were evaluated on a range of off-road courses with varying topology showing the capability of the system to correctly identify terrain features up to 20 m ahead of the vehicle and analyse surface roughness up to 15 m ahead of the vehicle. The speed was recommended correctly within +/- 5 kph. Further, the impact of the perception system on the speed adaptation was evaluated, showing the improvements in speed adaptation allowing for greater passenger comfort. The developed perception components facilitated the development of new off-road ADAS systems and were successfully applied in prototype vehicles. The proposed off-road ADAS are planned to be introduced in future generations of Land Rover products. The benefits of this research also included new Intellectual Property generated for Jaguar Land Rover. In the wider context, the enhanced off-road perception capability may facilitate further development of off-road automated driving and off-road autonomy within the constraints of the automotive platfor