Visual Prediction of Rover Slip: Learning Algorithms and Field Experiments

Perception of the surrounding environment is an essential tool for intelligent navigation in any autonomous vehicle. In the context of Mars exploration, there is a strong motivation to enhance the perception of the rovers beyond geometry-based obstacle avoidance, so as to be able to predict potential interactions with the terrain. In this thesis we propose to remotely predict the amount of slip, which reflects the mobility of the vehicle on future terrain. The method is based on learning from experience and uses visual information from stereo imagery as input. We test the algorithm on several robot platforms and in different terrains. We also demonstrate its usefulness in an integrated system, onboard a Mars prototype rover in the JPL Mars Yard. Another desirable capability for an autonomous robot is to be able to learn about its interactions with the environment in a fully automatic fashion. We propose an algorithm which uses the robot's sensors as supervision for vision-based learning of different terrain types. This algorithm can work with noisy and ambiguous signals provided from onboard sensors. To be able to cope with rich, high-dimensional visual representations we propose a novel, nonlinear dimensionality reduction technique which exploits automatic supervision. The method is the first to consider supervised nonlinear dimensionality reduction in a probabilistic framework using supervision which can be noisy or ambiguous. Finally, we consider the problem of learning to recognize different terrains, which addresses the time constraints of an onboard autonomous system. We propose a method which automatically learns a variable-length feature representation depending on the complexity of the classification task. The proposed approach achieves a good trade-off between decrease in computational time and recognition performance.</p

Simple but Effective Unsupervised Classification for Specified Domain Images: A Case Study on Fungi Images

High-quality labeled datasets are essential for deep learning. Traditional manual annotation methods are not only costly and inefficient but also pose challenges in specialized domains where expert knowledge is needed. Self-supervised methods, despite leveraging unlabeled data for feature extraction, still require hundreds or thousands of labeled instances to guide the model for effective specialized image classification. Current unsupervised learning methods offer automatic classification without prior annotation but often compromise on accuracy. As a result, efficiently procuring high-quality labeled datasets remains a pressing challenge for specialized domain images devoid of annotated data. Addressing this, an unsupervised classification method with three key ideas is introduced: 1) dual-step feature dimensionality reduction using a pre-trained model and manifold learning, 2) a voting mechanism from multiple clustering algorithms, and 3) post-hoc instead of prior manual annotation. This approach outperforms supervised methods in classification accuracy, as demonstrated with fungal image data, achieving 94.1% and 96.7% on public and private datasets respectively. The proposed unsupervised classification method reduces dependency on pre-annotated datasets, enabling a closed-loop for data classification. The simplicity and ease of use of this method will also bring convenience to researchers in various fields in building datasets, promoting AI applications for images in specialized domains

Gender classification based on gait analysis using ultrawide band radar augmented with artificial intelligence

The identification of individuals based on their walking patterns, also known as gait recognition, has garnered considerable interest as a biometric trait. The use of gait patterns for gender classification has emerged as a significant research domain with diverse applications across multiple fields. The present investigation centers on the classification of gender based on gait utilizing data from Ultra-wide band radar. A total of 181 participants were included in the study, and data was gathered using Ultra-wide band radar technology. This study investigates various preprocessing techniques, feature extraction methods, and dimensionality reduction approaches to efficiently process Ultra-wide band radar data. The data quality is improved through the utilization of a two-pulse canceller and discrete wavelet transform. The hybrid feature dataset is generated through the creation of gray-level co-occurrence matrices and subsequent extraction of statistical features. Principal Component Analysis is utilized for dimensionality reduction, and prediction probabilities are incorporated as features for classification optimization. The present study employs k-fold cross-validation to train and assess machine learning classifiers, Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, Multi-Layer Perceptron, K-Nearest Neighbors, and Extra Tree Classifier. The Multilayer Perceptron exhibits superior performance, achieving an accuracy of 0.936. The Support Vector Machine and k-Nearest Neighbors classifiers closely trail behind, both achieving an accuracy of 0.934. This research is of the utmost importance due to its capacity to offer solutions to crucial problems in multiple domains. The findings indicate that the utilization of UWB radar data for gait-based gender classification holds promise in diverse domains, including biometrics, surveillance, and healthcare. The present study makes a valuable contribution to the progress of gender classification systems that rely on gait patterns

CloudFCN: Accurate and robust cloud detection for satellite imagery with deep learning

Cloud masking is of central importance to the Earth Observation community. This paper deals with the problem of detecting clouds in visible and multispectral imagery from high-resolution satellite cameras. Recently, Machine Learning has offered promising solutions to the problem of cloud masking, allowing for more flexibility than traditional thresholding techniques, which are restricted to instruments with the requisite spectral bands. However, few studies use multi-scale features (as in, a combination of pixel-level and spatial) whilst also offering compelling experimental evidence for real-world performance. Therefore, we introduce CloudFCN, based on a Fully Convolutional Network architecture, known as U-net, which has become a standard Deep Learning approach to image segmentation. It fuses the shallowest and deepest layers of the network, thus routing low-level visible content to its deepest layers. We offer an extensive range of experiments on this, including data from two high-resolution sensors-Carbonite-2 and Landsat 8-and several complementary tests. Owing to a variety of performance-enhancing design choices and training techniques, it exhibits state-of-the-art performance where comparable to other methods, high speed, and robustness to many different terrains and sensor types

Wearable inertial sensor system towards daily human kinematic gait analysis: benchmarking analysis to MVN BIOMECH

This paper presents a cost- and time-effective wearable inertial sensor system, the InertialLAB. It includes gyroscopes and accelerometers for the real-time monitoring of 3D-angular velocity and 3D-acceleration of up to six lower limbs and trunk segment and sagittal joint angle up to six joints. InertialLAB followed an open architecture with a low computational load to be executed by wearable processing units up to 200 Hz for fostering kinematic gait data to third-party systems, advancing similar commercial systems. For joint angle estimation, we developed a trigonometric method based on the segments’ orientation previously computed by fusion-based methods. The validation covered healthy gait patterns in varying speed and terrain (flat, ramp, and stairs) and including turns, extending the experiments approached in the literature. The benchmarking analysis to MVN BIOMECH reported that InertialLAB provides more reliable measures in stairs than in flat terrain and ramp. The joint angle time-series of InertialLAB showed good waveform similarity (>0.898) with MVN BIOMECH, resulting in high reliability and excellent validity. User-independent neural network regression models successfully minimized the drift errors observed in InertialLAB’s joint angles (NRMSE < 0.092). Further, users ranked InertialLAB as good in terms of usability. InertialLAB shows promise for daily kinematic gait analysis and real-time kinematic feedback for wearable third-party systems.This work has been supported in part by the Fundação para a Ciência e Tecnologia (FCT) with the Reference Scholarship under Grant SFRH/BD/108309/2015 and SFRH/BD/147878/2019, by the FEDER Funds through the Programa Operacional Regional do Norte and national funds from FCT with the project SmartOs under Grant NORTE-01-0145-FEDER-030386, and through the COMPETE 2020—Programa Operacional Competitividade e Internacionalização (POCI)—with the Reference Project under Grant POCI-01-0145-FEDER-006941

LiDAR-based Semantic Labeling : Automotive 3D Scene Understanding

Mobile Roboter und autonome Fahrzeuge verwenden verschiedene Sensormodalitäten zur Erkennung und Interpretation ihrer Umgebung. Neben Kameras und RaDAR Sensoren repräsentieren LiDAR Sensoren eine zentrale Komponente für moderne Methoden der Umgebungswahrnehmung. Zusätzlich zu einer präzisen Distanzmessung dieser Sensoren, ist ein umfangreiches semantisches Szeneverständnis notwendig, um ein effizientes und sicheres Agieren autonomer Systeme zu ermöglichen. In dieser Arbeit wird das neu entwickelte LiLaNet, eine echtzeitfähige, neuronale Netzarchitektur zur semantischen, punktweisen Klassifikation von LiDAR Punktwolken, vorgestellt. Hierfür finden die Ansätze der 2D Bildverarbeitung Verwendung, indem die 3D LiDAR Punktwolke als 2D zylindrisches Bild dargestellt wird. Dadurch werden Ergebnisse moderner Ansätze zur LiDAR-basierten, punktweisen Klassifikation übertroffen, was an unterschiedlichen Datensätzen demonstriert wird. Zur Entwicklung von Ansätzen des maschinellen Lernens, wie sie in dieser Arbeit verwendet werden, spielen umfangreiche Datensätze eine elementare Rolle. Aus diesem Grund werden zwei Datensätze auf Basis von modernen LiDAR Sensoren erzeugt. Durch das in dieser Arbeit entwickelte automatische Verfahren zur Datensatzgenerierung auf Basis von mehreren Sensormodalitäten, speziell der Kamera und des LiDAR Sensors, werden Kosten und Zeit der typischerweise manuellen Datensatzgenerierung reduziert. Zusätzlich wird eine multimodale Datenkompression vorgestellt, welche ein Kompressionsverfahren der Stereokamera auf den LiDAR Sensor überträgt. Dies führt zu einer Reduktion der LiDAR Daten bei gleichzeitigem Erhalt der zugrundeliegenden semantischen und geometrischen Information. Daraus resultiert eine erhöhte Echtzeitfähigkeit nachgelagerter Algorithmen autonomer Systeme. Außerdem werden zwei Erweiterungen zum vorgestellten Verfahren der semantischen Klassifikation umrissen. Zum einen wird die Sensorabhängigkeit durch Einführung des PiLaNets, einer neuen 3D Netzarchitektur, reduziert indem die LiDAR Punktwolke im 3D kartesischen Raum belassen wird, um die eher sensorabhängige 2D zylindrische Projektion zu ersetzen. Zum anderen wird die Unsicherheit neuronaler Netze implizit modelliert, indem eine Klassenhierarchie in den Trainingsprozess integriert wird. Insgesamt stellt diese Arbeit neuartige, performante Ansätze des 3D LiDAR-basierten, semantischen Szeneverstehens vor, welche zu einer Verbesserung der Leistung, Zuverlässigkeit und Sicherheit zukünftiger mobile Roboter und autonomer Fahrzeuge beitragen

Introspective Perception for Mobile Robots

Perception algorithms that provide estimates of their uncertainty are crucial to the development of autonomous robots that can operate in challenging and uncontrolled environments. Such perception algorithms provide the means for having risk-aware robots that reason about the probability of successfully completing a task when planning. There exist perception algorithms that come with models of their uncertainty; however, these models are often developed with assumptions, such as perfect data associations, that do not hold in the real world. Hence the resultant estimated uncertainty is a weak lower bound. To tackle this problem we present introspective perception - a novel approach for predicting accurate estimates of the uncertainty of perception algorithms deployed on mobile robots. By exploiting sensing redundancy and consistency constraints naturally present in the data collected by a mobile robot, introspective perception learns an empirical model of the error distribution of perception algorithms in the deployment environment and in an autonomously supervised manner. In this paper, we present the general theory of introspective perception and demonstrate successful implementations for two different perception tasks. We provide empirical results on challenging real-robot data for introspective stereo depth estimation and introspective visual simultaneous localization and mapping and show that they learn to predict their uncertainty with high accuracy and leverage this information to significantly reduce state estimation errors for an autonomous mobile robot