Unsupervised learning for long-term autonomy

This thesis investigates methods to enable a robot to build and maintain an environment model in an automatic manner. Such capabilities are especially important in long-term autonomy, where robots operate for extended periods of time without human intervention. In such scenarios we can no longer assume that the environment and the models will remain static. Rather changes are expected and the robot needs to adapt to the new, unseen, circumstances automatically. The approach described in this thesis is based on clustering the robot’s sensing information. This provides a compact representation of the data which can be updated as more information becomes available. The work builds on affinity propagation (Frey and Dueck, 2007), a recent clustering method which obtains high quality clusters while only requiring similarities between pairs of points, and importantly, selecting the number of clusters automatically. This is essential for real autonomy as we typically do not know “a priori” how many clusters best represent the data. The contributions of this thesis a three fold. First a self-supervised method capable of learning a visual appearance model in long-term autonomy settings is presented. Secondly, affinity propagation is extended to handle multiple sensor modalities, often occurring in robotics, in a principle way. Third, a method for joint clustering and outlier selection is proposed which selects a user defined number of outlier while clustering the data. This is solved using an extension of affinity propagation as well as a Lagrangian duality approach which provides guarantees on the optimality of the solution

Design and validation of structural health monitoring system based on bio-inspired algorithms

The need of ensure the proper performance of the structures in service has made of structural health monitoring (SHM) a priority research area. Researchers all around the world have focused efforts on the development of new ways to continuous monitoring the structures and analyze the data collected from the inspection process in order to provide information about the current state and avoid possible catastrophes. To perform an effective analysis of the data, the development of methodologies is crucial in order to assess the structures with a low computational cost and with a high reliability. These desirable features can be found in biological systems, and these can be emulated by means of computational systems. The use of bio-inspired algorithms is a recent approach that has demonstrated its effectiveness in data analysis in different areas. Since these algorithms are based in the emulation of biological systems that have demonstrated its effectiveness for several generations, it is possible to mimic the evolution process and its adaptability characteristics by using computational algorithms. Specially in pattern recognition, several algorithms have shown good performance. Some widely used examples are the neural networks, the fuzzy systems and the genetic algorithms. This thesis is concerned about the development of bio-inspired methodologies for structural damage detection and classification. This document is organized in five chapters. First, an overview of the problem statement, the objectives, general results, a brief theoretical background and the description of the different experimental setups are included in Chapter 1 (Introduction). Chapters 2 to 4 include the journal papers published by the author of this thesis. The discussion of the results, some conclusions and the future work can be found on Chapter 5. Finally, Appendix A includes other contributions such as a book chapter and some conference papers.La necesidad de asegurar el correcto funcionamiento de las estructuras en servicio ha hecho de la monitorización de la integridad estructural un área de gran interés. Investigadores en todas las partes del mundo centran sus esfuerzos en el desarrollo de nuevas formas de monitorización contínua de estructuras que permitan analizar e interpretar los datos recogidos durante el proceso de inspección con el objetivo de proveer información sobre el estado actual de la estructura y evitar posibles catástrofes. Para desarrollar un análisis efectivo de los datos, es necesario el desarrollo de metodologías para inspeccionar la estructura con un bajo coste computacional y alta fiabilidad. Estas características deseadas pueden ser encontradas en los sistemas biológicos y pueden ser emuladas mediante herramientas computacionales. El uso de algoritmos bio-inspirados es una reciente técnica que ha demostrado su efectividad en el análisis de datos en diferentes áreas. Dado que estos algoritmos se basan en la emulación de sistemas biológicos que han demostrado su efectividad a lo largo de muchas generaciones, es posible imitar el proceso de evolución y sus características de adaptabilidad al medio usando algoritmos computacionales. Esto es así, especialmente, en reconocimiento de patrones, donde muchos de estos algoritmos brindan excelentes resultados. Algunos ejemplos ampliamente usados son las redes neuronales, los sistemas fuzzy y los algoritmos genéticos. Esta tesis involucra el desarrollo de unas metodologías bio-inspiradas para la detección y clasificación de daños estructurales. El documento está organizado en cinco capítulos. En primer lugar, se incluye una descripción general del problema, los objetivos del trabajo, los resultados obtenidos, un breve marco conceptual y la descripción de los diferentes escenarios experimentales en el Capítulo 1 (Introducción). Los Capítulos 2 a 4 incluyen los artículos publicados en diferentes revistas indexadas. La revisión de los resultados, conclusiones y el trabajo futuro se encuentra en el Capítulo 5. Finalmente, el Anexo A incluye otras contribuciones tales como un capítulo de libro y algunos trabajos publicados en conferencias

Extracting Information from Multimodal Remote Sensing Data for Sea Ice Characterization

Remote sensing is the discipline that studies acquisition, preparation and analysis of spectral, spatial and temporal properties of objects without direct touch or contact. It is a field of great importance to understanding the climate system and its changes, as well as for conducting operations in the Arctic. A current challenge however is that most sensory equipment can only capture one or fewer of the characteristics needed to accurately describe ground objects through their temporal, spatial, spectral and radiometric resolution characteristics. This in turn motivates the fusing of complimentary modalities for potentially improved accuracy and stability in analysis but it also leads to problems when trying to merge heterogeneous data with different statistical, geometric and physical qualities. Another concern in the remote sensing of arctic regions is the scarcity of high quality labeled data but simultaneous abundance of unlabeled data as the gathering of labeled data can be both costly and time consuming. It could therefore be of great value to explore routes that can automate this process in ways that target both the situation regarding available data and the difficulties from fusing of heterogeneous multimodal data. To this end Semi-Supervised methods were considered for their ability to leverage smaller amounts of carefully labeled data in combination with more widely available unlabeled data in achieving greater classification performance. Strengths and limitations of three algorithms for real life applications are assessed through experiments on datasets from arctic and urban areas. The first two algorithms, Deep Semi-Supervised Label Propagation (LP) and MixMatch Holistic SSL (MixMatch), consider simultaneous processing of multimodal remote sensing data with additional extracted Gray Level Co-occurrence Matrix texture features for image classification. LP trains in alternating steps of supervised learning on potentially pseudolabeled data and steps of deciding new labels through node propagation while MixMatch mixes loss terms from several leading algorithms to gain their respective benefits. Another method, Graph Fusion Merriman Bence Osher (GMBO), explores processing of modalities in parallel by constructing a fused graph from complimentary input modalities and Ginzburg-Landau minimization on an approximated Graph Laplacian. Results imply that inclusion of extracted GLCM features could be beneficial for classification of multimodal remote sensing data, and that GMBO has merits for operational use in the Arctic given that certain data prerequisites are met

Improved EMD-Based Complex Prediction Model for Wind Power Forecasting

As a response to rapidly increasing penetration of wind power generation in modern electric power grids, accurate prediction models are crucial to deal with the associated uncertainties. Due to the highly volatile and chaotic nature of wind power, employing complex intelligent prediction tools is necessary. Accordingly, this article proposes a novel improved version of empirical mode decomposition (IEMD) to decompose wind measurements. The decomposed signal is provided as input to a hybrid forecasting model built on a bagging neural network (BaNN) combined with K-means clustering. Moreover, a new intelligent optimization method named ChB-SSO is applied to automatically tune the BaNN parameters. The performance of the proposed forecasting framework is tested using different seasonal subsets of real-world wind farm case studies (Alberta and Sotavento) through a comprehensive comparative analysis against other well-known prediction strategies. Furthermore, to analyze the effectiveness of the proposed framework, different forecast horizons have been considered in different test cases. Several error assessment criteria were used and the obtained results demonstrate the superiority of the proposed method for wind forecasting compared to other methods for all test cases.© 2020 Institute of Electrical and Electronics Engineersfi=vertaisarvioitu|en=peerReviewed

Pixel-Level Deep Multi-Dimensional Embeddings for Homogeneous Multiple Object Tracking

The goal of Multiple Object Tracking (MOT) is to locate multiple objects and keep track of their individual identities and trajectories given a sequence of (video) frames. A popular approach to MOT is tracking by detection consisting of two processing components: detection (identification of objects of interest in individual frames) and data association (connecting data from multiple frames). This work addresses the detection component by introducing a method based on semantic instance segmentation, i.e., assigning labels to all visible pixels such that they are unique among different instances. Modern tracking methods often built around Convolutional Neural Networks (CNNs) and additional, explicitly-defined post-processing steps. This work introduces two detection methods that incorporate multi-dimensional embeddings. We train deep CNNs to produce easily-clusterable embeddings for semantic instance segmentation and to enable object detection through pose estimation. The use of embeddings allows the method to identify per-pixel instance membership for both tasks. Our method specifically targets applications that require long-term tracking of homogeneous targets using a stationary camera. Furthermore, this method was developed and evaluated on a livestock tracking application which presents exceptional challenges that generalized tracking methods are not equipped to solve. This is largely because contemporary datasets for multiple object tracking lack properties that are specific to livestock environments. These include a high degree of visual similarity between targets, complex physical interactions, long-term inter-object occlusions, and a fixed-cardinality set of targets. For the reasons stated above, our method is developed and tested with the livestock application in mind and, specifically, group-housed pigs are evaluated in this work. Our method reliably detects pigs in a group housed environment based on the publicly available dataset with 99% precision and 95% using pose estimation and achieves 80% accuracy when using semantic instance segmentation at 50% IoU threshold. Results demonstrate our method\u27s ability to achieve consistent identification and tracking of group-housed livestock, even in cases where the targets are occluded and despite the fact that they lack uniquely identifying features. The pixel-level embeddings used by the proposed method are thoroughly evaluated in order to demonstrate their properties and behaviors when applied to real data. Adivser: Lance C. Pére