Unmasking Clever Hans Predictors and Assessing What Machines Really Learn

Current learning machines have successfully solved hard application problems, reaching high accuracy and displaying seemingly "intelligent" behavior. Here we apply recent techniques for explaining decisions of state-of-the-art learning machines and analyze various tasks from computer vision and arcade games. This showcases a spectrum of problem-solving behaviors ranging from naive and short-sighted, to well-informed and strategic. We observe that standard performance evaluation metrics can be oblivious to distinguishing these diverse problem solving behaviors. Furthermore, we propose our semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. This helps to assess whether a learned model indeed delivers reliably for the problem that it was conceived for. Furthermore, our work intends to add a voice of caution to the ongoing excitement about machine intelligence and pledges to evaluate and judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication

Robust Modular Feature-Based Terrain-Aided Visual Navigation and Mapping

The visual feature-based Terrain-Aided Navigation (TAN) system presented in this thesis addresses the problem of constraining inertial drift introduced into the location estimate of Unmanned Aerial Vehicles (UAVs) in GPS-denied environment. The presented TAN system utilises salient visual features representing semantic or human-interpretable objects (roads, forest and water boundaries) from onboard aerial imagery and associates them to a database of reference features created a-priori, through application of the same feature detection algorithms to satellite imagery. Correlation of the detected features with the reference features via a series of the robust data association steps allows a localisation solution to be achieved with a finite absolute bound precision defined by the certainty of the reference dataset. The feature-based Visual Navigation System (VNS) presented in this thesis was originally developed for a navigation application using simulated multi-year satellite image datasets. The extension of the system application into the mapping domain, in turn, has been based on the real (not simulated) flight data and imagery. In the mapping study the full potential of the system, being a versatile tool for enhancing the accuracy of the information derived from the aerial imagery has been demonstrated. Not only have the visual features, such as road networks, shorelines and water bodies, been used to obtain a position ’fix’, they have also been used in reverse for accurate mapping of vehicles detected on the roads into an inertial space with improved precision. Combined correction of the geo-coding errors and improved aircraft localisation formed a robust solution to the defense mapping application. A system of the proposed design will provide a complete independent navigation solution to an autonomous UAV and additionally give it object tracking capability

Seguimento de pessoas com drones em espaços inteligentes

Recent technological progress made over the last decades in the field of Computer Vision has introduced new methods and algorithms with ever increasing performance results. Particularly, the emergence of machine learning algorithms enabled class based object detection on live video feeds. Alongside these advances, Unmanned Aerial Vehicles (more commonly known as drones), have also experienced advancements in both hardware miniaturization and software optimization. Thanks to these improvements, drones have emerged from their military usage based background and are now both used by the general public and the scientific community for applications as distinct as aerial photography and environmental monitoring. This dissertation aims to take advantage of these recent technological advancements and apply state of the art machine learning algorithms in order to create a Unmanned Aerial Vehicle (UAV) based network architecture capable of performing real time people tracking through image detection. To perform object detection, two distinct machine learning algorithms are presented. The first one uses an SVM based approach, while the second one uses an Convolutional Neural Network (CNN) based architecture. Both methods will be evaluated using an image dataset created for the purposes of this dissertation’s work. The evaluations performed regarding the object detectors performance showed that the method using a CNN based architecture was the best both in terms of processing time required and detection accuracy, and therefore, the most suitable method for our implementation. The developed network architecture was tested in a live scenario context, with the results showing that the system is capable of performing people tracking at average walking speeds.O recente progresso tecnológico registado nas últimas décadas no campo da Visão por Computador introduziu novos métodos e algoritmos com um desempenho cada vez mais elevado. Particularmente, a criação de algoritmos de aprendizagem automática tornou possível a detecção de objetos aplicada a feeds de vídeo capturadas em tempo real. Paralelo com este progresso, a tecnologia relativa a veículos aéreos não tripulados, ou drones, também beneficiaram de avanços tanto na miniaturização dos seus componentes de hardware assim como na optimização do software. Graças a essas melhorias, os drones emergiram do seu passado militar e são agora usados tanto pelo público em geral como pela comunidade científica para aplicações tão distintas como fotografia e monitorização ambiental. O objectivo da presente dissertação pretende tirar proveito destes recentes avanços tecnológicos e aplicar algoritmos de aprendizagem automática de última geração para criar um sistema capaz de realizar seguimento automático de pessoas com drones através de visão por computador. Para realizar a detecção de objetos, dois algoritmos distintos de aprendizagem automática são apresentados. O primeiro é dotado de uma abordagem baseada em Support Vector Machine (SVM), enquanto o segundo é caracterizado por uma arquitetura baseada em Redes Neuronais Convolucionais. Ambos os métodos serão avaliados usando uma base de dados de imagens criada para os propósitos da presente dissertação. As avaliações realizadas relativas ao desempenho dos algoritmos de detecção de objectos demonstraram que o método baseado numa arquitetura de Redes Neuronais Covolucionais foi o melhor tanto em termos de tempo de processamento médio assim como na precisão das detecções, revelando-se portanto, como sendo o método mais adequado de acordo com os objectivos pretendidos. O sistema desenvolvido foi testado num contexto real, com os resultados obtidos a demonstrarem que o sistema é capaz de realizar o seguimento de pessoas a velocidades comparáveis a um ritmo normal humano de caminhada.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

Discriminative tracking using tensor pooling

How to effectively organize local descriptors to build a global representation has a critical impact on the performance of vision tasks. Recently, local sparse representation has been successfully applied to visual tracking, owing to its discriminative nature and robustness against local noise and partial occlusions. Local sparse codes computed with a template actually form a three-order tensor according to their original layout, although most existing pooling operators convert the codes to a vector by concatenating or computing statistics on them. We argue that, compared to pooling vectors, the tensor form could deliver more intrinsic structural information for the target appearance, and can also avoid high dimensionality learning problems suffered in concatenation-based pooling methods. Therefore, in this paper, we propose to represent target templates and candidates directly with sparse coding tensors, and build the appearance model by incrementally learning on these tensors. We propose a discriminative framework to further improve robustness of our method against drifting and environmental noise. Experiments on a recent comprehensive benchmark indicate that our method performs better than state-of-the-art trackers

Dynamic Thermal Imaging for Intraoperative Monitoring of Neuronal Activity and Cortical Perfusion

Neurosurgery is a demanding medical discipline that requires a complex interplay of several neuroimaging techniques. This allows structural as well as functional information to be recovered and then visualized to the surgeon. In the case of tumor resections this approach allows more fine-grained differentiation of healthy and pathological tissue which positively influences the postoperative outcome as well as the patient's quality of life. In this work, we will discuss several approaches to establish thermal imaging as a novel neuroimaging technique to primarily visualize neural activity and perfusion state in case of ischaemic stroke. Both applications require novel methods for data-preprocessing, visualization, pattern recognition as well as regression analysis of intraoperative thermal imaging. Online multimodal integration of preoperative and intraoperative data is accomplished by a 2D-3D image registration and image fusion framework with an average accuracy of 2.46 mm. In navigated surgeries, the proposed framework generally provides all necessary tools to project intraoperative 2D imaging data onto preoperative 3D volumetric datasets like 3D MR or CT imaging. Additionally, a fast machine learning framework for the recognition of cortical NaCl rinsings will be discussed throughout this thesis. Hereby, the standardized quantification of tissue perfusion by means of an approximated heating model can be achieved. Classifying the parameters of these models yields a map of connected areas, for which we have shown that these areas correlate with the demarcation caused by an ischaemic stroke segmented in postoperative CT datasets. Finally, a semiparametric regression model has been developed for intraoperative neural activity monitoring of the somatosensory cortex by somatosensory evoked potentials. These results were correlated with neural activity of optical imaging. We found that thermal imaging yields comparable results, yet doesn't share the limitations of optical imaging. In this thesis we would like to emphasize that thermal imaging depicts a novel and valid tool for both intraoperative functional and structural neuroimaging

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available