Predictability, complexity and learning

We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm pred} (T)$ can remain finite, grow logarithmically, or grow as a fractional power law. If the time series allows us to learn a model with a finite number of parameters, then $I_{\rm pred} (T)$ grows logarithmically with a coefficient that counts the dimensionality of the model space. In contrast, power--law growth is associated, for example, with the learning of infinite parameter (or nonparametric) models such as continuous functions with smoothness constraints. There are connections between the predictive information and measures of complexity that have been defined both in learning theory and in the analysis of physical systems through statistical mechanics and dynamical systems theory. Further, in the same way that entropy provides the unique measure of available information consistent with some simple and plausible conditions, we argue that the divergent part of $I_{\rm pred} (T)$ provides the unique measure for the complexity of dynamics underlying a time series. Finally, we discuss how these ideas may be useful in different problems in physics, statistics, and biology.Comment: 53 pages, 3 figures, 98 references, LaTeX2

Comparing sets of data sets on the Grassmann and flag manifolds with applications to data analysis in high and low dimensions

Includes bibliographical references.2020 Summer.This dissertation develops numerical algorithms for comparing sets of data sets utilizing shape and orientation of data clouds. Two key components for "comparing" are the distance measure between data sets and correspondingly the geodesic path in between. Both components will play a core role which connects two parts of this dissertation, namely data analysis on the Grassmann manifold and flag manifold. For the first part, we build on the well known geometric framework for analyzing and optimizing over data on the Grassmann manifold. To be specific, we extend the classical self-organizing mappings to the Grassamann manifold to visualize sets of high dimensional data sets in 2D space. We also propose an optimization problem on the Grassmannian to recover missing data. In the second part, we extend the geometric framework to the flag manifold to encode the variability of nested subspaces. There we propose a numerical algorithm for computing a geodesic path and distance between nested subspaces. We also prove theorems to show how to reduce the dimension of the algorithm for practical computations. The approach is shown to have advantages for analyzing data when the number of data points is larger than the number of features

Data-driven covariance estimation for the iterative closest point algorithm

Les nuages de points en trois dimensions sont un format de données très commun en robotique mobile. Ils sont souvent produits par des capteurs spécialisés de type lidar. Les nuages de points générés par ces capteurs sont utilisés dans des tâches impliquant de l’estimation d’état, telles que la cartographie ou la localisation. Les algorithmes de recalage de nuages de points, notamment l’algorithme ICP (Iterative Closest Point), nous permettent de prendre des mesures d’égo-motion nécessaires à ces tâches. La fusion des recalages dans des chaînes existantes d’estimation d’état dépend d’une évaluation précise de leur incertitude. Cependant, les méthodes existantes d’estimation de l’incertitude se prêtent mal aux données en trois dimensions. Ce mémoire vise à estimer l’incertitude de recalages 3D issus d’Iterative Closest Point (ICP). Premièrement, il pose des fondations théoriques desquelles nous pouvons articuler une estimation de la covariance. Notamment, il révise l’algorithme ICP, avec une attention spéciale sur les parties qui sont importantes pour l’estimation de la covariance. Ensuite, un article scientifique inséré présente CELLO-3D, notre algorithme d’estimation de la covariance d’ICP. L’article inséré contient une validation expérimentale complète du nouvel algorithme. Il montre que notre algorithme performe mieux que les méthodes existantes dans une grande variété d’environnements. Finalement, ce mémoire est conclu par des expérimentations supplémentaires, qui sont complémentaires à l’article.Three-dimensional point clouds are an ubiquitous data format in robotics. They are produced by specialized sensors such as lidars or depth cameras. The point clouds generated by those sensors are used for state estimation tasks like mapping and localization. Point cloud registration algorithms, such as Iterative Closest Point (ICP), allow us to make ego-motion measurements necessary to those tasks. The fusion of ICP registrations in existing state estimation frameworks relies on an accurate estimation of their uncertainty. Unfortunately, existing covariance estimation methods often scale poorly to the 3D case. This thesis aims to estimate the uncertainty of ICP registrations for 3D point clouds. First, it poses theoretical foundations from which we can articulate a covariance estimation method. It reviews the ICP algorithm, with a special focus on the parts of it that are pertinent to covariance estimation. Then, an inserted article introduces CELLO-3D, our data-driven covariance estimation method for ICP. The article contains a thorough experimental validation of the new algorithm. The latter is shown to perform better than existing covariance estimation techniques in a wide variety of environments. Finally, this thesis comprises supplementary experiments, which complement the article

Learning of Surgical Gestures for Robotic Minimally Invasive Surgery Using Dynamic Movement Primitives and Latent Variable Models

Full and partial automation of Robotic Minimally Invasive Surgery holds significant promise to improve patient treatment, reduce recovery time, and reduce the fatigue of the surgeons. However, to accomplish this ambitious goal, a mathematical model of the intervention is needed. In this thesis, we propose to use Dynamic Movement Primitives (DMPs) to encode the gestures a surgeon has to perform to achieve a task. DMPs allow to learn a trajectory, thus imitating the dexterity of the surgeon, and to execute it while allowing to generalize it both spatially (to new starting and goal positions) and temporally (to different speeds of executions). Moreover, they have other desirable properties that make them well suited for surgical applications, such as online adaptability, robustness to perturbations, and the possibility to implement obstacle avoidance. We propose various modifications to improve the state-of-the-art of the framework, as well as novel methods to handle obstacles. Moreover, we validate the usage of DMPs to model gestures by automating a surgical-related task and using DMPs as the low-level trajectory generator. In the second part of the thesis, we introduce the problem of unsupervised segmentation of tasks' execution in gestures. We will introduce latent variable models to tackle the problem, proposing further developments to combine such models with the DMP theory. We will review the Auto-Regressive Hidden Markov Model (AR-HMM) and test it on surgical-related datasets. Then, we will propose a generalization of the AR-HMM to general, non-linear, dynamics, showing that this results in a more accurate segmentation, with a less severe over-segmentation. Finally, we propose a further generalization of the AR-HMM that aims at integrating a DMP-like dynamic into the latent variable model