Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise

We introduce coroICA, confounding-robust independent component analysis, a novel ICA algorithm which decomposes linearly mixed multivariate observations into independent components that are corrupted (and rendered dependent) by hidden group-wise stationary confounding. It extends the ordinary ICA model in a theoretically sound and explicit way to incorporate group-wise (or environment-wise) confounding. We show that our proposed general noise model allows to perform ICA in settings where other noisy ICA procedures fail. Additionally, it can be used for applications with grouped data by adjusting for different stationary noise within each group. Our proposed noise model has a natural relation to causality and we explain how it can be applied in the context of causal inference. In addition to our theoretical framework, we provide an efficient estimation procedure and prove identifiability of the unmixing matrix under mild assumptions. Finally, we illustrate the performance and robustness of our method on simulated data, provide audible and visual examples, and demonstrate the applicability to real-world scenarios by experiments on publicly available Antarctic ice core data as well as two EEG data sets. We provide a scikit-learn compatible pip-installable Python package coroICA as well as R and Matlab implementations accompanied by a documentation at https://sweichwald.de/coroICA/Comment: equal contribution between Pfister and Weichwal

Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise

We introduce coroICA, confounding-robust independent component analysis, a novel ICA algorithm which decomposes linearly mixed multivariate observations into independent components that are corrupted (and rendered dependent) by hidden group-wise stationary confounding. It extends the ordinary ICA model in a theoretically sound and explicit way to incorporate group-wise (or environment-wise) confounding. We show that our proposed general noise model allows to perform ICA in settings where other noisy ICA procedures fail. Additionally, it can be used for applications with grouped data by adjusting for different stationary noise within each group. Our proposed noise model has a natural relation to causality and we explain how it can be applied in the context of causal inference. In addition to our theoretical framework, we provide an efficient estimation procedure and prove identifiability of the unmixing matrix under mild assumptions. Finally, we illustrate the performance and robustness of our method on simulated data, provide audible and visual examples, and demonstrate the applicability to real-world scenarios by experiments on publicly available Antarctic ice core data as well as two EEG data sets. We provide a scikit-learn compatible pip-installable Python package coroICA as well as R and Matlab implementations accompanied by a documentation at https://sweichwald.de/coroICA/ISSN:1532-4435ISSN:1533-792

Advances in identifiability of nonlinear probabilistic models

Identifiability is a highly prized property of statistical models. This thesis investigates this property in nonlinear models encountered in two fields of statistics: representation learning and causal discovery. In representation learning, identifiability leads to learning interpretable and reproducible representations, while in causal discovery, it is necessary for the estimation of correct causal directions. We begin by leveraging recent advances in nonlinear ICA to show that the latent space of a VAE is identifiable up to a permutation and pointwise nonlinear transformations of its components. A factorized prior distribution over the latent variables conditioned on an auxiliary observed variable, such as a class label or nearly any other observation, is required for our result. We also extend previous identifiability results in nonlinear ICA to the case of noisy or undercomplete observations, and incorporate them into a maximum likelihood framework. Our second contribution is to develop the Independently Modulated Component Analysis (IMCA) framework, a generalization of nonlinear ICA to non-independent latent variables. We show that we can drop the independence assumption in ICA while maintaining identifiability, resulting in a very flexible and generic framework for principled disentangled representation learning. This finding is predicated on the existence of an auxiliary variable that modulates the joint distribution of the latent variables in a factorizable manner. As a third contribution, we extend the identifiability theory to a broad family of conditional energy-based models (EBMs). This novel model generalizes earlier results by removing any distributional assumptions on the representations, which are ubiquitous in the latent variable setting. The conditional EBM can learn identifiable overcomplete representations and has universal approximation capabilities/. Finally, we investigate a connection between the framework of autoregressive normalizing flow models and causal discovery. Causal models derived from affine autoregressive flows are shown to be identifiable, generalizing the wellknown additive noise model. Using normalizing flows, we can compute the exact likelihood of the causal model, which is subsequently used to derive a likelihood ratio measure for causal discovery. They are also invertible, making them perfectly suitable for performing causal inference tasks like interventions and counterfactuals

Artificial vision by thermography : calving prediction and defect detection in carbon fiber reinforced polymer

La vision par ordinateur est un domaine qui consiste à extraire ou identifier une ou plusieurs informations à partir d’une ou plusieurs images dans le but soit d’automatiser une tache, soit de fournir une aide à la décision. Avec l’augmentation de la capacité de calcul des ordinateurs, la vulgarisation et la diversification des moyens d’imagerie tant dans la vie quotidienne, que dans le milieu industriel,ce domaine a subi une évolution rapide lors de dernières décennies. Parmi les différentes modalités d’imagerie pour lesquels il est possible d’utiliser la vision artificielle cette thèse se concentre sur l’imagerie infrarouge. Plus particulièrement sur l’imagerie infrarouge pour les longueurs d’ondes comprises dans les bandes moyennes et longues. Cette thèse se porte sur deux applications industrielles radicalement différentes. Dans la première partie de cette thèse, nous présentons une application de la vision artificielle pour la détection du moment de vêlage en milieux industriel pour des vaches Holstein. Plus précisément l’objectif de cette recherche est de déterminer le moment de vêlage en n’utilisant que des données comportementales de l’animal. À cette fin, nous avons acquis des données en continu sur différents animaux pendant plusieurs mois. Parmi les nombreux défis présentés par cette application l’un d’entre eux concerne l’acquisition des données. En effet, les caméras que nous avons utilisées sont basées sur des capteurs bolométriques, lesquels sont sensibles à un grand nombre de variables. Ces variables peuvent être classées en quatre catégories : intrinsèque, environnemental, radiométrique et géométrique. Un autre défit important de cette recherche concerne le traitement des données. Outre le fait que les données acquises utilisent une dynamique plus élevée que les images naturelles ce qui complique le traitement des données ; l’identification de schéma récurrent dans les images et la reconnaissance automatique de ces derniers grâce à l’apprentissage automatique représente aussi un défi majeur. Nous avons proposé une solution à ce problème. Dans le reste de cette thèse nous nous sommes penchés sur la problématique de la détection de défaut dans les matériaux, en utilisant la technique de la thermographie pulsée. La thermographie pulsée est une méthode très populaire grâce à sa simplicité, la possibilité d’être utilisée avec un grand nombre de matériaux, ainsi que son faible coût. Néanmoins, cette méthode est connue pour produire des données bruitées. La cause principale de cette réputation vient des diverses sources de distorsion auquel les cameras thermiques sont sensibles. Dans cette thèse, nous avons choisi d’explorer deux axes. Le premier concerne l’amélioration des méthodes de traitement de données existantes. Dans le second axe, nous proposons plusieurs méthodes pour améliorer la détection de défauts. Chaque méthode est comparée à plusieurs méthodes constituant l’état de l’art du domaine.Abstract Computer vision is a field which consists in extracting or identifying one or more information from one or more images in order either to automate a task or to provide decision support. With the increase in the computing capacity of computers, the popularization and diversification of imaging means, both in industry, as well as in everyone’s life, this field has undergone a rapid development in recent decades. Among the different imaging modalities for which it is possible to use artificial vision, this thesis focuses on infrared imaging. More particularly on infrared imagery for wavelengths included in the medium and long bands. This thesis focuses on two radically different industrial applications. In the first part of this thesis, we present an application of artificial vision for the detection of the calving moment in industrial environments for Holstein cows. More precisely, the objective of this research is to determine the time of calving using only physiological data from the animal. To this end, we continuously acquired data on different animals over several days. Among the many challenges presented by this application, one of them concerns data acquisition. Indeed, the cameras we used are based on bolometric sensors, which are sensitive to a large number of variables. These variables can be classified into four categories: intrinsic, environmental, radiometric and geometric. Another important challenge in this research concerns the processing of data. Besides the fact that the acquired data uses a higher dynamic range than the natural images which complicates the processing of the data; Identifying recurring patterns in images and automatically recognizing them through machine learning is a major challenge. We have proposed a solution to this problem. In the rest of this thesis we have focused on the problem of defect detection in materials, using the technique of pulsed thermography. Pulse thermography is a very popular method due toits simplicity, the possibility of being used with a large number of materials, as well as its low cost. However, this method is known to produce noisy data. The main cause of this reputation comes from the various sources of distortion to which thermal cameras are sensitive. In this thesis, we have chosen to explore two axes. The first concerns the improvement of existing data processing methods. In the second axis, we propose several methods to improve fault detection. Each method is compared to several methods constituting the state of the art in the field