Deep neural networks architectures from the perspective of manifold learning

Despite significant advances in the field of deep learning in ap-plications to various areas, an explanation of the learning pro-cess of neural network models remains an important open ques-tion. The purpose of this paper is a comprehensive comparison and description of neural network architectures in terms of ge-ometry and topology. We focus on the internal representation of neural networks and on the dynamics of changes in the topology and geometry of a data manifold on different layers. In this paper, we use the concepts of topological data analysis (TDA) and persistent homological fractal dimension. We present a wide range of experiments with various datasets and configurations of convolutional neural network (CNNs) architectures and Transformers in CV and NLP tasks. Our work is a contribution to the development of the important field of explainable and interpretable AI within the framework of geometrical deep learning.Comment: 11 pages, 12 figures, PRAI2023. arXiv admin note: substantial text overlap with arXiv:2204.0862

Unsupervised Anomaly Detection of High Dimensional Data with Low Dimensional Embedded Manifold

Anomaly detection techniques are supposed to identify anomalies from loads of seemingly homogeneous data and being able to do so can lead us to timely, pivotal and actionable decisions, saving us from potential human, financial and informational loss. In anomaly detection, an often encountered situation is the absence of prior knowledge about the nature of anomalies. Such circumstances advocate for ‘unsupervised’ learning-based anomaly detection techniques. Compared to its ‘supervised’ counterpart, which possesses the luxury to utilize a labeled training dataset containing both normal and anomalous samples, unsupervised problems are far more difficult. Moreover, high dimensional streaming data from tons of interconnected sensors present in modern day industries makes the task more challenging. To carry out an investigative effort to address these challenges is the overarching theme of this dissertation. In this dissertation, the fundamental issue of similarity measure among observations, which is a central piece in any anomaly detection techniques, is reassessed. Manifold hypotheses suggests the possibility of low dimensional manifold structure embedded in high dimensional data. In the presence of such structured space, traditional similarity measures fail to measure the true intrinsic similarity. In light of this revelation, reevaluating the notion of similarity measure seems more pressing rather than providing incremental improvements over any of the existing techniques. A graph theoretic similarity measure is proposed to differentiate and thus identify the anomalies from normal observations. Specifically, the minimum spanning tree (MST), a graph-based approach is proposed to approximate the similarities among data points in the presence of high dimensional structured space. It can track the structure of the embedded manifold better than the existing measures and help to distinguish the anomalies from normal observations. This dissertation investigates further three different aspects of the anomaly detection problem and develops three sets of solution approaches with all of them revolving around the newly proposed MST based similarity measure. In the first part of the dissertation, a local MST (LoMST) based anomaly detection approach is proposed to detect anomalies using the data in the original space. A two-step procedure is developed to detect both cluster and point anomalies. The next two sets of methods are proposed in the subsequent two parts of the dissertation, for anomaly detection in reduced data space. In the second part of the dissertation, a neighborhood structure assisted version of the nonnegative matrix factorization approach (NS-NMF) is proposed. To detect anomalies, it uses the neighborhood information captured by a sparse MST similarity matrix along with the original attribute information. To meet the industry demands, the online version of both LoMST and NS-NMF is also developed for real-time anomaly detection. In the last part of the dissertation, a graph regularized autoencoder is proposed which uses an MST regularizer in addition to the original loss function and is thus capable of maintaining the local invariance property. All of the approaches proposed in the dissertation are tested on 20 benchmark datasets and one real-life hydropower dataset. When compared with the state of art approaches, all three approaches produce statistically significant better outcomes. “Industry 4.0” is a reality now and it calls for anomaly detection techniques capable of processing a large amount of high dimensional data generated in real-time. The proposed MST based similarity measure followed by the individual techniques developed in this dissertation are equipped to tackle each of these issues and provide an effective and reliable real-time anomaly identification platform

Principes de méthodes " non classiques, non statistiques et massivement multivariées " et de réduction de la complexité. Applications en épidémiologie sociale et en médecine légale

Social epidemiology and clinical legal medicine are hybrid objects that articulate several fields, accounting for social and interpersonal relationships. The complexity that characterizes them both is investigated through different viewpoints, scales and dimensions: the individual scale, the group scale and the society scale. The techniques used in biomedicine are not designed to properly deal with such a complexity, in a non-normative way. A wide range of alternative non-statistical, “non-classical” methods exist that can process simultaneously various dimensions so that we can reduce the apparent complexity of data while discovering scientific objects. Here, we present the principles and the use of clustering techniques, applied to social epidemiology. We applied different clustering techniques on data from the SIRS cohort to build a typology of healthcare utilization in the Paris metropolitan area. From an epistemological and technical viewpoint, we explain why these methods should take place beside other recognized but limited techniques such as randomized controlled trials. We introduce another but complementary kind of complexity reduction technique. The concept of intrinsic dimension is explained – the littlest dimension needed to describe properly data – and nonlinear dimensionality reduction techniques are applied in clinical legal medicine. With these tools, we explore whether the integration of multiple information sources is relevant in age estimation of living migrants. Finally, we discuss the pros and cons of these methods, as well as the opportunities they may create for both fields of social epidemiology and clinical legal medicine.La complexité qui traverse l'épidémiologie sociale et la médecine légale du vivant est de celle que l'on cherche à saisir par la variété des observations et par l'intrication de points de vue et d'échelles différentes - l'individu, le groupe, la société. Les méthodes du biomédical sont encore peu adaptées au traitement de la complexité, à sa représentation qui ne soit pas normative, statistique. Il existe un ensemble d'approches non statistiques, " non classiques ", qui puissent traiter simultanément un grand nombre de dimensions et qui permettent de réduire la complexité apparente en dégageant des objets d'étude spécifique. Nous présentons ici les principes et l'utilisation des techniques de reconnaissance de forme dans le cadre de l'épidémiologie sociale, en les appliquant à la recherche d'une typologie de recours aux soins, sur la base des données de la cohorte SIRS. Nous expliquons en quoi ces approches ont leur place, épistémologiquement et techniquement parlant, aux côtés des méthodes expérimentales classiques type essais randomisés contrôlés. Nous exposons également un autre moyen de réduire la complexité des données, tout en en préservant les qualités topologiques. Nous introduisons en médecine légale la notion de dimension intrinsèque, plus petite dimension nécessaire et suffisante à la description des données, et de techniques non linéaires de réduction de la dimension. Nous en appliquons les principes au cas de l'intégration de sources d'information multiples pour l'estimation de l'âge chez les adolescents migrants. Enfin, nous discutons les avantages et limites de ces approches ainsi que les perspectives qu'elles ouvrent à ces deux disciplines complémentaires