28,771 research outputs found

    Unsupervised Learning via Total Correlation Explanation

    Full text link
    Learning by children and animals occurs effortlessly and largely without obvious supervision. Successes in automating supervised learning have not translated to the more ambiguous realm of unsupervised learning where goals and labels are not provided. Barlow (1961) suggested that the signal that brains leverage for unsupervised learning is dependence, or redundancy, in the sensory environment. Dependence can be characterized using the information-theoretic multivariate mutual information measure called total correlation. The principle of Total Cor-relation Ex-planation (CorEx) is to learn representations of data that "explain" as much dependence in the data as possible. We review some manifestations of this principle along with successes in unsupervised learning problems across diverse domains including human behavior, biology, and language.Comment: Invited contribution for IJCAI 2017 Early Career Spotlight. 5 pages, 1 figur

    Nonparametric Feature Extraction from Dendrograms

    Full text link
    We propose feature extraction from dendrograms in a nonparametric way. The Minimax distance measures correspond to building a dendrogram with single linkage criterion, with defining specific forms of a level function and a distance function over that. Therefore, we extend this method to arbitrary dendrograms. We develop a generalized framework wherein different distance measures can be inferred from different types of dendrograms, level functions and distance functions. Via an appropriate embedding, we compute a vector-based representation of the inferred distances, in order to enable many numerical machine learning algorithms to employ such distances. Then, to address the model selection problem, we study the aggregation of different dendrogram-based distances respectively in solution space and in representation space in the spirit of deep representations. In the first approach, for example for the clustering problem, we build a graph with positive and negative edge weights according to the consistency of the clustering labels of different objects among different solutions, in the context of ensemble methods. Then, we use an efficient variant of correlation clustering to produce the final clusters. In the second approach, we investigate the sequential combination of different distances and features sequentially in the spirit of multi-layered architectures to obtain the final features. Finally, we demonstrate the effectiveness of our approach via several numerical studies

    Evolutionary connectionism: algorithmic principles underlying the evolution of biological organisation in evo-devo, evo-eco and evolutionary transitions

    Get PDF
    The mechanisms of variation, selection and inheritance, on which evolution by natural selection depends, are not fixed over evolutionary time. Current evolutionary biology is increasingly focussed on understanding how the evolution of developmental organisations modifies the distribution of phenotypic variation, the evolution of ecological relationships modifies the selective environment, and the evolution of reproductive relationships modifies the heritability of the evolutionary unit. The major transitions in evolution, in particular, involve radical changes in developmental, ecological and reproductive organisations that instantiate variation, selection and inheritance at a higher level of biological organisation. However, current evolutionary theory is poorly equipped to describe how these organisations change over evolutionary time and especially how that results in adaptive complexes at successive scales of organisation (the key problem is that evolution is self-referential, i.e. the products of evolution change the parameters of the evolutionary process). Here we first reinterpret the central open questions in these domains from a perspective that emphasises the common underlying themes. We then synthesise the findings from a developing body of work that is building a new theoretical approach to these questions by converting well-understood theory and results from models of cognitive learning. Specifically, connectionist models of memory and learning demonstrate how simple incremental mechanisms, adjusting the relationships between individually-simple components, can produce organisations that exhibit complex system-level behaviours and improve the adaptive capabilities of the system. We use the term “evolutionary connectionism” to recognise that, by functionally equivalent processes, natural selection acting on the relationships within and between evolutionary entities can result in organisations that produce complex system-level behaviours in evolutionary systems and modify the adaptive capabilities of natural selection over time. We review the evidence supporting the functional equivalences between the domains of learning and of evolution, and discuss the potential for this to resolve conceptual problems in our understanding of the evolution of developmental, ecological and reproductive organisations and, in particular, the major evolutionary transitions

    The critical temperature of the 2D-Ising model through Deep Learning Autoencoders

    Full text link
    We investigate deep learning autoencoders for the unsupervised recognition of phase transitions in physical systems formulated on a lattice. We focus our investigation on the 2-dimensional ferromagnetic Ising model and then test the application of the autoencoder on the anti-ferromagnetic Ising model. We use spin configurations produced for the 2-dimensional ferromagnetic and anti-ferromagnetic Ising model in zero external magnetic field. For the ferromagnetic Ising model, we study numerically the relation between one latent variable extracted from the autoencoder to the critical temperature TcT_c. The proposed autoencoder reveals the two phases, one for which the spins are ordered and the other for which spins are disordered, reflecting the restoration of the Z2\mathbb{Z}_2 symmetry as the temperature increases. We provide a finite volume analysis for a sequence of increasing lattice sizes. For the largest volume studied, the transition between the two phases occurs very close to the theoretically extracted critical temperature. We define as a quasi-order parameter the absolute average latent variable z~{\tilde z}, which enables us to predict the critical temperature. One can define a latent susceptibility and use it to quantify the value of the critical temperature Tc(L)T_c(L) at different lattice sizes and that these values suffer from only small finite scaling effects. We demonstrate that Tc(L)T_c(L) extrapolates to the known theoretical value as LL \to \infty suggesting that the autoencoder can also be used to extract the critical temperature of the phase transition to an adequate precision. Subsequently, we test the application of the autoencoder on the anti-ferromagnetic Ising model, demonstrating that the proposed network can detect the phase transition successfully in a similar way.Comment: 17 pages, 14 figures, accepted for publication in Eur. Phys. J.
    corecore