Deep Extreme Multi-label Learning

Extreme multi-label learning (XML) or classification has been a practical and important problem since the boom of big data. The main challenge lies in the exponential label space which involves $2^L$ possible label sets especially when the label dimension $L$ is huge, e.g., in millions for Wikipedia labels. This paper is motivated to better explore the label space by originally establishing an explicit label graph. In the meanwhile, deep learning has been widely studied and used in various classification problems including multi-label classification, however it has not been properly introduced to XML, where the label space can be as large as in millions. In this paper, we propose a practical deep embedding method for extreme multi-label classification, which harvests the ideas of non-linear embedding and graph priors-based label space modeling simultaneously. Extensive experiments on public datasets for XML show that our method performs competitive against state-of-the-art result

The spatial distribution of substellar objects in IC348 and the Orion Trapezium Cluster

Aims: Some theoretical scenarios suggest the formation of brown dwarfs as ejected stellar embryos in star-forming clusters. Such a formation mechanism can result in different spatial distributions of stars and substellar objects. We aim to investigate the spatial structure of stellar and substellar objects in two well sampled and nearby embedded clusters, namely IC348 and the Orion Trapezium Cluster (OTC) to test this hypothesis. Methods:Deep near-infrared K-band data complete enough to sample the substellar population in IC348 and OTC are obtained from the literature. The spatial distribution of the K-band point sources is analysed using the Minimum Spanning Tree (MST) method. The Q parameter and the spanning trees are evaluated for stellar and substellar objects as a function of cluster core radius R$_c$. Results: The stellar population in both IC348 and OTC display a clustered distribution whereas the substellar population is distributed homogeneously in space within twice the cluster core radius. Although the substellar objects do not appear to be bound by the cluster potential well, they are still within the limits of the cluster and not significantly displaced from their birth sites. Conclusions: The spatially homogeneous distribution of substellar objects is best explained by assuming higher initial velocities, distributed in a random manner and going through multiple interactions. The overall spatial coincidence of these objects with the cluster locations can be understood if these objects are nevertheless travelling slowly enough so as to feel the gravitational effect of the cluster. The observations support the formation of substellar objects as ``ejected stellar embryos''. Higher ejection velocities are necessary but net spatial displacements may not be necessary to explain the observational data.Comment: 4 pages. Accepted by A&A Letter

A mathematical theory of semantic development in deep neural networks

An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: what are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep learning dynamics to give rise to these regularities

An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor Analytics

Near-sensor data analytics is a promising direction for IoT endpoints, as it minimizes energy spent on communication and reduces network load - but it also poses security concerns, as valuable data is stored or sent over the network at various stages of the analytics pipeline. Using encryption to protect sensitive data at the boundary of the on-chip analytics engine is a way to address data security issues. To cope with the combined workload of analytics and encryption in a tight power envelope, we propose Fulmine, a System-on-Chip based on a tightly-coupled multi-core cluster augmented with specialized blocks for compute-intensive data processing and encryption functions, supporting software programmability for regular computing tasks. The Fulmine SoC, fabricated in 65nm technology, consumes less than 20mW on average at 0.8V achieving an efficiency of up to 70pJ/B in encryption, 50pJ/px in convolution, or up to 25MIPS/mW in software. As a strong argument for real-life flexible application of our platform, we show experimental results for three secure analytics use cases: secure autonomous aerial surveillance with a state-of-the-art deep CNN consuming 3.16pJ per equivalent RISC op; local CNN-based face detection with secured remote recognition in 5.74pJ/op; and seizure detection with encrypted data collection from EEG within 12.7pJ/op.Comment: 15 pages, 12 figures, accepted for publication to the IEEE Transactions on Circuits and Systems - I: Regular Paper