18 research outputs found
Integrating Flexible Normalization into Mid-Level Representations of Deep Convolutional Neural Networks
Deep convolutional neural networks (CNNs) are becoming increasingly popular
models to predict neural responses in visual cortex. However, contextual
effects, which are prevalent in neural processing and in perception, are not
explicitly handled by current CNNs, including those used for neural prediction.
In primary visual cortex, neural responses are modulated by stimuli spatially
surrounding the classical receptive field in rich ways. These effects have been
modeled with divisive normalization approaches, including flexible models,
where spatial normalization is recruited only to the degree responses from
center and surround locations are deemed statistically dependent. We propose a
flexible normalization model applied to mid-level representations of deep CNNs
as a tractable way to study contextual normalization mechanisms in mid-level
cortical areas. This approach captures non-trivial spatial dependencies among
mid-level features in CNNs, such as those present in textures and other visual
stimuli, that arise from tiling high order features, geometrically. We expect
that the proposed approach can make predictions about when spatial
normalization might be recruited in mid-level cortical areas. We also expect
this approach to be useful as part of the CNN toolkit, therefore going beyond
more restrictive fixed forms of normalization
DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies
We introduce an information-theoretic quantity with similar properties to
mutual information that can be estimated from data without making explicit
assumptions on the underlying distribution. This quantity is based on a
recently proposed matrix-based entropy that uses the eigenvalues of a
normalized Gram matrix to compute an estimate of the eigenvalues of an
uncentered covariance operator in a reproducing kernel Hilbert space. We show
that a difference of matrix-based entropies (DiME) is well suited for problems
involving the maximization of mutual information between random variables.
While many methods for such tasks can lead to trivial solutions, DiME naturally
penalizes such outcomes. We compare DiME to several baseline estimators of
mutual information on a toy Gaussian dataset. We provide examples of use cases
for DiME, such as latent factor disentanglement and a multiview representation
learning problem where DiME is used to learn a shared representation among
views with high mutual information
Consistent patterns of common species across tropical tree communities
Trees structure the Earth’s most biodiverse ecosystem, tropical forests. The vast number of tree species presents a formidable challenge to understanding these forests, including their response to environmental change, as very little is known about most tropical tree species. A focus on the common species may circumvent this challenge. Here we investigate abundance patterns of common tree species using inventory data on 1,003,805 trees with trunk diameters of at least 10 cm across 1,568 locations1,2,3,4,5,6 in closed-canopy, structurally intact old-growth tropical forests in Africa, Amazonia and Southeast Asia. We estimate that 2.2%, 2.2% and 2.3% of species comprise 50% of the tropical trees in these regions, respectively. Extrapolating across all closed-canopy tropical forests, we estimate that just 1,053 species comprise half of Earth’s 800 billion tropical trees with trunk diameters of at least 10 cm. Despite differing biogeographic, climatic and anthropogenic histories7, we find notably consistent patterns of common species and species abundance distributions across the continents. This suggests that fundamental mechanisms of tree community assembly may apply to all tropical forests. Resampling analyses show that the most common species are likely to belong to a manageable list of known species, enabling targeted efforts to understand their ecology. Although they do not detract from the importance of rare species, our results open new opportunities to understand the world’s most diverse forests, including modelling their response to environmental change, by focusing on the common species that constitute the majority of their trees.Publisher PDFPeer reviewe
Behavioral and neural constraints on hierarchical representations
Central to behavior and cognition is the way that sensory stimuli are represented in neural systems. The distributions over such stimuli enjoy rich structure; however, how the brain captures and exploits these regularities is unclear. Here, we consider different sources of perhaps the most prevalent form of structure, namely hierarchies, in one of its most prevalent cases, namely the representation of images. We review experimental approaches across a range of subfields, spanning inference, memory recall, and visual adaptation, to investigate how these constrain hierarchical representations. We also discuss progress in building hierarchical models of the representation of images-this has the potential to clarify how the structure of the world is reflected in biological systems. We suggest there is a need for a closer embedding of recent advances in machine learning and computer vision into the design and interpretation of experiments, notably by utilizing the understanding of the structure of natural scenes and through the creation of hierarchically structured synthetic stimuli
Recommended from our members
Deep neural networks capture texture sensitivity in V2
Deep convolutional neural networks (CNNs) trained on visual objects have shown intriguing ability to predict some response properties of visual cortical neurons. However, the factors (e.g., if the model is trained or not, receptive field size) and computations (e.g., convolution, rectification, pooling, normalization) that give rise to such ability, at what level, and the role of intermediate processing stages in explaining changes that develop across areas of the cortical hierarchy are poorly understood. We focused on the sensitivity to textures as a paradigmatic example, since recent neurophysiology experiments provide rich data pointing to texture sensitivity in secondary (but not primary) visual cortex (V2). We initially explored the CNN without any fitting to the neural data and found that the first two layers of the CNN showed qualitative correspondence to the first two cortical areas in terms of texture sensitivity. We therefore developed a quantitative approach to select a population of CNN model neurons that best fits the brain neural recordings. We found that the CNN could develop compatibility to secondary cortex in the second layer following rectification and that this was improved following pooling but only mildly influenced by the local normalization operation. Higher layers of the CNN could further, though modestly, improve the compatibility with the V2 data. The compatibility was reduced when incorporating random rather than learned weights. Our results show that the CNN class of model is effective for capturing changes that develop across early areas of cortex, and has the potential to help identify the computations that give rise to hierarchical processing in the brain (code is available in
GitHub
)
Recommended from our members
Stimulus- and goal-oriented frameworks for understanding natural vision
Our knowledge of sensory processing has advanced dramatically in the last few decades, but this understanding remains far from complete, especially for stimuli with the large dynamic range and strong temporal and spatial correlations characteristic of natural visual inputs. Here we describe some of the issues that make understanding the encoding of natural images a challenge. We highlight two broad strategies for approaching this problem: a stimulus-oriented framework and a goal-oriented one. Different contexts can call for one framework or the other. Looking forward, recent advances, particularly those based in machine learning, show promise in borrowing key strengths of both frameworks and by doing so illuminating a path to a more comprehensive understanding of the encoding of natural stimuli
The Representation Jensen-R\'enyi Divergence
We introduce a divergence measure between data distributions based on
operators in reproducing kernel Hilbert spaces defined by kernels. The
empirical estimator of the divergence is computed using the eigenvalues of
positive definite Gram matrices that are obtained by evaluating the kernel over
pairs of data points. The new measure shares similar properties to
Jensen-Shannon divergence. Convergence of the proposed estimators follows from
concentration results based on the difference between the ordered spectrum of
the Gram matrices and the integral operators associated with the population
quantities. The proposed measure of divergence avoids the estimation of the
probability distribution underlying the data. Numerical experiments involving
comparing distributions and applications to sampling unbalanced data for
classification show that the proposed divergence can achieve state of the art
results.Comment: We added acknowledgment