46 research outputs found
Deep SimNets
We present a deep layered architecture that generalizes convolutional neural
networks (ConvNets). The architecture, called SimNets, is driven by two
operators: (i) a similarity function that generalizes inner-product, and (ii) a
log-mean-exp function called MEX that generalizes maximum and average. The two
operators applied in succession give rise to a standard neuron but in "feature
space". The feature spaces realized by SimNets depend on the choice of the
similarity operator. The simplest setting, which corresponds to a convolution,
realizes the feature space of the Exponential kernel, while other settings
realize feature spaces of more powerful kernels (Generalized Gaussian, which
includes as special cases RBF and Laplacian), or even dynamically learned
feature spaces (Generalized Multiple Kernel Learning). As a result, the SimNet
contains a higher abstraction level compared to a traditional ConvNet. We argue
that enhanced expressiveness is important when the networks are small due to
run-time constraints (such as those imposed by mobile applications). Empirical
evaluation validates the superior expressiveness of SimNets, showing a
significant gain in accuracy over ConvNets when computational resources at
run-time are limited. We also show that in large-scale settings, where
computational complexity is less of a concern, the additional capacity of
SimNets can be controlled with proper regularization, yielding accuracies
comparable to state of the art ConvNets
Neural tensor contractions and the expressive power of deep neural quantum states
We establish a direct connection between general tensor networks and deep
feed-forward artificial neural networks. The core of our results is the
construction of neural-network layers that efficiently perform tensor
contractions, and that use commonly adopted non-linear activation functions.
The resulting deep networks feature a number of edges that closely matches the
contraction complexity of the tensor networks to be approximated. In the
context of many-body quantum states, this result establishes that
neural-network states have strictly the same or higher expressive power than
practically usable variational tensor networks. As an example, we show that all
matrix product states can be efficiently written as neural-network states with
a number of edges polynomial in the bond dimension and depth logarithmic in the
system size. The opposite instead does not hold true, and our results imply
that there exist quantum states that are not efficiently expressible in terms
of matrix product states or practically usable PEPS, but that are instead
efficiently expressible with neural network states
Continuously Growing Rodent Molars Result from a Predictable Quantitative Evolutionary Change over 50 Million Years
The fossil record is widely informative about evolution, but fossils are not systematically used to study the evolution of stem-cell-driven renewal. Here, we examined evolution of the continuous growth (hypselodonty) of rodent molar teeth, which is fuelled by the presence of dental stem cells. We studied occurrences of 3,500 North American rodent fossils, ranging from 50 million years ago (mya) to 2 mya. We examined changes in molar height to determine whether evolution of hypselodonty shows distinct patterns in the fossil record, and we found that hypselodont taxa emerged through intermediate forms of increasing crown height. Next, we designed a Markov simulation model, which replicated molar height increases throughout the Cenozoic and, moreover, evolution of hypselodonty. Thus, by extension, the retention of the adult stem cell niche appears to be a predictable quantitative rather than a stochastic qualitative process. Our analyses predict that hypselodonty will eventually become the dominant phenotype.Peer reviewe