42 research outputs found
Cortical region interactions and the functional role of apical dendrites
The basal and distal apical dendrites of pyramidal cells occupy distinct
cortical layers and are targeted by axons originating in different cortical
regions. Hence, apical and basal dendrites receive information from distinct
sources. Physiological evidence suggests that this anatomically observed
segregation of input sources may have functional significance. This possibility
has been explored in various connectionist models that employ neurons with
functionally distinct apical and basal compartments. A neuron in which separate
sets of inputs can be integrated independently has the potential to operate in a
variety of ways which are not possible for the conventional model of a neuron in
which all inputs are treated equally. This article thus considers how
functionally distinct apical and basal dendrites can contribute to the
information processing capacities of single neurons and, in particular, how
information from different cortical regions could have disparate affects on
neural activity and learning
Piecewise Latent Variables for Neural Variational Text Processing
Advances in neural variational inference have facilitated the learning of
powerful directed graphical models with continuous latent variables, such as
variational autoencoders. The hope is that such models will learn to represent
rich, multi-modal latent factors in real-world data, such as natural language
text. However, current models often assume simplistic priors on the latent
variables - such as the uni-modal Gaussian distribution - which are incapable
of representing complex latent factors efficiently. To overcome this
restriction, we propose the simple, but highly flexible, piecewise constant
distribution. This distribution has the capacity to represent an exponential
number of modes of a latent target distribution, while remaining mathematically
tractable. Our results demonstrate that incorporating this new latent
distribution into different models yields substantial improvements in natural
language processing tasks such as document modeling and natural language
generation for dialogue.Comment: 19 pages, 2 figures, 8 tables; EMNLP 201