12,610 research outputs found
Deep Complex Networks
At present, the vast majority of building blocks, techniques, and
architectures for deep learning are based on real-valued operations and
representations. However, recent work on recurrent neural networks and older
fundamental theoretical analysis suggests that complex numbers could have a
richer representational capacity and could also facilitate noise-robust memory
retrieval mechanisms. Despite their attractive properties and potential for
opening up entirely new neural architectures, complex-valued deep neural
networks have been marginalized due to the absence of the building blocks
required to design such models. In this work, we provide the key atomic
components for complex-valued deep neural networks and apply them to
convolutional feed-forward networks and convolutional LSTMs. More precisely, we
rely on complex convolutions and present algorithms for complex
batch-normalization, complex weight initialization strategies for
complex-valued neural nets and we use them in experiments with end-to-end
training schemes. We demonstrate that such complex-valued models are
competitive with their real-valued counterparts. We test deep complex models on
several computer vision tasks, on music transcription using the MusicNet
dataset and on Speech Spectrum Prediction using the TIMIT dataset. We achieve
state-of-the-art performance on these audio-related tasks
Modeling Quantum Mechanical Observers via Neural-Glial Networks
We investigate the theory of observers in the quantum mechanical world by
using a novel model of the human brain which incorporates the glial network
into the Hopfield model of the neural network. Our model is based on a
microscopic construction of a quantum Hamiltonian of the synaptic junctions.
Using the Eguchi-Kawai large N reduction, we show that, when the number of
neurons and astrocytes is exponentially large, the degrees of freedom of the
dynamics of the neural and glial networks can be completely removed and,
consequently, that the retention time of the superposition of the wave
functions in the brain is as long as that of the microscopic quantum system of
pre-synaptics sites. Based on this model, the classical information entropy of
the neural-glial network is introduced. Using this quantity, we propose a
criterion for the brain to be a quantum mechanical observer.Comment: 24 pages, published versio
- …