9,755 research outputs found
A Survey on Bayesian Deep Learning
A comprehensive artificial intelligence system needs to not only perceive the
environment with different `senses' (e.g., seeing and hearing) but also infer
the world's conditional (or even causal) relations and corresponding
uncertainty. The past decade has seen major advances in many perception tasks
such as visual object recognition and speech recognition using deep learning
models. For higher-level inference, however, probabilistic graphical models
with their Bayesian nature are still more powerful and flexible. In recent
years, Bayesian deep learning has emerged as a unified probabilistic framework
to tightly integrate deep learning and Bayesian models. In this general
framework, the perception of text or images using deep learning can boost the
performance of higher-level inference and in turn, the feedback from the
inference process is able to enhance the perception of text or images. This
survey provides a comprehensive introduction to Bayesian deep learning and
reviews its recent applications on recommender systems, topic models, control,
etc. Besides, we also discuss the relationship and differences between Bayesian
deep learning and other related topics such as Bayesian treatment of neural
networks.Comment: To appear in ACM Computing Surveys (CSUR) 202
Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes
A method is provided for designing and training noise-driven recurrent neural
networks as models of stochastic processes. The method unifies and generalizes
two known separate modeling approaches, Echo State Networks (ESN) and Linear
Inverse Modeling (LIM), under the common principle of relative entropy
minimization. The power of the new method is demonstrated on a stochastic
approximation of the El Nino phenomenon studied in climate research
Discovering Functional Communities in Dynamical Networks
Many networks are important because they are substrates for dynamical
systems, and their pattern of functional connectivity can itself be dynamic --
they can functionally reorganize, even if their underlying anatomical structure
remains fixed. However, the recent rapid progress in discovering the community
structure of networks has overwhelmingly focused on that constant anatomical
connectivity. In this paper, we lay out the problem of discovering_functional
communities_, and describe an approach to doing so. This method combines recent
work on measuring information sharing across stochastic networks with an
existing and successful community-discovery algorithm for weighted networks. We
illustrate it with an application to a large biophysical model of the
transition from beta to gamma rhythms in the hippocampus.Comment: 18 pages, 4 figures, Springer "Lecture Notes in Computer Science"
style. Forthcoming in the proceedings of the workshop "Statistical Network
Analysis: Models, Issues and New Directions", at ICML 2006. Version 2: small
clarifications, typo corrections, added referenc
- …