891 research outputs found
On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models
Deep neural networks (DNNs) lack the precise semantics and definitive
probabilistic interpretation of probabilistic graphical models (PGMs). In this
paper, we propose an innovative solution by constructing infinite
tree-structured PGMs that correspond exactly to neural networks. Our research
reveals that DNNs, during forward propagation, indeed perform approximations of
PGM inference that are precise in this alternative PGM structure. Not only does
our research complement existing studies that describe neural networks as
kernel machines or infinite-sized Gaussian processes, it also elucidates a more
direct approximation that DNNs make to exact inference in PGMs. Potential
benefits include improved pedagogy and interpretation of DNNs, and algorithms
that can merge the strengths of PGMs and DNNs
Scalable Data Augmentation for Deep Learning
Scalable Data Augmentation (SDA) provides a framework for training deep
learning models using auxiliary hidden layers. Scalable MCMC is available for
network training and inference. SDA provides a number of computational
advantages over traditional algorithms, such as avoiding backtracking, local
modes and can perform optimization with stochastic gradient descent (SGD) in
TensorFlow. Standard deep neural networks with logit, ReLU and SVM activation
functions are straightforward to implement. To illustrate our architectures and
methodology, we use P\'{o}lya-Gamma logit data augmentation for a number of
standard datasets. Finally, we conclude with directions for future research
- …