335 research outputs found
Modeling Documents with Deep Boltzmann Machines
We introduce a Deep Boltzmann Machine model suitable for modeling and
extracting latent semantic representations from a large unstructured collection
of documents. We overcome the apparent difficulty of training a DBM with
judicious parameter tying. This parameter tying enables an efficient
pretraining algorithm and a state initialization scheme that aids inference.
The model can be trained just as efficiently as a standard Restricted Boltzmann
Machine. Our experiments show that the model assigns better log probability to
unseen data than the Replicated Softmax model. Features extracted from our
model outperform LDA, Replicated Softmax, and DocNADE models on document
retrieval and document classification tasks.Comment: Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty
in Artificial Intelligence (UAI2013
Joint Visual Denoising and Classification using Deep Learning
Visual restoration and recognition are traditionally addressed in pipeline
fashion, i.e. denoising followed by classification. Instead, observing
correlations between the two tasks, for example clearer image will lead to
better categorization and vice visa, we propose a joint framework for visual
restoration and recognition for handwritten images, inspired by advances in
deep autoencoder and multi-modality learning. Our model is a 3-pathway deep
architecture with a hidden-layer representation which is shared by multi-inputs
and outputs, and each branch can be composed of a multi-layer deep model. Thus,
visual restoration and classification can be unified using shared
representation via non-linear mapping, and model parameters can be learnt via
backpropagation. Using MNIST and USPS data corrupted with structured noise, the
proposed framework performs at least 20\% better in classification than
separate pipelines, as well as clearer recovered images. The noise model and
the reproducible source code is available at
{\url{https://github.com/ganggit/jointmodel}}.Comment: 5 pages, 7 figures, ICIP 201
- …