3,705 research outputs found
On the Equivalence Between Deep NADE and Generative Stochastic Networks
Neural Autoregressive Distribution Estimators (NADEs) have recently been
shown as successful alternatives for modeling high dimensional multimodal
distributions. One issue associated with NADEs is that they rely on a
particular order of factorization for . This issue has been
recently addressed by a variant of NADE called Orderless NADEs and its deeper
version, Deep Orderless NADE. Orderless NADEs are trained based on a criterion
that stochastically maximizes with all possible orders of
factorizations. Unfortunately, ancestral sampling from deep NADE is very
expensive, corresponding to running through a neural net separately predicting
each of the visible variables given some others. This work makes a connection
between this criterion and the training criterion for Generative Stochastic
Networks (GSNs). It shows that training NADEs in this way also trains a GSN,
which defines a Markov chain associated with the NADE model. Based on this
connection, we show an alternative way to sample from a trained Orderless NADE
that allows to trade-off computing time and quality of the samples: a 3 to
10-fold speedup (taking into account the waste due to correlations between
consecutive samples of the chain) can be obtained without noticeably reducing
the quality of the samples. This is achieved using a novel sampling procedure
for GSNs called annealed GSN sampling, similar to tempering methods that
combines fast mixing (obtained thanks to steps at high noise levels) with
accurate samples (obtained thanks to steps at low noise levels).Comment: ECML/PKDD 201
Generative Models For Deep Learning with Very Scarce Data
The goal of this paper is to deal with a data scarcity scenario where deep
learning techniques use to fail. We compare the use of two well established
techniques, Restricted Boltzmann Machines and Variational Auto-encoders, as
generative models in order to increase the training set in a classification
framework. Essentially, we rely on Markov Chain Monte Carlo (MCMC) algorithms
for generating new samples. We show that generalization can be improved
comparing this methodology to other state-of-the-art techniques, e.g.
semi-supervised learning with ladder networks. Furthermore, we show that RBM is
better than VAE generating new samples for training a classifier with good
generalization capabilities
Who Said What: Modeling Individual Labelers Improves Classification
Data are often labeled by many different experts with each expert only
labeling a small fraction of the data and each data point being labeled by
several experts. This reduces the workload on individual experts and also gives
a better estimate of the unobserved ground truth. When experts disagree, the
standard approaches are to treat the majority opinion as the correct label or
to model the correct label as a distribution. These approaches, however, do not
make any use of potentially valuable information about which expert produced
which label. To make use of this extra information, we propose modeling the
experts individually and then learning averaging weights for combining them,
possibly in sample-specific ways. This allows us to give more weight to more
reliable experts and take advantage of the unique strengths of individual
experts at classifying certain types of data. Here we show that our approach
leads to improvements in computer-aided diagnosis of diabetic retinopathy. We
also show that our method performs better than competing algorithms by Welinder
and Perona (2010), and by Mnih and Hinton (2012). Our work offers an innovative
approach for dealing with the myriad real-world settings that use expert
opinions to define labels for training.Comment: AAAI 201
Inducing Language Networks from Continuous Space Word Representations
Recent advancements in unsupervised feature learning have developed powerful
latent representations of words. However, it is still not clear what makes one
representation better than another and how we can learn the ideal
representation. Understanding the structure of latent spaces attained is key to
any future advancement in unsupervised learning. In this work, we introduce a
new view of continuous space word representations as language networks. We
explore two techniques to create language networks from learned features by
inducing them for two popular word representation methods and examining the
properties of their resulting networks. We find that the induced networks
differ from other methods of creating language networks, and that they contain
meaningful community structure.Comment: 14 page
The Monoceros very-high-energy gamma-ray source
The H.E.S.S. telescope array has observed the complex Monoceros Loop
SNR/Rosette Nebula region which contains unidentified high energy EGRET sources
and potential very-high-energy (VHE) gamma-ray source. We announce the
discovery of a new point-like VHE gamma-ray sources, HESS J0632+057. It is
located close to the rim of the Monoceros SNR and has no clear counterpart at
other wavelengths. Data from the NANTEN telescope have been used to investigate
hadronic interactions with nearby molecular clouds. We found no evidence for a
clear association. The VHE gamma-ray emission is possibly associated with the
lower energy gamma-ray source 3EG J0634+0521, a weak X-ray source 1RXS
J063258.3+054857 and the Be-star MWC 148.Comment: 4 pages, 4 figures, Contribution to the 30th ICRC, Merida Mexico,
July 200
Learning Temporal Transformations From Time-Lapse Videos
Based on life-long observations of physical, chemical, and biologic phenomena
in the natural world, humans can often easily picture in their minds what an
object will look like in the future. But, what about computers? In this paper,
we learn computational models of object transformations from time-lapse videos.
In particular, we explore the use of generative models to create depictions of
objects at future times. These models explore several different prediction
tasks: generating a future state given a single depiction of an object,
generating a future state given two depictions of an object at different times,
and generating future states recursively in a recurrent framework. We provide
both qualitative and quantitative evaluations of the generated results, and
also conduct a human evaluation to compare variations of our models.Comment: ECCV201
Deep Over-sampling Framework for Classifying Imbalanced Data
Class imbalance is a challenging issue in practical classification problems
for deep learning models as well as traditional models. Traditionally
successful countermeasures such as synthetic over-sampling have had limited
success with complex, structured data handled by deep learning models. In this
paper, we propose Deep Over-sampling (DOS), a framework for extending the
synthetic over-sampling method to exploit the deep feature space acquired by a
convolutional neural network (CNN). Its key feature is an explicit, supervised
representation learning, for which the training data presents each raw input
sample with a synthetic embedding target in the deep feature space, which is
sampled from the linear subspace of in-class neighbors. We implement an
iterative process of training the CNN and updating the targets, which induces
smaller in-class variance among the embeddings, to increase the discriminative
power of the deep representation. We present an empirical study using public
benchmarks, which shows that the DOS framework not only counteracts class
imbalance better than the existing method, but also improves the performance of
the CNN in the standard, balanced settings
A potential anti-corrosive ionic liquid coating for MG alloy AZ31 in simulated body fluids
Magnesium alloys are attractive materials for biomedical applications, due to their excellent biocompatibility. However, these alloys show fast corrosion rates in the body that limits their clinical applications. Low-toxic ionic liquid (IL) trimethyl(butyl)phosphonium diphenyl phosphate P1444dpp has been investigated to provide corrosion protection for magnesium alloy AZ31 in simulated body fluids (SBFs). This work reports a preliminary exploration of the influence of different treatment temperatures on the corrosion protection properties of IL films for the magnesium alloy AZ31 in SBFs. Results show that the IL treatment at room temperature did not bring significant improvement in the corrosion performance of the AZ31 in SBF. However, when the treatment temperature was increased to 75°C, the IL treatment resulted in a substantial reduction of the corrosion, in particular the reduction of localized pitting corrosion. The influence of ionic liquid treatment on the corrosion performance of the magnesium alloys AZ31 in SBFs has been investigated by electrochemical impedance spectroscopy (EIS) tests and immersion tests
Content Based Image Retrieval by Convolutional Neural Networks
Hamreras S., Benítez-Rochel R., Boucheham B., Molina-Cabello M.A., López-Rubio E. (2019) Content Based Image Retrieval by Convolutional Neural Networks. In: Ferrández Vicente J., Álvarez-Sánchez J., de la Paz López F., Toledo Moreo J., Adeli H. (eds) From Bioinspired Systems and Biomedical Applications to Machine Learning. IWINAC 2019. Lecture Notes in Computer Science, vol 11487. Springer.In this paper, we present a Convolutional Neural Network (CNN) for feature extraction in Content based Image Retrieval (CBIR). The proposed CNN aims at reducing the semantic gap between low level and high-level features. Thus, improving retrieval results. Our CNN is the result of a transfer learning technique using Alexnet pretrained network. It learns how to extract representative features from a learning database and then uses this knowledge in query feature extraction. Experimentations performed on Wang (Corel 1K) database show a significant improvement in terms of precision over the state of the art classic approaches.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech
ShapeCodes: Self-Supervised Feature Learning by Lifting Views to Viewgrids
We introduce an unsupervised feature learning approach that embeds 3D shape
information into a single-view image representation. The main idea is a
self-supervised training objective that, given only a single 2D image, requires
all unseen views of the object to be predictable from learned features. We
implement this idea as an encoder-decoder convolutional neural network. The
network maps an input image of an unknown category and unknown viewpoint to a
latent space, from which a deconvolutional decoder can best "lift" the image to
its complete viewgrid showing the object from all viewing angles. Our
class-agnostic training procedure encourages the representation to capture
fundamental shape primitives and semantic regularities in a data-driven
manner---without manual semantic labels. Our results on two widely-used shape
datasets show 1) our approach successfully learns to perform "mental rotation"
even for objects unseen during training, and 2) the learned latent space is a
powerful representation for object recognition, outperforming several existing
unsupervised feature learning methods.Comment: To appear at ECCV 201
- …