8,915 research outputs found
Implicitly Constrained Semi-Supervised Least Squares Classification
We introduce a novel semi-supervised version of the least squares classifier.
This implicitly constrained least squares (ICLS) classifier minimizes the
squared loss on the labeled data among the set of parameters implied by all
possible labelings of the unlabeled data. Unlike other discriminative
semi-supervised methods, our approach does not introduce explicit additional
assumptions into the objective function, but leverages implicit assumptions
already present in the choice of the supervised least squares classifier. We
show this approach can be formulated as a quadratic programming problem and its
solution can be found using a simple gradient descent procedure. We prove that,
in a certain way, our method never leads to performance worse than the
supervised classifier. Experimental results corroborate this theoretical result
in the multidimensional case on benchmark datasets, also in terms of the error
rate.Comment: 12 pages, 2 figures, 1 table. The Fourteenth International Symposium
on Intelligent Data Analysis (2015), Saint-Etienne, Franc
Implicitly Constrained Semi-Supervised Linear Discriminant Analysis
Semi-supervised learning is an important and active topic of research in
pattern recognition. For classification using linear discriminant analysis
specifically, several semi-supervised variants have been proposed. Using any
one of these methods is not guaranteed to outperform the supervised classifier
which does not take the additional unlabeled data into account. In this work we
compare traditional Expectation Maximization type approaches for
semi-supervised linear discriminant analysis with approaches based on intrinsic
constraints and propose a new principled approach for semi-supervised linear
discriminant analysis, using so-called implicit constraints. We explore the
relationships between these methods and consider the question if and in what
sense we can expect improvement in performance over the supervised procedure.
The constraint based approaches are more robust to misspecification of the
model, and may outperform alternatives that make more assumptions on the data,
in terms of the log-likelihood of unseen objects.Comment: 6 pages, 3 figures and 3 tables. International Conference on Pattern
Recognition (ICPR) 2014, Stockholm, Swede
Manifold Constrained Low-Rank Decomposition
Low-rank decomposition (LRD) is a state-of-the-art method for visual data
reconstruction and modelling. However, it is a very challenging problem when
the image data contains significant occlusion, noise, illumination variation,
and misalignment from rotation or viewpoint changes. We leverage the specific
structure of data in order to improve the performance of LRD when the data are
not ideal. To this end, we propose a new framework that embeds manifold priors
into LRD. To implement the framework, we design an alternating direction method
of multipliers (ADMM) method which efficiently integrates the manifold
constraints during the optimization process. The proposed approach is
successfully used to calculate low-rank models from face images, hand-written
digits and planar surface images. The results show a consistent increase of
performance when compared to the state-of-the-art over a wide range of
realistic image misalignments and corruptions
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Generative Adversarial Networks (GANs) is a novel class of deep generative
models which has recently gained significant attention. GANs learns complex and
high-dimensional distributions implicitly over images, audio, and data.
However, there exists major challenges in training of GANs, i.e., mode
collapse, non-convergence and instability, due to inappropriate design of
network architecture, use of objective function and selection of optimization
algorithm. Recently, to address these challenges, several solutions for better
design and optimization of GANs have been investigated based on techniques of
re-engineered network architectures, new objective functions and alternative
optimization algorithms. To the best of our knowledge, there is no existing
survey that has particularly focused on broad and systematic developments of
these solutions. In this study, we perform a comprehensive survey of the
advancements in GANs design and optimization solutions proposed to handle GANs
challenges. We first identify key research issues within each design and
optimization technique and then propose a new taxonomy to structure solutions
by key research issues. In accordance with the taxonomy, we provide a detailed
discussion on different GANs variants proposed within each solution and their
relationships. Finally, based on the insights gained, we present the promising
research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table
- …