1,402 research outputs found
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Generative Adversarial Networks (GANs) is a novel class of deep generative
models which has recently gained significant attention. GANs learns complex and
high-dimensional distributions implicitly over images, audio, and data.
However, there exists major challenges in training of GANs, i.e., mode
collapse, non-convergence and instability, due to inappropriate design of
network architecture, use of objective function and selection of optimization
algorithm. Recently, to address these challenges, several solutions for better
design and optimization of GANs have been investigated based on techniques of
re-engineered network architectures, new objective functions and alternative
optimization algorithms. To the best of our knowledge, there is no existing
survey that has particularly focused on broad and systematic developments of
these solutions. In this study, we perform a comprehensive survey of the
advancements in GANs design and optimization solutions proposed to handle GANs
challenges. We first identify key research issues within each design and
optimization technique and then propose a new taxonomy to structure solutions
by key research issues. In accordance with the taxonomy, we provide a detailed
discussion on different GANs variants proposed within each solution and their
relationships. Finally, based on the insights gained, we present the promising
research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table
Deep Clustering: A Comprehensive Survey
Cluster analysis plays an indispensable role in machine learning and data
mining. Learning a good data representation is crucial for clustering
algorithms. Recently, deep clustering, which can learn clustering-friendly
representations using deep neural networks, has been broadly applied in a wide
range of clustering tasks. Existing surveys for deep clustering mainly focus on
the single-view fields and the network architectures, ignoring the complex
application scenarios of clustering. To address this issue, in this paper we
provide a comprehensive survey for deep clustering in views of data sources.
With different data sources and initial conditions, we systematically
distinguish the clustering methods in terms of methodology, prior knowledge,
and architecture. Concretely, deep clustering methods are introduced according
to four categories, i.e., traditional single-view deep clustering,
semi-supervised deep clustering, deep multi-view clustering, and deep transfer
clustering. Finally, we discuss the open challenges and potential future
opportunities in different fields of deep clustering
Variational Clustering: Leveraging Variational Autoencoders for Image Clustering
Recent advances in deep learning have shown their ability to learn strong
feature representations for images. The task of image clustering naturally
requires good feature representations to capture the distribution of the data
and subsequently differentiate data points from one another. Often these two
aspects are dealt with independently and thus traditional feature learning
alone does not suffice in partitioning the data meaningfully. Variational
Autoencoders (VAEs) naturally lend themselves to learning data distributions in
a latent space. Since we wish to efficiently discriminate between different
clusters in the data, we propose a method based on VAEs where we use a Gaussian
Mixture prior to help cluster the images accurately. We jointly learn the
parameters of both the prior and the posterior distributions. Our method
represents a true Gaussian Mixture VAE. This way, our method simultaneously
learns a prior that captures the latent distribution of the images and a
posterior to help discriminate well between data points. We also propose a
novel reparametrization of the latent space consisting of a mixture of discrete
and continuous variables. One key takeaway is that our method generalizes
better across different datasets without using any pre-training or learnt
models, unlike existing methods, allowing it to be trained from scratch in an
end-to-end manner. We verify our efficacy and generalizability experimentally
by achieving state-of-the-art results among unsupervised methods on a variety
of datasets. To the best of our knowledge, we are the first to pursue image
clustering using VAEs in a purely unsupervised manner on real image datasets
Survey of deep representation learning for speech emotion recognition
Traditionally, speech emotion recognition (SER) research has relied on manually handcrafted acoustic features using feature engineering. However, the design of handcrafted features for complex SER tasks requires significant manual eort, which impedes generalisability and slows the pace of innovation. This has motivated the adoption of representation learning techniques that can automatically learn an intermediate representation of the input signal without any manual feature engineering. Representation learning has led to improved SER performance and enabled rapid innovation. Its effectiveness has further increased with advances in deep learning (DL), which has facilitated \textit{deep representation learning} where hierarchical representations are automatically learned in a data-driven manner. This paper presents the first comprehensive survey on the important topic of deep representation learning for SER. We highlight various techniques, related challenges and identify important future areas of research. Our survey bridges the gap in the literature since existing surveys either focus on SER with hand-engineered features or representation learning in the general setting without focusing on SER
- …