109 research outputs found
Subdomain Adaptation with Manifolds Discrepancy Alignment
Reducing domain divergence is a key step in transfer learning problems.
Existing works focus on the minimization of global domain divergence. However,
two domains may consist of several shared subdomains, and differ from each
other in each subdomain. In this paper, we take the local divergence of
subdomains into account in transfer. Specifically, we propose to use
low-dimensional manifold to represent subdomain, and align the local data
distribution discrepancy in each manifold across domains. A Manifold Maximum
Mean Discrepancy (M3D) is developed to measure the local distribution
discrepancy in each manifold. We then propose a general framework, called
Transfer with Manifolds Discrepancy Alignment (TMDA), to couple the discovery
of data manifolds with the minimization of M3D. We instantiate TMDA in the
subspace learning case considering both the linear and nonlinear mappings. We
also instantiate TMDA in the deep learning framework. Extensive experimental
studies demonstrate that TMDA is a promising method for various transfer
learning tasks
Deep Clustering: A Comprehensive Survey
Cluster analysis plays an indispensable role in machine learning and data
mining. Learning a good data representation is crucial for clustering
algorithms. Recently, deep clustering, which can learn clustering-friendly
representations using deep neural networks, has been broadly applied in a wide
range of clustering tasks. Existing surveys for deep clustering mainly focus on
the single-view fields and the network architectures, ignoring the complex
application scenarios of clustering. To address this issue, in this paper we
provide a comprehensive survey for deep clustering in views of data sources.
With different data sources and initial conditions, we systematically
distinguish the clustering methods in terms of methodology, prior knowledge,
and architecture. Concretely, deep clustering methods are introduced according
to four categories, i.e., traditional single-view deep clustering,
semi-supervised deep clustering, deep multi-view clustering, and deep transfer
clustering. Finally, we discuss the open challenges and potential future
opportunities in different fields of deep clustering
Transfer Learning with Mixtures of Manifolds
Advances in scientific instrumentation technology have increased the speed of data acquisition and the precision of sampling, creating an abundance of high-dimensional data sets. The ability to combine these disparate data sets and to transfer information between them is critical to accurate scientific analysis. Many modern-day instruments can record data at many thousands of channels, far greater than the actual degrees of freedom in the sample data. This makes manifold learning, a class of methods that exploit the observation that high-dimensional data tend to lie on lower-dimensional manifolds, especially well-suited to this transfer learning task.
Existing manifold-based transfer learning methods can align related data sets in differing feature representations, but their inherent single manifold assumption causes them to fail in the presence of complex mixtures of manifolds. In this dissertation, a new class of transfer learning algorithms is developed for high-dimensional data sets that intrinsically lie on multiple low-dimensional manifolds. With a more realistic mixture of manifolds assumption, this class of algorithms allows for accurate and efficient transfer of information between data sets by aligning their complex underlying geometries.
In this dissertation, algorithms are presented that leverage corresponding samples between data sets and available label information, continuous or categorical. The two primary tasks are aligning mixtures of manifolds and heterogeneous domain adaptation of multi-manifold data sets. Linear, non-linear, and robust versions of the algorithm are described, as well as a method for actively selecting cross-data set correspondences. To show the practical effectiveness of these algorithms, they are compared across a number of synthetic and real-world domains, but most notably to align data recorded by spectroscopic instruments during space exploration, a new domain for transfer learning
Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing
Hyperspectral imaging, also known as image spectrometry, is a landmark
technique in geoscience and remote sensing (RS). In the past decade, enormous
efforts have been made to process and analyze these hyperspectral (HS) products
mainly by means of seasoned experts. However, with the ever-growing volume of
data, the bulk of costs in manpower and material resources poses new challenges
on reducing the burden of manual labor and improving efficiency. For this
reason, it is, therefore, urgent to develop more intelligent and automatic
approaches for various HS RS applications. Machine learning (ML) tools with
convex optimization have successfully undertaken the tasks of numerous
artificial intelligence (AI)-related applications. However, their ability in
handling complex practical problems remains limited, particularly for HS data,
due to the effects of various spectral variabilities in the process of HS
imaging and the complexity and redundancy of higher dimensional HS signals.
Compared to the convex models, non-convex modeling, which is capable of
characterizing more complex real scenes and providing the model
interpretability technically and theoretically, has been proven to be a
feasible solution to reduce the gap between challenging HS vision tasks and
currently advanced intelligent data processing models
Continual learning from stationary and non-stationary data
Continual learning aims at developing models that are capable of working on constantly evolving problems over a long-time horizon. In such environments, we can distinguish three essential aspects of training and maintaining machine learning models - incorporating new knowledge, retaining it and reacting to changes. Each of them poses its own challenges, constituting a compound problem with multiple goals.
Remembering previously incorporated concepts is the main property of a model that is required when dealing with stationary distributions. In non-stationary environments, models should be capable of selectively forgetting outdated decision boundaries and adapting to new concepts. Finally, a significant difficulty can be found in combining these two abilities within a single learning algorithm, since, in such scenarios, we have to balance remembering and forgetting instead of focusing only on one aspect.
The presented dissertation addressed these problems in an exploratory way. Its main goal was to grasp the continual learning paradigm as a whole, analyze its different branches and tackle identified issues covering various aspects of learning from sequentially incoming data. By doing so, this work not only filled several gaps in the current continual learning research but also emphasized the complexity and diversity of challenges existing in this domain. Comprehensive experiments conducted for all of the presented contributions have demonstrated their effectiveness and substantiated the validity of the stated claims
- …