12,113 research outputs found
A survey on trajectory clustering analysis
This paper comprehensively surveys the development of trajectory clustering.
Considering the critical role of trajectory data mining in modern intelligent
systems for surveillance security, abnormal behavior detection, crowd behavior
analysis, and traffic control, trajectory clustering has attracted growing
attention. Existing trajectory clustering methods can be grouped into three
categories: unsupervised, supervised and semi-supervised algorithms. In spite
of achieving a certain level of development, trajectory clustering is limited
in its success by complex conditions such as application scenarios and data
dimensions. This paper provides a holistic understanding and deep insight into
trajectory clustering, and presents a comprehensive analysis of representative
methods and promising future directions
Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization
Image clustering is one of the most important computer vision applications,
which has been extensively studied in literature. However, current clustering
methods mostly suffer from lack of efficiency and scalability when dealing with
large-scale and high-dimensional data. In this paper, we propose a new
clustering model, called DEeP Embedded RegularIzed ClusTering (DEPICT), which
efficiently maps data into a discriminative embedding subspace and precisely
predicts cluster assignments. DEPICT generally consists of a multinomial
logistic regression function stacked on top of a multi-layer convolutional
autoencoder. We define a clustering objective function using relative entropy
(KL divergence) minimization, regularized by a prior for the frequency of
cluster assignments. An alternating strategy is then derived to optimize the
objective by updating parameters and estimating cluster assignments.
Furthermore, we employ the reconstruction loss functions in our autoencoder, as
a data-dependent regularization term, to prevent the deep embedding function
from overfitting. In order to benefit from end-to-end optimization and
eliminate the necessity for layer-wise pretraining, we introduce a joint
learning framework to minimize the unified clustering and reconstruction loss
functions together and train all network layers simultaneously. Experimental
results indicate the superiority and faster running time of DEPICT in
real-world clustering tasks, where no labeled data is available for
hyper-parameter tuning
A Survey on Multi-Task Learning
Multi-Task Learning (MTL) is a learning paradigm in machine learning and its
aim is to leverage useful information contained in multiple related tasks to
help improve the generalization performance of all the tasks. In this paper, we
give a survey for MTL. First, we classify different MTL algorithms into several
categories, including feature learning approach, low-rank approach, task
clustering approach, task relation learning approach, and decomposition
approach, and then discuss the characteristics of each approach. In order to
improve the performance of learning tasks further, MTL can be combined with
other learning paradigms including semi-supervised learning, active learning,
unsupervised learning, reinforcement learning, multi-view learning and
graphical models. When the number of tasks is large or the data dimensionality
is high, batch MTL models are difficult to handle this situation and online,
parallel and distributed MTL models as well as dimensionality reduction and
feature hashing are reviewed to reveal their computational and storage
advantages. Many real-world applications use MTL to boost their performance and
we review representative works. Finally, we present theoretical analyses and
discuss several future directions for MTL
Transfer Adaptation Learning: A Decade Survey
The world we see is ever-changing and it always changes with people, things,
and the environment. Domain is referred to as the state of the world at a
certain moment. A research problem is characterized as transfer adaptation
learning (TAL) when it needs knowledge correspondence between different
moments/domains. Conventional machine learning aims to find a model with the
minimum expected risk on test data by minimizing the regularized empirical risk
on the training data, which, however, supposes that the training and test data
share similar joint probability distribution. TAL aims to build models that can
perform tasks of target domain by learning knowledge from a semantic related
but distribution different source domain. It is an energetic research filed of
increasing influence and importance, which is presenting a blowout publication
trend. This paper surveys the advances of TAL methodologies in the past decade,
and the technical challenges and essential problems of TAL have been observed
and discussed with deep insights and new perspectives. Broader solutions of
transfer adaptation learning being created by researchers are identified, i.e.,
instance re-weighting adaptation, feature adaptation, classifier adaptation,
deep network adaptation and adversarial adaptation, which are beyond the early
semi-supervised and unsupervised split. The survey helps researchers rapidly
but comprehensively understand and identify the research foundation, research
status, theoretical limitations, future challenges and under-studied issues
(universality, interpretability, and credibility) to be broken in the field
toward universal representation and safe applications in open-world scenarios.Comment: 26 pages, 4 figure
Semi-Supervised Graph Classification: A Hierarchical Graph Perspective
Node classification and graph classification are two graph learning problems
that predict the class label of a node and the class label of a graph
respectively. A node of a graph usually represents a real-world entity, e.g., a
user in a social network, or a protein in a protein-protein interaction
network. In this work, we consider a more challenging but practically useful
setting, in which a node itself is a graph instance. This leads to a
hierarchical graph perspective which arises in many domains such as social
network, biological network and document collection. For example, in a social
network, a group of people with shared interests forms a user group, whereas a
number of user groups are interconnected via interactions or common members. We
study the node classification problem in the hierarchical graph where a `node'
is a graph instance, e.g., a user group in the above example. As labels are
usually limited in real-world data, we design two novel semi-supervised
solutions named \underline{SE}mi-supervised gr\underline{A}ph
c\underline{L}assification via \underline{C}autious/\underline{A}ctive
\underline{I}teration (or SEAL-C/AI in short). SEAL-C/AI adopt an iterative
framework that takes turns to build or update two classifiers, one working at
the graph instance level and the other at the hierarchical graph level. To
simplify the representation of the hierarchical graph, we propose a novel
supervised, self-attentive graph embedding method called SAGE, which embeds
graph instances of arbitrary size into fixed-length vectors. Through
experiments on synthetic data and Tencent QQ group data, we demonstrate that
SEAL-C/AI not only outperform competing methods by a significant margin in
terms of accuracy/Macro-F1, but also generate meaningful interpretations of the
learned representations.Comment: 12 pages, WWW-201
Adaptive Image Stream Classification via Convolutional Neural Network with Intrinsic Similarity Metrics
When performing data classification over a stream of continuously occurring
instances, a key challenge is to develop an open-world classifier that
anticipates instances from an unknown class. Studies addressing this problem,
typically called novel class detection, have considered classification methods
that reactively adapt to such changes along the stream. Importantly, they rely
on the property of cohesion and separation among instances in feature space.
Instances belonging to the same class are assumed to be closer to each other
(cohesion) than those belonging to different classes (separation).
Unfortunately, this assumption may not have large support when dealing with
high dimensional data such as images. In this paper, we address this key
challenge by proposing a semisupervised multi-task learning framework called
CSIM which aims to intrinsically search for a latent space suitable for
detecting labels of instances from both known and unknown classes.
Particularly, we utilize a convolution neural network layer that aids in the
learning of a latent feature space suitable for novel class detection. We
empirically measure the performance of CSIM over multiple realworld image
datasets and demonstrate its superiority by comparing its performance with
existing semi-supervised methods.Comment: 10 pages; KDD'18 Deep Learning Day, August 2018, London, U
Low-rank Kernel Learning for Graph-based Clustering
Constructing the adjacency graph is fundamental to graph-based clustering.
Graph learning in kernel space has shown impressive performance on a number of
benchmark data sets. However, its performance is largely determined by the
chosen kernel matrix. To address this issue, the previous multiple kernel
learning algorithm has been applied to learn an optimal kernel from a group of
predefined kernels. This approach might be sensitive to noise and limits the
representation ability of the consensus kernel. In contrast to existing
methods, we propose to learn a low-rank kernel matrix which exploits the
similarity nature of the kernel matrix and seeks an optimal kernel from the
neighborhood of candidate kernels. By formulating graph construction and kernel
learning in a unified framework, the graph and consensus kernel can be
iteratively enhanced by each other. Extensive experimental results validate the
efficacy of the proposed method
Recent Advances in Autoencoder-Based Representation Learning
Learning useful representations with little or no supervision is a key
challenge in artificial intelligence. We provide an in-depth review of recent
advances in representation learning with a focus on autoencoder-based models.
To organize these results we make use of meta-priors believed useful for
downstream tasks, such as disentanglement and hierarchical organization of
features. In particular, we uncover three main mechanisms to enforce such
properties, namely (i) regularizing the (approximate or aggregate) posterior
distribution, (ii) factorizing the encoding and decoding distribution, or (iii)
introducing a structured prior distribution. While there are some promising
results, implicit or explicit supervision remains a key enabler and all current
methods use strong inductive biases and modeling assumptions. Finally, we
provide an analysis of autoencoder-based representation learning through the
lens of rate-distortion theory and identify a clear tradeoff between the amount
of prior knowledge available about the downstream tasks, and how useful the
representation is for this task.Comment: Presented at the third workshop on Bayesian Deep Learning (NeurIPS
2018
Ensemble p-Laplacian Regularization for Remote Sensing Image Recognition
Recently, manifold regularized semi-supervised learning (MRSSL) received
considerable attention because it successfully exploits the geometry of the
intrinsic data probability distribution including both labeled and unlabeled
samples to leverage the performance of a learning model. As a natural nonlinear
generalization of graph Laplacian, p-Laplacian has been proved having the rich
theoretical foundations to better preserve the local structure. However, it is
difficult to determine the fitting graph p-Lapalcian i.e. the parameter which
is a critical factor for the performance of graph p-Laplacian. Therefore, we
develop an ensemble p-Laplacian regularization (EpLapR) to fully approximate
the intrinsic manifold of the data distribution. EpLapR incorporates multiple
graphs into a regularization term in order to sufficiently explore the
complementation of graph p-Laplacian. Specifically, we construct a fused graph
by introducing an optimization approach to assign suitable weights on different
p-value graphs. And then, we conduct semi-supervised learning framework on the
fused graph. Extensive experiments on UC-Merced data set demonstrate the
effectiveness and efficiency of the proposed method.Comment: 13 pages, 7 figures. arXiv admin note: text overlap with
arXiv:1806.0810
A Survey of Heterogeneous Information Network Analysis
Most real systems consist of a large number of interacting, multi-typed
components, while most contemporary researches model them as homogeneous
networks, without distinguishing different types of objects and links in the
networks. Recently, more and more researchers begin to consider these
interconnected, multi-typed data as heterogeneous information networks, and
develop structural analysis approaches by leveraging the rich semantic meaning
of structural types of objects and links in the networks. Compared to widely
studied homogeneous network, the heterogeneous information network contains
richer structure and semantic information, which provides plenty of
opportunities as well as a lot of challenges for data mining. In this paper, we
provide a survey of heterogeneous information network analysis. We will
introduce basic concepts of heterogeneous information network analysis, examine
its developments on different data mining tasks, discuss some advanced topics,
and point out some future research directions.Comment: 45 pages, 12 figure
- …