1,776 research outputs found
Deep Temporal Clustering: Fully Unsupervised Learning of Time-Domain Features
abstract: Unsupervised learning of time series data, also known as temporal clustering, is a challenging problem in machine learning. This thesis presents a novel algorithm, Deep Temporal Clustering (DTC), to naturally integrate dimensionality reduction and temporal clustering into a single end-to-end learning framework, fully unsupervised. The algorithm utilizes an autoencoder for temporal dimensionality reduction and a novel temporal clustering layer for cluster assignment. Then it jointly optimizes the clustering objective and the dimensionality reduction objective. Based on requirement and application, the temporal clustering layer can be customized with any temporal similarity metric. Several similarity metrics and state-of-the-art algorithms are considered and compared. To gain insight into temporal features that the network has learned for its clustering, a visualization method is applied that generates a region of interest heatmap for the time series. The viability of the algorithm is demonstrated using time series data from diverse domains, ranging from earthquakes to spacecraft sensor data. In each case, the proposed algorithm outperforms traditional methods. The superior performance is attributed to the fully integrated temporal dimensionality reduction and clustering criterion.Dissertation/ThesisMasters Thesis Computer Engineering 201
Modern Machine Learning for LHC Physicists
Modern machine learning is transforming particle physics, faster than we can
follow, and bullying its way into our numerical tool box. For young researchers
it is crucial to stay on top of this development, which means applying
cutting-edge methods and tools to the full range of LHC physics problems. These
lecture notes are meant to lead students with basic knowledge of particle
physics and significant enthusiasm for machine learning to relevant
applications as fast as possible. They start with an LHC-specific motivation
and a non-standard introduction to neural networks and then cover
classification, unsupervised classification, generative networks, and inverse
problems. Two themes defining much of the discussion are well-defined loss
functions reflecting the problem at hand and uncertainty-aware networks. As
part of the applications, the notes include some aspects of theoretical LHC
physics. All examples are chosen from particle physics publications of the last
few years. Given that these notes will be outdated already at the time of
submission, the week of ML4Jets 2022, they will be updated frequently.Comment: First version, we very much appreciate feedbac
A Survey on Metric Learning for Feature Vectors and Structured Data
The need for appropriate ways to measure the distance or similarity between
data is ubiquitous in machine learning, pattern recognition and data mining,
but handcrafting such good metrics for specific problems is generally
difficult. This has led to the emergence of metric learning, which aims at
automatically learning a metric from data and has attracted a lot of interest
in machine learning and related fields for the past ten years. This survey
paper proposes a systematic review of the metric learning literature,
highlighting the pros and cons of each approach. We pay particular attention to
Mahalanobis distance metric learning, a well-studied and successful framework,
but additionally present a wide range of methods that have recently emerged as
powerful alternatives, including nonlinear metric learning, similarity learning
and local metric learning. Recent trends and extensions, such as
semi-supervised metric learning, metric learning for histogram data and the
derivation of generalization guarantees, are also covered. Finally, this survey
addresses metric learning for structured data, in particular edit distance
learning, and attempts to give an overview of the remaining challenges in
metric learning for the years to come.Comment: Technical report, 59 pages. Changes in v2: fixed typos and improved
presentation. Changes in v3: fixed typos. Changes in v4: fixed typos and new
method
Discriminating different classes of biological networks by analyzing the graphs spectra distribution
The brain's structural and functional systems, protein-protein interaction,
and gene networks are examples of biological systems that share some features
of complex networks, such as highly connected nodes, modularity, and
small-world topology. Recent studies indicate that some pathologies present
topological network alterations relative to norms seen in the general
population. Therefore, methods to discriminate the processes that generate the
different classes of networks (e.g., normal and disease) might be crucial for
the diagnosis, prognosis, and treatment of the disease. It is known that
several topological properties of a network (graph) can be described by the
distribution of the spectrum of its adjacency matrix. Moreover, large networks
generated by the same random process have the same spectrum distribution,
allowing us to use it as a "fingerprint". Based on this relationship, we
introduce and propose the entropy of a graph spectrum to measure the
"uncertainty" of a random graph and the Kullback-Leibler and Jensen-Shannon
divergences between graph spectra to compare networks. We also introduce
general methods for model selection and network model parameter estimation, as
well as a statistical procedure to test the nullity of divergence between two
classes of complex networks. Finally, we demonstrate the usefulness of the
proposed methods by applying them on (1) protein-protein interaction networks
of different species and (2) on networks derived from children diagnosed with
Attention Deficit Hyperactivity Disorder (ADHD) and typically developing
children. We conclude that scale-free networks best describe all the
protein-protein interactions. Also, we show that our proposed measures
succeeded in the identification of topological changes in the network while
other commonly used measures (number of edges, clustering coefficient, average
path length) failed
Towards Effective Codebookless Model for Image Classification
The bag-of-features (BoF) model for image classification has been thoroughly
studied over the last decade. Different from the widely used BoF methods which
modeled images with a pre-trained codebook, the alternative codebook free image
modeling method, which we call Codebookless Model (CLM), attracted little
attention. In this paper, we present an effective CLM that represents an image
with a single Gaussian for classification. By embedding Gaussian manifold into
a vector space, we show that the simple incorporation of our CLM into a linear
classifier achieves very competitive accuracy compared with state-of-the-art
BoF methods (e.g., Fisher Vector). Since our CLM lies in a high dimensional
Riemannian manifold, we further propose a joint learning method of low-rank
transformation with support vector machine (SVM) classifier on the Gaussian
manifold, in order to reduce computational and storage cost. To study and
alleviate the side effect of background clutter on our CLM, we also present a
simple yet effective partial background removal method based on saliency
detection. Experiments are extensively conducted on eight widely used databases
to demonstrate the effectiveness and efficiency of our CLM method
- …