43,284 research outputs found
Accelerating Random Kaczmarz Algorithm Based on Clustering Information
Kaczmarz algorithm is an efficient iterative algorithm to solve
overdetermined consistent system of linear equations. During each updating
step, Kaczmarz chooses a hyperplane based on an individual equation and
projects the current estimate for the exact solution onto that space to get a
new estimate. Many vairants of Kaczmarz algorithms are proposed on how to
choose better hyperplanes. Using the property of randomly sampled data in
high-dimensional space, we propose an accelerated algorithm based on clustering
information to improve block Kaczmarz and Kaczmarz via Johnson-Lindenstrauss
lemma. Additionally, we theoretically demonstrate convergence improvement on
block Kaczmarz algorithm
Global optimization algorithms for image registration and clustering
Global optimization is a classical problem of finding the minimum or maximum value of an objective function. It has applications in many areas, such as biological image analysis, chemistry, mechanical engineering, financial analysis, deep learning and image processing. For practical applications, it is important to understand the efficiency of global optimization algorithms. This dissertation develops and analyzes some new global optimization algorithms and applies them to practical problems, mainly for image registration and data clustering.
First, the dissertation presents a new global optimization algorithm which approximates the optimum using only function values. The basic idea is to use the points at which the function has been evaluated to decompose the domain into a collection of hyper-rectangles. At each step of the algorithm, it chooses a hyper-rectangle according to a certain criterion and the next function evaluation is at the center of the hyper-rectangle. The dissertation includes a proof that the algorithm converges to the global optimum as the number of function evaluations goes to infinity, and also establishes the convergence rate. Standard test functions are used to experimentally evaluate the algorithm.
The second part focuses on applying algorithms from the first part to solve some practical problems. Image processing tasks often require optimizing over some set of parameters. In the image registration problem, one attempts to determine the best transformation for aligning similar images. Such problems typically require minimizing a dissimilarity measure with multiple local minima. The dissertation describes a global optimization algorithm and applies it to the problem of identifying the best transformation for aligning two images.
Global optimization algorithms can also be applied to the data clustering problem. The basic purpose of clustering is to categorize data into different groups by their similarity. The objective cost functions for clustering usually are non-convex. -means is a popular algorithm which can find local optima quickly but may not obtain global optima. The different starting points for -means can output different local optima. This dissertation describes a global optimization algorithm for approximating the global minimum of the clustering problem.
The third part of the dissertation presents variations of the proposed algorithm that work with different assumptions on the available information, including a version that uses derivatives
Operator norm convergence of spectral clustering on level sets
Following Hartigan, a cluster is defined as a connected component of the
t-level set of the underlying density, i.e., the set of points for which the
density is greater than t. A clustering algorithm which combines a density
estimate with spectral clustering techniques is proposed. Our algorithm is
composed of two steps. First, a nonparametric density estimate is used to
extract the data points for which the estimated density takes a value greater
than t. Next, the extracted points are clustered based on the eigenvectors of a
graph Laplacian matrix. Under mild assumptions, we prove the almost sure
convergence in operator norm of the empirical graph Laplacian operator
associated with the algorithm. Furthermore, we give the typical behavior of the
representation of the dataset into the feature space, which establishes the
strong consistency of our proposed algorithm
Factor PD-Clustering
Factorial clustering methods have been developed in recent years thanks to
the improving of computational power. These methods perform a linear
transformation of data and a clustering on transformed data optimizing a common
criterion. Factorial PD-clustering is based on Probabilistic Distance
clustering (PD-clustering). PD-clustering is an iterative, distribution free,
probabilistic, clustering method. Factor PD-clustering make a linear
transformation of original variables into a reduced number of orthogonal ones
using a common criterion with PD-Clustering. It is demonstrated that Tucker 3
decomposition allows to obtain this transformation. Factor PD-clustering makes
alternatively a Tucker 3 decomposition and a PD-clustering on transformed data
until convergence. This method could significantly improve the algorithm
performance and allows to work with large dataset, to improve the stability and
the robustness of the method
Representation Learning for Clustering: A Statistical Framework
We address the problem of communicating domain knowledge from a user to the
designer of a clustering algorithm. We propose a protocol in which the user
provides a clustering of a relatively small random sample of a data set. The
algorithm designer then uses that sample to come up with a data representation
under which -means clustering results in a clustering (of the full data set)
that is aligned with the user's clustering. We provide a formal statistical
model for analyzing the sample complexity of learning a clustering
representation with this paradigm. We then introduce a notion of capacity of a
class of possible representations, in the spirit of the VC-dimension, showing
that classes of representations that have finite such dimension can be
successfully learned with sample size error bounds, and end our discussion with
an analysis of that dimension for classes of representations induced by linear
embeddings.Comment: To be published in Proceedings of UAI 201
- …