1,320 research outputs found
Regularized Laplacian Estimation and Fast Eigenvector Approximation
Recently, Mahoney and Orecchia demonstrated that popular diffusion-based
procedures to compute a quick \emph{approximation} to the first nontrivial
eigenvector of a data graph Laplacian \emph{exactly} solve certain regularized
Semi-Definite Programs (SDPs). In this paper, we extend that result by
providing a statistical interpretation of their approximation procedure. Our
interpretation will be analogous to the manner in which -regularized or
-regularized -regression (often called Ridge regression and
Lasso regression, respectively) can be interpreted in terms of a Gaussian prior
or a Laplace prior, respectively, on the coefficient vector of the regression
problem. Our framework will imply that the solutions to the Mahoney-Orecchia
regularized SDP can be interpreted as regularized estimates of the
pseudoinverse of the graph Laplacian. Conversely, it will imply that the
solution to this regularized estimation problem can be computed very quickly by
running, e.g., the fast diffusion-based PageRank procedure for computing an
approximation to the first nontrivial eigenvector of the graph Laplacian.
Empirical results are also provided to illustrate the manner in which
approximate eigenvector computation \emph{implicitly} performs statistical
regularization, relative to running the corresponding exact algorithm.Comment: 13 pages and 3 figures. A more detailed version of a paper appearing
in the 2011 NIPS Conferenc
Large-scale wave-front reconstruction for adaptive optics systems by use of a recursive filtering algorithm
We propose a new recursive filtering algorithm for wave-front reconstruction in a large-scale adaptive optics system. An embedding step is used in this recursive filtering algorithm to permit fast methods to be used for wave-front reconstruction on an annular aperture. This embedding step can be used alone with a direct residual error updating procedure or used with the preconditioned conjugate-gradient method as a preconditioning step. We derive the Hudgin and Fried filters for spectral-domain filtering, using the eigenvalue decomposition method. Using Monte Carlo simulations, we compare the performance of discrete Fourier transform domain filtering, discrete cosine transform domain filtering, multigrid, and alternative-direction-implicit methods in the embedding step of the recursive filtering algorithm. We also simulate the performance of this recursive filtering in a closed-loop adaptive optics system
Approximate Computation and Implicit Regularization for Very Large-scale Data Analysis
Database theory and database practice are typically the domain of computer
scientists who adopt what may be termed an algorithmic perspective on their
data. This perspective is very different than the more statistical perspective
adopted by statisticians, scientific computers, machine learners, and other who
work on what may be broadly termed statistical data analysis. In this article,
I will address fundamental aspects of this algorithmic-statistical disconnect,
with an eye to bridging the gap between these two very different approaches. A
concept that lies at the heart of this disconnect is that of statistical
regularization, a notion that has to do with how robust is the output of an
algorithm to the noise properties of the input data. Although it is nearly
completely absent from computer science, which historically has taken the input
data as given and modeled algorithms discretely, regularization in one form or
another is central to nearly every application domain that applies algorithms
to noisy data. By using several case studies, I will illustrate, both
theoretically and empirically, the nonobvious fact that approximate
computation, in and of itself, can implicitly lead to statistical
regularization. This and other recent work suggests that, by exploiting in a
more principled way the statistical properties implicit in worst-case
algorithms, one can in many cases satisfy the bicriteria of having algorithms
that are scalable to very large-scale databases and that also have good
inferential or predictive properties.Comment: To appear in the Proceedings of the 2012 ACM Symposium on Principles
of Database Systems (PODS 2012
Graph Spectral Image Processing
Recent advent of graph signal processing (GSP) has spurred intensive studies
of signals that live naturally on irregular data kernels described by graphs
(e.g., social networks, wireless sensor networks). Though a digital image
contains pixels that reside on a regularly sampled 2D grid, if one can design
an appropriate underlying graph connecting pixels with weights that reflect the
image structure, then one can interpret the image (or image patch) as a signal
on a graph, and apply GSP tools for processing and analysis of the signal in
graph spectral domain. In this article, we overview recent graph spectral
techniques in GSP specifically for image / video processing. The topics covered
include image compression, image restoration, image filtering and image
segmentation
Estimating the inverse trace using random forests on graphs
Some data analysis problems require the computation of (regularised) inverse
traces, i.e. quantities of the form \Tr (q \bI + \bL)^{-1}. For large
matrices, direct methods are unfeasible and one must resort to approximations,
for example using a conjugate gradient solver combined with Girard's trace
estimator (also known as Hutchinson's trace estimator). Here we describe an
unbiased estimator of the regularized inverse trace, based on Wilson's
algorithm, an algorithm that was initially designed to draw uniform spanning
trees in graphs. Our method is fast, easy to implement, and scales to very
large matrices. Its main drawback is that it is limited to diagonally dominant
matrices \bL.Comment: Submitted to GRETSI conferenc
- …