8,822 research outputs found
An Identity for Kernel Ridge Regression
This paper derives an identity connecting the square loss of ridge regression
in on-line mode with the loss of the retrospectively best regressor. Some
corollaries about the properties of the cumulative loss of on-line ridge
regression are also obtained.Comment: 35 pages; extended version of ALT 2010 paper (Proceedings of ALT
2010, LNCS 6331, Springer, 2010
Adaptive Mantel Test for AssociationTesting in Imaging Genetics Data
Mantel's test (MT) for association is conducted by testing the linear
relationship of similarity of all pairs of subjects between two observational
domains. Motivated by applications to neuroimaging and genetics data, and
following the succes of shrinkage and kernel methods for prediction with
high-dimensional data, we here introduce the adaptive Mantel test as an
extension of the MT. By utilizing kernels and penalized similarity measures,
the adaptive Mantel test is able to achieve higher statistical power relative
to the classical MT in many settings. Furthermore, the adaptive Mantel test is
designed to simultaneously test over multiple similarity measures such that the
correct type I error rate under the null hypothesis is maintained without the
need to directly adjust the significance threshold for multiple testing. The
performance of the adaptive Mantel test is evaluated on simulated data, and is
used to investigate associations between genetics markers related to
Alzheimer's Disease and heatlhy brain physiology with data from a working
memory study of 350 college students from Beijing Normal University
Spectral Norm of Random Kernel Matrices with Applications to Privacy
Kernel methods are an extremely popular set of techniques used for many
important machine learning and data analysis applications. In addition to
having good practical performances, these methods are supported by a
well-developed theory. Kernel methods use an implicit mapping of the input data
into a high dimensional feature space defined by a kernel function, i.e., a
function returning the inner product between the images of two data points in
the feature space. Central to any kernel method is the kernel matrix, which is
built by evaluating the kernel function on a given sample dataset.
In this paper, we initiate the study of non-asymptotic spectral theory of
random kernel matrices. These are n x n random matrices whose (i,j)th entry is
obtained by evaluating the kernel function on and , where
are a set of n independent random high-dimensional vectors. Our
main contribution is to obtain tight upper bounds on the spectral norm (largest
eigenvalue) of random kernel matrices constructed by commonly used kernel
functions based on polynomials and Gaussian radial basis.
As an application of these results, we provide lower bounds on the distortion
needed for releasing the coefficients of kernel ridge regression under
attribute privacy, a general privacy notion which captures a large class of
privacy definitions. Kernel ridge regression is standard method for performing
non-parametric regression that regularly outperforms traditional regression
approaches in various domains. Our privacy distortion lower bounds are the
first for any kernel technique, and our analysis assumes realistic scenarios
for the input, unlike all previous lower bounds for other release problems
which only hold under very restrictive input settings.Comment: 16 pages, 1 Figur
- …