1,001 research outputs found
Matrix Completion via Max-Norm Constrained Optimization
Matrix completion has been well studied under the uniform sampling model and
the trace-norm regularized methods perform well both theoretically and
numerically in such a setting. However, the uniform sampling model is
unrealistic for a range of applications and the standard trace-norm relaxation
can behave very poorly when the underlying sampling scheme is non-uniform.
In this paper we propose and analyze a max-norm constrained empirical risk
minimization method for noisy matrix completion under a general sampling model.
The optimal rate of convergence is established under the Frobenius norm loss in
the context of approximately low-rank matrix reconstruction. It is shown that
the max-norm constrained method is minimax rate-optimal and yields a unified
and robust approximate recovery guarantee, with respect to the sampling
distributions. The computational effectiveness of this method is also
discussed, based on first-order algorithms for solving convex optimizations
involving max-norm regularization.Comment: 33 page
A Max-Norm Constrained Minimization Approach to 1-Bit Matrix Completion
We consider in this paper the problem of noisy 1-bit matrix completion under
a general non-uniform sampling distribution using the max-norm as a convex
relaxation for the rank. A max-norm constrained maximum likelihood estimate is
introduced and studied. The rate of convergence for the estimate is obtained.
Information-theoretical methods are used to establish a minimax lower bound
under the general sampling model. The minimax upper and lower bounds together
yield the optimal rate of convergence for the Frobenius norm loss.
Computational algorithms and numerical performance are also discussed.Comment: 33 pages, 3 figure
ENO-wavelet transforms for piecewise smooth functions
We have designed an adaptive essentially nonoscillatory (ENO)-wavelet transform for approximating discontinuous functions without oscillations near the discontinuities. Our approach is to apply the main idea from ENO schemes for numerical shock capturing to standard wavelet transforms. The crucial point is that the wavelet coefficients are computed without differencing function values across jumps. However, we accomplish this in a different way than in the standard ENO schemes. Whereas in the standard ENO schemes the stencils are adaptively chosen, in the ENO-wavelet transforms we adaptively change the function and use the same uniform stencils. The ENO-wavelet transform retains the essential properties and advantages of standard wavelet transforms such as concentrating the energy to the low frequencies, obtaining maximum accuracy, maintained up to the discontinuities, and having a multiresolution framework and fast algorithms, all without any edge artifacts. We have obtained a rigorous approximation error bound which shows that the error in the ENO-wavelet approximation depends only on the size of the derivative of the function away from the discontinuities. We will show some numerical examples to illustrate this error estimate
Asymptotic equivalence and adaptive estimation for robust nonparametric regression
Asymptotic equivalence theory developed in the literature so far are only for
bounded loss functions. This limits the potential applications of the theory
because many commonly used loss functions in statistical inference are
unbounded. In this paper we develop asymptotic equivalence results for robust
nonparametric regression with unbounded loss functions. The results imply that
all the Gaussian nonparametric regression procedures can be robustified in a
unified way. A key step in our equivalence argument is to bin the data and then
take the median of each bin. The asymptotic equivalence results have
significant practical implications. To illustrate the general principles of the
equivalence argument we consider two important nonparametric inference
problems: robust estimation of the regression function and the estimation of a
quadratic functional. In both cases easily implementable procedures are
constructed and are shown to enjoy simultaneously a high degree of robustness
and adaptivity. Other problems such as construction of confidence sets and
nonparametric hypothesis testing can be handled in a similar fashion.Comment: Published in at http://dx.doi.org/10.1214/08-AOS681 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Security in User- Assisted Communications
Today, companies called service providers enable communications and control the
related infrastructures. However, with increased computing power, advanced wireless
technologies and more standardized terminals, users in the future will be able to take
more control of communications. In this paper, we define and discuss a disruptive
communication model called User-Assisted Communications (UAC), which allows
users to assist other users to establish communications, and propose a method for
managing trust and security, which are the most challenging variables in UAC and
must be addressed before UAC can be implemented successfully. A Social Network
based Trust Establishment (SN-TE) is proposed for UAC implementation
Law of Log Determinant of Sample Covariance Matrix and Optimal Estimation of Differential Entropy for High-Dimensional Gaussian Distributions
Differential entropy and log determinant of the covariance matrix of a
multivariate Gaussian distribution have many applications in coding,
communications, signal processing and statistical inference. In this paper we
consider in the high dimensional setting optimal estimation of the differential
entropy and the log-determinant of the covariance matrix. We first establish a
central limit theorem for the log determinant of the sample covariance matrix
in the high dimensional setting where the dimension can grow with the
sample size . An estimator of the differential entropy and the log
determinant is then considered. Optimal rate of convergence is obtained. It is
shown that in the case the estimator is asymptotically
sharp minimax. The ultra-high dimensional setting where is also
discussed.Comment: 19 page
Robust nonparametric estimation via wavelet median regression
In this paper we develop a nonparametric regression method that is
simultaneously adaptive over a wide range of function classes for the
regression function and robust over a large collection of error distributions,
including those that are heavy-tailed, and may not even possess variances or
means. Our approach is to first use local medians to turn the problem of
nonparametric regression with unknown noise distribution into a standard
Gaussian regression problem and then apply a wavelet block thresholding
procedure to construct an estimator of the regression function. It is shown
that the estimator simultaneously attains the optimal rate of convergence over
a wide range of the Besov classes, without prior knowledge of the smoothness of
the underlying functions or prior knowledge of the error distribution. The
estimator also automatically adapts to the local smoothness of the underlying
function, and attains the local adaptive minimax rate for estimating functions
at a point. A key technical result in our development is a quantile coupling
theorem which gives a tight bound for the quantile coupling between the sample
medians and a normal variable. This median coupling inequality may be of
independent interest.Comment: Published in at http://dx.doi.org/10.1214/07-AOS513 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Optimal rates of convergence for covariance matrix estimation
Covariance matrix plays a central role in multivariate statistical analysis.
Significant advances have been made recently on developing both theory and
methodology for estimating large covariance matrices. However, a minimax theory
has yet been developed. In this paper we establish the optimal rates of
convergence for estimating the covariance matrix under both the operator norm
and Frobenius norm. It is shown that optimal procedures under the two norms are
different and consequently matrix estimation under the operator norm is
fundamentally different from vector estimation. The minimax upper bound is
obtained by constructing a special class of tapering estimators and by studying
their risk properties. A key step in obtaining the optimal rate of convergence
is the derivation of the minimax lower bound. The technical analysis requires
new ideas that are quite different from those used in the more conventional
function/sequence estimation problems.Comment: Published in at http://dx.doi.org/10.1214/09-AOS752 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Nonparametric regression in exponential families
Most results in nonparametric regression theory are developed only for the
case of additive noise. In such a setting many smoothing techniques including
wavelet thresholding methods have been developed and shown to be highly
adaptive. In this paper we consider nonparametric regression in exponential
families with the main focus on the natural exponential families with a
quadratic variance function, which include, for example, Poisson regression,
binomial regression and gamma regression. We propose a unified approach of
using a mean-matching variance stabilizing transformation to turn the
relatively complicated problem of nonparametric regression in exponential
families into a standard homoscedastic Gaussian regression problem. Then in
principle any good nonparametric Gaussian regression procedure can be applied
to the transformed data. To illustrate our general methodology, in this paper
we use wavelet block thresholding to construct the final estimators of the
regression function. The procedures are easily implementable. Both theoretical
and numerical properties of the estimators are investigated. The estimators are
shown to enjoy a high degree of adaptivity and spatial adaptivity with
near-optimal asymptotic performance over a wide range of Besov spaces. The
estimators also perform well numerically.Comment: Published in at http://dx.doi.org/10.1214/09-AOS762 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
- …