10,972 research outputs found
Weighted Polynomial Approximations: Limits for Learning and Pseudorandomness
Polynomial approximations to boolean functions have led to many positive
results in computer science. In particular, polynomial approximations to the
sign function underly algorithms for agnostically learning halfspaces, as well
as pseudorandom generators for halfspaces. In this work, we investigate the
limits of these techniques by proving inapproximability results for the sign
function.
Firstly, the polynomial regression algorithm of Kalai et al. (SIAM J. Comput.
2008) shows that halfspaces can be learned with respect to log-concave
distributions on in the challenging agnostic learning model. The
power of this algorithm relies on the fact that under log-concave
distributions, halfspaces can be approximated arbitrarily well by low-degree
polynomials. We ask whether this technique can be extended beyond log-concave
distributions, and establish a negative result. We show that polynomials of any
degree cannot approximate the sign function to within arbitrarily low error for
a large class of non-log-concave distributions on the real line, including
those with densities proportional to .
Secondly, we investigate the derandomization of Chernoff-type concentration
inequalities. Chernoff-type tail bounds on sums of independent random variables
have pervasive applications in theoretical computer science. Schmidt et al.
(SIAM J. Discrete Math. 1995) showed that these inequalities can be established
for sums of random variables with only -wise independence,
for a tail probability of . We show that their results are tight up to
constant factors.
These results rely on techniques from weighted approximation theory, which
studies how well functions on the real line can be approximated by polynomials
under various distributions. We believe that these techniques will have further
applications in other areas of computer science.Comment: 22 page
A sequential sampling strategy for extreme event statistics in nonlinear dynamical systems
We develop a method for the evaluation of extreme event statistics associated
with nonlinear dynamical systems, using a small number of samples. From an
initial dataset of design points, we formulate a sequential strategy that
provides the 'next-best' data point (set of parameters) that when evaluated
results in improved estimates of the probability density function (pdf) for a
scalar quantity of interest. The approach utilizes Gaussian process regression
to perform Bayesian inference on the parameter-to-observation map describing
the quantity of interest. We then approximate the desired pdf along with
uncertainty bounds utilizing the posterior distribution of the inferred map.
The 'next-best' design point is sequentially determined through an optimization
procedure that selects the point in parameter space that maximally reduces
uncertainty between the estimated bounds of the pdf prediction. Since the
optimization process utilizes only information from the inferred map it has
minimal computational cost. Moreover, the special form of the metric emphasizes
the tails of the pdf. The method is practical for systems where the
dimensionality of the parameter space is of moderate size, i.e. order O(10). We
apply the method to estimate the extreme event statistics for a very
high-dimensional system with millions of degrees of freedom: an offshore
platform subjected to three-dimensional irregular waves. It is demonstrated
that the developed approach can accurately determine the extreme event
statistics using limited number of samples
Uniform Sampling for Matrix Approximation
Random sampling has become a critical tool in solving massive matrix
problems. For linear regression, a small, manageable set of data rows can be
randomly selected to approximate a tall, skinny data matrix, improving
processing time significantly. For theoretical performance guarantees, each row
must be sampled with probability proportional to its statistical leverage
score. Unfortunately, leverage scores are difficult to compute.
A simple alternative is to sample rows uniformly at random. While this often
works, uniform sampling will eliminate critical row information for many
natural instances. We take a fresh look at uniform sampling by examining what
information it does preserve. Specifically, we show that uniform sampling
yields a matrix that, in some sense, well approximates a large fraction of the
original. While this weak form of approximation is not enough for solving
linear regression directly, it is enough to compute a better approximation.
This observation leads to simple iterative row sampling algorithms for matrix
approximation that run in input-sparsity time and preserve row structure and
sparsity at all intermediate steps. In addition to an improved understanding of
uniform sampling, our main proof introduces a structural result of independent
interest: we show that every matrix can be made to have low coherence by
reweighting a small subset of its rows
- …