4,341 research outputs found
The chaining lemma and its application
We present a new information-theoretic result which we call the Chaining Lemma. It considers a so-called “chain” of random variables, defined by a source distribution X(0)with high min-entropy and a number (say, t in total) of arbitrary functions (T1,…, Tt) which are applied in succession to that source to generate the chain (Formula presented). Intuitively, the Chaining Lemma guarantees that, if the chain is not too long, then either (i) the entire chain is “highly random”, in that every variable has high min-entropy; or (ii) it is possible to find a point j (1 ≤ j ≤ t) in the chain such that, conditioned on the end of the chain i.e. (Formula presented), the preceding part (Formula presented) remains highly random. We think this is an interesting information-theoretic result which is intuitive but nevertheless requires rigorous case-analysis to prove. We believe that the above lemma will find applications in cryptography. We give an example of this, namely we show an application of the lemma to protect essentially any cryptographic scheme against memory tampering attacks. We allow several tampering requests, the tampering functions can be arbitrary, however, they must be chosen from a bounded size set of functions that is fixed a prior
Tail bounds via generic chaining
We modify Talagrand's generic chaining method to obtain upper bounds for all
p-th moments of the supremum of a stochastic process. These bounds lead to an
estimate for the upper tail of the supremum with optimal deviation parameters.
We apply our procedure to improve and extend some known deviation inequalities
for suprema of unbounded empirical processes and chaos processes. As an
application we give a significantly simplified proof of the restricted isometry
property of the subsampled discrete Fourier transform.Comment: Added detailed proof of Theorem 3.5; Application to dimensionality
reduction expanded and moved to separate note arXiv:1402.397
Algorithmic linear dimension reduction in the l_1 norm for sparse vectors
This paper develops a new method for recovering m-sparse signals that is
simultaneously uniform and quick. We present a reconstruction algorithm whose
run time, O(m log^2(m) log^2(d)), is sublinear in the length d of the signal.
The reconstruction error is within a logarithmic factor (in m) of the optimal
m-term approximation error in l_1. In particular, the algorithm recovers
m-sparse signals perfectly and noisy signals are recovered with polylogarithmic
distortion. Our algorithm makes O(m log^2 (d)) measurements, which is within a
logarithmic factor of optimal. We also present a small-space implementation of
the algorithm. These sketching techniques and the corresponding reconstruction
algorithms provide an algorithmic dimension reduction in the l_1 norm. In
particular, vectors of support m in dimension d can be linearly embedded into
O(m log^2 d) dimensions with polylogarithmic distortion. We can reconstruct a
vector from its low-dimensional sketch in time O(m log^2(m) log^2(d)).
Furthermore, this reconstruction is stable and robust under small
perturbations
Error Bounds for Piecewise Smooth and Switching Regression
The paper deals with regression problems, in which the nonsmooth target is
assumed to switch between different operating modes. Specifically, piecewise
smooth (PWS) regression considers target functions switching deterministically
via a partition of the input space, while switching regression considers
arbitrary switching laws. The paper derives generalization error bounds in
these two settings by following the approach based on Rademacher complexities.
For PWS regression, our derivation involves a chaining argument and a
decomposition of the covering numbers of PWS classes in terms of the ones of
their component functions and the capacity of the classifier partitioning the
input space. This yields error bounds with a radical dependency on the number
of modes. For switching regression, the decomposition can be performed directly
at the level of the Rademacher complexities, which yields bounds with a linear
dependency on the number of modes. By using once more chaining and a
decomposition at the level of covering numbers, we show how to recover a
radical dependency. Examples of applications are given in particular for PWS
and swichting regression with linear and kernel-based component functions.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice,after which this version may no
longer be accessibl
Chaining, Interpolation, and Convexity
We show that classical chaining bounds on the suprema of random processes in
terms of entropy numbers can be systematically improved when the underlying set
is convex: the entropy numbers need not be computed for the entire set, but
only for certain "thin" subsets. This phenomenon arises from the observation
that real interpolation can be used as a natural chaining mechanism. Unlike the
general form of Talagrand's generic chaining method, which is sharp but often
difficult to use, the resulting bounds involve only entropy numbers but are
nonetheless sharp in many situations in which classical entropy bounds are
suboptimal. Such bounds are readily amenable to explicit computations in
specific examples, and we discover some old and new geometric principles for
the control of chaining functionals as special cases.Comment: 21 pages; final version, to appear in J. Eur. Math. So
Small Deviation Probability via Chaining
We obtain several extensions of Talagrand's lower bound for the small
deviation probability using metric entropy. For Gaussian processes, our
investigations are focused on processes with sub-polynomial and, respectively,
exponential behaviour of covering numbers. The corresponding results are also
proved for non-Gaussian symmetric stable processes, both for the cases of
critically small and critically large entropy. The results extensively use the
classical chaining technique; at the same time they are meant to explore the
limits of this method.Comment: to appear in: Stochastic Processes and Their Application
Discrepancy, chaining and subgaussian processes
We show that for a typical coordinate projection of a subgaussian class of
functions, the infimum over signs is asymptotically smaller than the
expectation over signs as a function of the dimension , if the canonical
Gaussian process indexed by is continuous. To that end, we establish a
bound on the discrepancy of an arbitrary subset of using
properties of the canonical Gaussian process the set indexes, and then obtain
quantitative structural information on a typical coordinate projection of a
subgaussian class.Comment: Published in at http://dx.doi.org/10.1214/10-AOP575 the Annals of
Probability (http://www.imstat.org/aop/) by the Institute of Mathematical
Statistics (http://www.imstat.org
- …