The shattering dimension of sets of linear functionals

We evaluate the shattering dimension of various classes of linear functionals on various symmetric convex sets. The proofs here relay mostly on methods from the local theory of normed spaces and include volume estimates, factorization techniques and tail estimates of norms, viewed as random variables on Euclidean spheres. The estimates of shattering dimensions can be applied to obtain error bounds for certain classes of functions, a fact which was the original motivation of this study. Although this can probably be done in a more traditional manner, we also use the approach presented here to determine whether several classes of linear functionals satisfy the uniform law of large numbers and the uniform central limit theorem.Comment: Published by the Institute of Mathematical Statistics (http://www.imstat.org) in the Annals of Probability (http://www.imstat.org/aop/) at http://dx.doi.org/10.1214/00911790400000038

Inapproximability of Truthful Mechanisms via Generalizations of the VC Dimension

Algorithmic mechanism design (AMD) studies the delicate interplay between computational efficiency, truthfulness, and optimality. We focus on AMD's paradigmatic problem: combinatorial auctions. We present a new generalization of the VC dimension to multivalued collections of functions, which encompasses the classical VC dimension, Natarajan dimension, and Steele dimension. We present a corresponding generalization of the Sauer-Shelah Lemma and harness this VC machinery to establish inapproximability results for deterministic truthful mechanisms. Our results essentially unify all inapproximability results for deterministic truthful mechanisms for combinatorial auctions to date and establish new separation gaps between truthful and non-truthful algorithms

Shattering Thresholds for Random Systems of Sets, Words, and Permutations

This paper considers a problem that relates to the theories of covering arrays, permutation patterns, Vapnik-Chervonenkis (VC) classes, and probability thresholds. Specifically, we want to find the number of subsets of [n]:={1,2,....,n} we need to randomly select, in a certain probability space, so as to respectively "shatter" all t-subsets of [n]. Moving from subsets to words, we ask for the number of n-letter words on a q-letter alphabet that are needed to shatter all t-subwords of the q^n words of length n. Finally, we explore the number of random permutations of [n] needed to shatter (specializing to t=3), all length 3 permutation patterns in specified positions. We uncover a very sharp zero-one probability threshold for the emergence of such shattering; Talagrand's isoperimetric inequality in product spaces is used as a key tool.Comment: 25 page

Entropy, dimension and the Elton-Pajor Theorem

The Vapnik-Chervonenkis dimension of a set K in R^n is the maximal dimension of the coordinate cube of a given size, which can be found in coordinate projections of K. We show that the VC dimension of a convex body governs its entropy. This has a number of consequences, including the optimal Elton's theorem and a uniform central limit theorem in the real valued case

VC-saturated set systems

The well-known Sauer lemma states that a family $\mathcal{F}\subseteq 2^{[n]}$ of VC-dimension at most $d$ has size at most $\sum_{i=0}^d\binom{n}{i}$. We obtain both random and explicit constructions to prove that the corresponding saturation number, i.e., the size of the smallest maximal family with VC-dimension $d\ge 2$, is at most $4^{d+1}$, and thus is independent of $n$

Expected Worst Case Regret via Stochastic Sequential Covering

We study the problem of sequential prediction and online minimax regret with stochastically generated features under a general loss function. We introduce a notion of expected worst case minimax regret that generalizes and encompasses prior known minimax regrets. For such minimax regrets we establish tight upper bounds via a novel concept of stochastic global sequential covering. We show that for a hypothesis class of VC-dimension $\mathsf{VC}$ and $i.i.d.$ generated features of length $T$, the cardinality of the stochastic global sequential covering can be upper bounded with high probability (whp) by $e^{O(\mathsf{VC} \cdot \log^2 T)}$. We then improve this bound by introducing a new complexity measure called the Star-Littlestone dimension, and show that classes with Star-Littlestone dimension $\mathsf{SL}$ admit a stochastic global sequential covering of order $e^{O(\mathsf{SL} \cdot \log T)}$. We further establish upper bounds for real valued classes with finite fat-shattering numbers. Finally, by applying information-theoretic tools of the fixed design minimax regrets, we provide lower bounds for the expected worst case minimax regret. We demonstrate the effectiveness of our approach by establishing tight bounds on the expected worst case minimax regrets for logarithmic loss and general mixable losses.Comment: Fixed hyperlink line break proble