Tight bounds on the maximum size of a set of permutations with bounded VC-dimension

The VC-dimension of a family P of n-permutations is the largest integer k such that the set of restrictions of the permutations in P on some k-tuple of positions is the set of all k! permutation patterns. Let r_k(n) be the maximum size of a set of n-permutations with VC-dimension k. Raz showed that r_2(n) grows exponentially in n. We show that r_3(n)=2^Theta(n log(alpha(n))) and for every s >= 4, we have almost tight upper and lower bounds of the form 2^{n poly(alpha(n))}. We also study the maximum number p_k(n) of 1-entries in an n x n (0,1)-matrix with no (k+1)-tuple of columns containing all (k+1)-permutation matrices. We determine that p_3(n) = Theta(n alpha(n)) and that p_s(n) can be bounded by functions of the form n 2^poly(alpha(n)) for every fixed s >= 4. We also show that for every positive s there is a slowly growing function zeta_s(m) (of the form 2^poly(alpha(m)) for every fixed s >= 5) satisfying the following. For all positive integers n and B and every n x n (0,1)-matrix M with zeta_s(n)Bn 1-entries, the rows of M can be partitioned into s intervals so that at least B columns contain at least B 1-entries in each of the intervals.Comment: 22 pages, 4 figures, correction of the bound on r_3 in the abstract and other minor change

Set Systems and Families of Permutations with Small Traces

We study the maximum size of a set system on $n$ elements whose trace on any $b$ elements has size at most $k$. We show that if for some $b \ge i \ge 0$ the shatter function $f_R$ of a set system $([n],R)$ satisfies $f_R(b) < 2^i(b-i+1)$ then $|R| = O(n^i)$; this generalizes Sauer's Lemma on the size of set systems with bounded VC-dimension. We use this bound to delineate the main growth rates for the same problem on families of permutations, where the trace corresponds to the inclusion for permutations. This is related to a question of Raz on families of permutations with bounded VC-dimension that generalizes the Stanley-Wilf conjecture on permutations with excluded patterns

An Active Learning Algorithm for Ranking from Pairwise Preferences with an Almost Optimal Query Complexity

We study the problem of learning to rank from pairwise preferences, and solve a long-standing open problem that has led to development of many heuristics but no provable results for our particular problem. Given a set $V$ of $n$ elements, we wish to linearly order them given pairwise preference labels. A pairwise preference label is obtained as a response, typically from a human, to the question "which if preferred, u or v?$for two elements$u,v\in V$. We assume possible non-transitivity paradoxes which may arise naturally due to human mistakes or irrationality. The goal is to linearly order the elements from the most preferred to the least preferred, while disagreeing with as few pairwise preference labels as possible. Our performance is measured by two parameters: The loss and the query complexity (number of pairwise preference labels we obtain). This is a typical learning problem, with the exception that the space from which the pairwise preferences is drawn is finite, consisting of${n\choose 2}$ possibilities only. We present an active learning algorithm for this problem, with query bounds significantly beating general (non active) bounds for the same error guarantee, while almost achieving the information theoretical lower bound. Our main construct is a decomposition of the input s.t. (i) each block incurs high loss at optimum, and (ii) the optimal solution respecting the decomposition is not much worse than the true opt. The decomposition is done by adapting a recent result by Kenyon and Schudy for a related combinatorial optimization problem to the query efficient setting. We thus settle an open problem posed by learning-to-rank theoreticians and practitioners: What is a provably correct way to sample preference labels? To further show the power and practicality of our solution, we show how to use it in concert with an SVM relaxation.Comment: Fixed a tiny error in theorem 3.1 statemen

Shattering Thresholds for Random Systems of Sets, Words, and Permutations

This paper considers a problem that relates to the theories of covering arrays, permutation patterns, Vapnik-Chervonenkis (VC) classes, and probability thresholds. Specifically, we want to find the number of subsets of [n]:={1,2,....,n} we need to randomly select, in a certain probability space, so as to respectively "shatter" all t-subsets of [n]. Moving from subsets to words, we ask for the number of n-letter words on a q-letter alphabet that are needed to shatter all t-subwords of the q^n words of length n. Finally, we explore the number of random permutations of [n] needed to shatter (specializing to t=3), all length 3 permutation patterns in specified positions. We uncover a very sharp zero-one probability threshold for the emergence of such shattering; Talagrand's isoperimetric inequality in product spaces is used as a key tool.Comment: 25 page

Maximum size of reverse-free sets of permutations

Two words have a reverse if they have the same pair of distinct letters on the same pair of positions, but in reversed order. A set of words no two of which have a reverse is said to be reverse-free. Let F(n,k) be the maximum size of a reverse-free set of words from [n]^k where no letter repeats within a word. We show the following lower and upper bounds in the case n >= k: F(n,k) \in n^k k^{-k/2 + O(k/log k)}. As a consequence of the lower bound, a set of n-permutations each two having a reverse has size at most n^{n/2 + O(n/log n)}.Comment: 10 page

Pattern Recognition for Conditionally Independent Data

In this work we consider the task of relaxing the i.i.d assumption in pattern recognition (or classification), aiming to make existing learning algorithms applicable to a wider range of tasks. Pattern recognition is guessing a discrete label of some object based on a set of given examples (pairs of objects and labels). We consider the case of deterministically defined labels. Traditionally, this task is studied under the assumption that examples are independent and identically distributed. However, it turns out that many results of pattern recognition theory carry over a weaker assumption. Namely, under the assumption of conditional independence and identical distribution of objects, while the only assumption on the distribution of labels is that the rate of occurrence of each label should be above some positive threshold. We find a broad class of learning algorithms for which estimations of the probability of a classification error achieved under the classical i.i.d. assumption can be generalised to the similar estimates for the case of conditionally i.i.d. examples.Comment: parts of results published at ALT'04 and ICML'0