724 research outputs found
From average case complexity to improper learning complexity
The basic problem in the PAC model of computational learning theory is to
determine which hypothesis classes are efficiently learnable. There is
presently a dearth of results showing hardness of learning problems. Moreover,
the existing lower bounds fall short of the best known algorithms.
The biggest challenge in proving complexity results is to establish hardness
of {\em improper learning} (a.k.a. representation independent learning).The
difficulty in proving lower bounds for improper learning is that the standard
reductions from -hard problems do not seem to apply in this
context. There is essentially only one known approach to proving lower bounds
on improper learning. It was initiated in (Kearns and Valiant 89) and relies on
cryptographic assumptions.
We introduce a new technique for proving hardness of improper learning, based
on reductions from problems that are hard on average. We put forward a (fairly
strong) generalization of Feige's assumption (Feige 02) about the complexity of
refuting random constraint satisfaction problems. Combining this assumption
with our new technique yields far reaching implications. In particular,
1. Learning 's is hard.
2. Agnostically learning halfspaces with a constant approximation ratio is
hard.
3. Learning an intersection of halfspaces is hard.Comment: 34 page
Parameterized bounded-depth Frege is not optimal
A general framework for parameterized proof complexity was introduced by Dantchev, Martin, and Szeider [9]. There the authors concentrate on tree-like Parameterized Resolution-a parameterized version of classical Resolution-and their gap complexity theorem implies lower bounds for that system. The main result of the present paper significantly improves upon this by showing optimal lower bounds for a parameterized version of bounded-depth Frege. More precisely, we prove that the pigeonhole principle requires proofs of size n in parameterized bounded-depth Frege, and, as a special case, in dag-like Parameterized Resolution. This answers an open question posed in [9]. In the opposite direction, we interpret a well-known technique for FPT algorithms as a DPLL procedure for Parameterized Resolution. Its generalization leads to a proof search algorithm for Parameterized Resolution that in particular shows that tree-like Parameterized Resolution allows short refutations of all parameterized contradictions given as bounded-width CNF's
Understanding Space in Proof Complexity: Separations and Trade-offs via Substitutions
For current state-of-the-art DPLL SAT-solvers the two main bottlenecks are
the amounts of time and memory used. In proof complexity, these resources
correspond to the length and space of resolution proofs. There has been a long
line of research investigating these proof complexity measures, but while
strong results have been established for length, our understanding of space and
how it relates to length has remained quite poor. In particular, the question
whether resolution proofs can be optimized for length and space simultaneously,
or whether there are trade-offs between these two measures, has remained
essentially open.
In this paper, we remedy this situation by proving a host of length-space
trade-off results for resolution. Our collection of trade-offs cover almost the
whole range of values for the space complexity of formulas, and most of the
trade-offs are superpolynomial or even exponential and essentially tight. Using
similar techniques, we show that these trade-offs in fact extend to the
exponentially stronger k-DNF resolution proof systems, which operate with
formulas in disjunctive normal form with terms of bounded arity k. We also
answer the open question whether the k-DNF resolution systems form a strict
hierarchy with respect to space in the affirmative.
Our key technical contribution is the following, somewhat surprising,
theorem: Any CNF formula F can be transformed by simple variable substitution
into a new formula F' such that if F has the right properties, F' can be proven
in essentially the same length as F, whereas on the other hand the minimal
number of lines one needs to keep in memory simultaneously in any proof of F'
is lower-bounded by the minimal number of variables needed simultaneously in
any proof of F. Applying this theorem to so-called pebbling formulas defined in
terms of pebble games on directed acyclic graphs, we obtain our results.Comment: This paper is a merged and updated version of the two ECCC technical
reports TR09-034 and TR09-047, and it hence subsumes these two report
Space complexity in polynomial calculus
During the last decade, an active line of research in proof complexity has been to study space
complexity and time-space trade-offs for proofs. Besides being a natural complexity measure of
intrinsic interest, space is also an important issue in SAT solving, and so research has mostly focused
on weak systems that are used by SAT solvers.
There has been a relatively long sequence of papers on space in resolution, which is now reasonably
well understood from this point of view. For other natural candidates to study, however, such as
polynomial calculus or cutting planes, very little has been known. We are not aware of any nontrivial
space lower bounds for cutting planes, and for polynomial calculus the only lower bound has been
for CNF formulas of unbounded width in [Alekhnovich et al. ’02], where the space lower bound is
smaller than the initial width of the clauses in the formulas. Thus, in particular, it has been consistent
with current knowledge that polynomial calculus could be able to refute any k-CNF formula in
constant space.
In this paper, we prove several new results on space in polynomial calculus (PC), and in the
extended proof system polynomial calculus resolution (PCR) studied in [Alekhnovich et al. ’02]:
1. We prove an Ω(n) space lower bound in PC for the canonical 3-CNF version of the pigeonhole
principle formulas PHPm
n with m pigeons and n holes, and show that this is tight.
2. For PCR, we prove an Ω(n) space lower bound for a bitwise encoding of the functional pigeonhole
principle. These formulas have width O(log n), and hence this is an exponential
improvement over [Alekhnovich et al. ’02] measured in the width of the formulas.
3. We then present another encoding of the pigeonhole principle that has constant width, and
prove an Ω(n) space lower bound in PCR for these formulas as well.
4. Finally, we prove that any k-CNF formula can be refuted in PC in simultaneous exponential
size and linear space (which holds for resolution and thus for PCR, but was not obviously
the case for PC). We also characterize a natural class of CNF formulas for which the space
complexity in resolution and PCR does not change when the formula is transformed into 3-CNF
in the canonical way, something that we believe can be useful when proving PCR space lower
bounds for other well-studied formula families in proof complexity
- …