30,803 research outputs found
Descriptive Complexity Approaches to Inductive Inference
We present a critical review of descriptive complexity approaches to inductive inference. Inductive inference is defined as any process by which a model of the world is formed from observations. The descriptive complexity approach is a formalization of Occam\u27s razor: choose the simplest model consistent with the data. Descriptive complexity as defined by Kolmogorov, Chaitin and Solomonoff is presented as a generalization of Shannon\u27s entropy. We discuss its relationship with randomness and present examples. However, a major result of the theory is negative: descriptive complexity is uncomputable.
Rissanen\u27s minimum description length (MDL) principle is presented as a restricted form of the descriptive complexity which avoids the uncomputability problem. We demonstrate the effectiveness of MDL through its application to AR processes. Lastly, we present and discuss LeClerc\u27s application of MDL to the problem of image segmentation
Frequentist statistics as a theory of inductive inference
After some general remarks about the interrelation between philosophical and
statistical thinking, the discussion centres largely on significance tests.
These are defined as the calculation of -values rather than as formal
procedures for ``acceptance'' and ``rejection.'' A number of types of null
hypothesis are described and a principle for evidential interpretation set out
governing the implications of -values in the specific circumstances of each
application, as contrasted with a long-run interpretation. A variety of more
complicated situations are discussed in which modification of the simple
-value may be essential.Comment: Published at http://dx.doi.org/10.1214/074921706000000400 in the IMS
Lecture Notes--Monograph Series
(http://www.imstat.org/publications/lecnotes.htm) by the Institute of
Mathematical Statistics (http://www.imstat.org
Space complexity in polynomial calculus
During the last decade, an active line of research in proof complexity has been to study space
complexity and time-space trade-offs for proofs. Besides being a natural complexity measure of
intrinsic interest, space is also an important issue in SAT solving, and so research has mostly focused
on weak systems that are used by SAT solvers.
There has been a relatively long sequence of papers on space in resolution, which is now reasonably
well understood from this point of view. For other natural candidates to study, however, such as
polynomial calculus or cutting planes, very little has been known. We are not aware of any nontrivial
space lower bounds for cutting planes, and for polynomial calculus the only lower bound has been
for CNF formulas of unbounded width in [Alekhnovich et al. ’02], where the space lower bound is
smaller than the initial width of the clauses in the formulas. Thus, in particular, it has been consistent
with current knowledge that polynomial calculus could be able to refute any k-CNF formula in
constant space.
In this paper, we prove several new results on space in polynomial calculus (PC), and in the
extended proof system polynomial calculus resolution (PCR) studied in [Alekhnovich et al. ’02]:
1. We prove an Ω(n) space lower bound in PC for the canonical 3-CNF version of the pigeonhole
principle formulas PHPm
n with m pigeons and n holes, and show that this is tight.
2. For PCR, we prove an Ω(n) space lower bound for a bitwise encoding of the functional pigeonhole
principle. These formulas have width O(log n), and hence this is an exponential
improvement over [Alekhnovich et al. ’02] measured in the width of the formulas.
3. We then present another encoding of the pigeonhole principle that has constant width, and
prove an Ω(n) space lower bound in PCR for these formulas as well.
4. Finally, we prove that any k-CNF formula can be refuted in PC in simultaneous exponential
size and linear space (which holds for resolution and thus for PCR, but was not obviously
the case for PC). We also characterize a natural class of CNF formulas for which the space
complexity in resolution and PCR does not change when the formula is transformed into 3-CNF
in the canonical way, something that we believe can be useful when proving PCR space lower
bounds for other well-studied formula families in proof complexity
Language and Proofs for Higher-Order SMT (Work in Progress)
Satisfiability modulo theories (SMT) solvers have throughout the years been
able to cope with increasingly expressive formulas, from ground logics to full
first-order logic modulo theories. Nevertheless, higher-order logic within SMT
is still little explored. One main goal of the Matryoshka project, which
started in March 2017, is to extend the reasoning capabilities of SMT solvers
and other automatic provers beyond first-order logic. In this preliminary
report, we report on an extension of the SMT-LIB language, the standard input
format of SMT solvers, to handle higher-order constructs. We also discuss how
to augment the proof format of the SMT solver veriT to accommodate these new
constructs and the solving techniques they require.Comment: In Proceedings PxTP 2017, arXiv:1712.0089
On the relative proof complexity of deep inference via atomic flows
We consider the proof complexity of the minimal complete fragment, KS, of
standard deep inference systems for propositional logic. To examine the size of
proofs we employ atomic flows, diagrams that trace structural changes through a
proof but ignore logical information. As results we obtain a polynomial
simulation of versions of Resolution, along with some extensions. We also show
that these systems, as well as bounded-depth Frege systems, cannot polynomially
simulate KS, by giving polynomial-size proofs of certain variants of the
propositional pigeonhole principle in KS.Comment: 27 pages, 2 figures, full version of conference pape
Symbolic Exact Inference for Discrete Probabilistic Programs
The computational burden of probabilistic inference remains a hurdle for
applying probabilistic programming languages to practical problems of interest.
In this work, we provide a semantic and algorithmic foundation for efficient
exact inference on discrete-valued finite-domain imperative probabilistic
programs. We leverage and generalize efficient inference procedures for
Bayesian networks, which exploit the structure of the network to decompose the
inference task, thereby avoiding full path enumeration. To do this, we first
compile probabilistic programs to a symbolic representation. Then we adapt
techniques from the probabilistic logic programming and artificial intelligence
communities in order to perform inference on the symbolic representation. We
formalize our approach, prove it sound, and experimentally validate it against
existing exact and approximate inference techniques. We show that our inference
approach is competitive with inference procedures specialized for Bayesian
networks, thereby expanding the class of probabilistic programs that can be
practically analyzed
Understanding Space in Proof Complexity: Separations and Trade-offs via Substitutions
For current state-of-the-art DPLL SAT-solvers the two main bottlenecks are
the amounts of time and memory used. In proof complexity, these resources
correspond to the length and space of resolution proofs. There has been a long
line of research investigating these proof complexity measures, but while
strong results have been established for length, our understanding of space and
how it relates to length has remained quite poor. In particular, the question
whether resolution proofs can be optimized for length and space simultaneously,
or whether there are trade-offs between these two measures, has remained
essentially open.
In this paper, we remedy this situation by proving a host of length-space
trade-off results for resolution. Our collection of trade-offs cover almost the
whole range of values for the space complexity of formulas, and most of the
trade-offs are superpolynomial or even exponential and essentially tight. Using
similar techniques, we show that these trade-offs in fact extend to the
exponentially stronger k-DNF resolution proof systems, which operate with
formulas in disjunctive normal form with terms of bounded arity k. We also
answer the open question whether the k-DNF resolution systems form a strict
hierarchy with respect to space in the affirmative.
Our key technical contribution is the following, somewhat surprising,
theorem: Any CNF formula F can be transformed by simple variable substitution
into a new formula F' such that if F has the right properties, F' can be proven
in essentially the same length as F, whereas on the other hand the minimal
number of lines one needs to keep in memory simultaneously in any proof of F'
is lower-bounded by the minimal number of variables needed simultaneously in
any proof of F. Applying this theorem to so-called pebbling formulas defined in
terms of pebble games on directed acyclic graphs, we obtain our results.Comment: This paper is a merged and updated version of the two ECCC technical
reports TR09-034 and TR09-047, and it hence subsumes these two report
- …