13,852 research outputs found
From average case complexity to improper learning complexity
The basic problem in the PAC model of computational learning theory is to
determine which hypothesis classes are efficiently learnable. There is
presently a dearth of results showing hardness of learning problems. Moreover,
the existing lower bounds fall short of the best known algorithms.
The biggest challenge in proving complexity results is to establish hardness
of {\em improper learning} (a.k.a. representation independent learning).The
difficulty in proving lower bounds for improper learning is that the standard
reductions from -hard problems do not seem to apply in this
context. There is essentially only one known approach to proving lower bounds
on improper learning. It was initiated in (Kearns and Valiant 89) and relies on
cryptographic assumptions.
We introduce a new technique for proving hardness of improper learning, based
on reductions from problems that are hard on average. We put forward a (fairly
strong) generalization of Feige's assumption (Feige 02) about the complexity of
refuting random constraint satisfaction problems. Combining this assumption
with our new technique yields far reaching implications. In particular,
1. Learning 's is hard.
2. Agnostically learning halfspaces with a constant approximation ratio is
hard.
3. Learning an intersection of halfspaces is hard.Comment: 34 page
An Enhanced Features Extractor for a Portfolio of Constraint Solvers
Recent research has shown that a single arbitrarily efficient solver can be
significantly outperformed by a portfolio of possibly slower on-average
solvers. The solver selection is usually done by means of (un)supervised
learning techniques which exploit features extracted from the problem
specification. In this paper we present an useful and flexible framework that
is able to extract an extensive set of features from a Constraint
(Satisfaction/Optimization) Problem defined in possibly different modeling
languages: MiniZinc, FlatZinc or XCSP. We also report some empirical results
showing that the performances that can be obtained using these features are
effective and competitive with state of the art CSP portfolio techniques
Proteus: A Hierarchical Portfolio of Solvers and Transformations
In recent years, portfolio approaches to solving SAT problems and CSPs have
become increasingly common. There are also a number of different encodings for
representing CSPs as SAT instances. In this paper, we leverage advances in both
SAT and CSP solving to present a novel hierarchical portfolio-based approach to
CSP solving, which we call Proteus, that does not rely purely on CSP solvers.
Instead, it may decide that it is best to encode a CSP problem instance into
SAT, selecting an appropriate encoding and a corresponding SAT solver. Our
experimental evaluation used an instance of Proteus that involved four CSP
solvers, three SAT encodings, and six SAT solvers, evaluated on the most
challenging problem instances from the CSP solver competitions, involving
global and intensional constraints. We show that significant performance
improvements can be achieved by Proteus obtained by exploiting alternative
view-points and solvers for combinatorial problem-solving.Comment: 11th International Conference on Integration of AI and OR Techniques
in Constraint Programming for Combinatorial Optimization Problems. The final
publication is available at link.springer.co
Sum of squares lower bounds for refuting any CSP
Let be a nontrivial -ary predicate. Consider a
random instance of the constraint satisfaction problem on
variables with constraints, each being applied to randomly
chosen literals. Provided the constraint density satisfies , such
an instance is unsatisfiable with high probability. The \emph{refutation}
problem is to efficiently find a proof of unsatisfiability.
We show that whenever the predicate supports a -\emph{wise uniform}
probability distribution on its satisfying assignments, the sum of squares
(SOS) algorithm of degree
(which runs in time ) \emph{cannot} refute a random instance of
. In particular, the polynomial-time SOS algorithm requires
constraints to refute random instances of
CSP when supports a -wise uniform distribution on its satisfying
assignments. Together with recent work of Lee et al. [LRS15], our result also
implies that \emph{any} polynomial-size semidefinite programming relaxation for
refutation requires at least constraints.
Our results (which also extend with no change to CSPs over larger alphabets)
subsume all previously known lower bounds for semialgebraic refutation of
random CSPs. For every constraint predicate~, they give a three-way hardness
tradeoff between the density of constraints, the SOS degree (hence running
time), and the strength of the refutation. By recent algorithmic results of
Allen et al. [AOW15] and Raghavendra et al. [RRS16], this full three-way
tradeoff is \emph{tight}, up to lower-order factors.Comment: 39 pages, 1 figur
A Learning Theoretic Approach to Energy Harvesting Communication System Optimization
A point-to-point wireless communication system in which the transmitter is
equipped with an energy harvesting device and a rechargeable battery, is
studied. Both the energy and the data arrivals at the transmitter are modeled
as Markov processes. Delay-limited communication is considered assuming that
the underlying channel is block fading with memory, and the instantaneous
channel state information is available at both the transmitter and the
receiver. The expected total transmitted data during the transmitter's
activation time is maximized under three different sets of assumptions
regarding the information available at the transmitter about the underlying
stochastic processes. A learning theoretic approach is introduced, which does
not assume any a priori information on the Markov processes governing the
communication system. In addition, online and offline optimization problems are
studied for the same setting. Full statistical knowledge and causal information
on the realizations of the underlying stochastic processes are assumed in the
online optimization problem, while the offline optimization problem assumes
non-causal knowledge of the realizations in advance. Comparing the optimal
solutions in all three frameworks, the performance loss due to the lack of the
transmitter's information regarding the behaviors of the underlying Markov
processes is quantified
Induction of Interpretable Possibilistic Logic Theories from Relational Data
The field of Statistical Relational Learning (SRL) is concerned with learning
probabilistic models from relational data. Learned SRL models are typically
represented using some kind of weighted logical formulas, which make them
considerably more interpretable than those obtained by e.g. neural networks. In
practice, however, these models are often still difficult to interpret
correctly, as they can contain many formulas that interact in non-trivial ways
and weights do not always have an intuitive meaning. To address this, we
propose a new SRL method which uses possibilistic logic to encode relational
models. Learned models are then essentially stratified classical theories,
which explicitly encode what can be derived with a given level of certainty.
Compared to Markov Logic Networks (MLNs), our method is faster and produces
considerably more interpretable models.Comment: Longer version of a paper appearing in IJCAI 201
Decentralized Constraint Satisfaction
We show that several important resource allocation problems in wireless
networks fit within the common framework of Constraint Satisfaction Problems
(CSPs). Inspired by the requirements of these applications, where variables are
located at distinct network devices that may not be able to communicate but may
interfere, we define natural criteria that a CSP solver must possess in order
to be practical. We term these algorithms decentralized CSP solvers. The best
known CSP solvers were designed for centralized problems and do not meet these
criteria. We introduce a stochastic decentralized CSP solver and prove that it
will find a solution in almost surely finite time, should one exist, also
showing it has many practically desirable properties. We benchmark the
algorithm's performance on a well-studied class of CSPs, random k-SAT,
illustrating that the time the algorithm takes to find a satisfying assignment
is competitive with stochastic centralized solvers on problems with order a
thousand variables despite its decentralized nature. We demonstrate the
solver's practical utility for the problems that motivated its introduction by
using it to find a non-interfering channel allocation for a network formed from
data from downtown Manhattan
- …