2,249 research outputs found
Stochastic Constraint Programming
To model combinatorial decision problems involving uncertainty and
probability, we introduce stochastic constraint programming. Stochastic
constraint programs contain both decision variables (which we can set) and
stochastic variables (which follow a probability distribution). They combine
together the best features of traditional constraint satisfaction, stochastic
integer programming, and stochastic satisfiability. We give a semantics for
stochastic constraint programs, and propose a number of complete algorithms and
approximation procedures. Finally, we discuss a number of extensions of
stochastic constraint programming to relax various assumptions like the
independence between stochastic variables, and compare with other approaches
for decision making under uncertainty.Comment: Proceedings of the 15th Eureopean Conference on Artificial
Intelligenc
Symmetry Breaking Constraints: Recent Results
Symmetry is an important problem in many combinatorial problems. One way of
dealing with symmetry is to add constraints that eliminate symmetric solutions.
We survey recent results in this area, focusing especially on two common and
useful cases: symmetry breaking constraints for row and column symmetry, and
symmetry breaking constraints for eliminating value symmetryComment: To appear in Proceedings of Twenty-Sixth Conference on Artificial
Intelligence (AAAI-12
Recommended from our members
Understanding Model-Based Reinforcement Learning and its Application in Safe Reinforcement Learning
Model-based reinforcement learning algorithms have been shown to achieve successful results on various continuous control benchmarks, but the understanding of model-based methods is limited. We try to interpret how model-based method works through novel experiments on state-of-the-art algorithms with an emphasis on the model learning part. We evaluate the role of the model learning in policy optimization and propose methods to learn a more accurate model. With a better understanding of model-based reinforcement learning, we then apply model-based methods to solve safe reinforcement learning (RL) problems with near-zero violation of hard constraints throughout training. Drawing an analogy with how humans and animals learn to perform safe actions, we break down the safe RL problem into three stages. First, we train agents in a constraint-free environment to learn a performant policy for reaching high rewards, and simultaneously learn a model of the dynamics. Second, we use model-based methods to plan safe actions and train a safeguarding policy from these actions through imitation. Finally, we propose a factored framework to train an overall policy that mixes the performant policy and the safeguarding policy. This three-step curriculum ensures near-zero violation of safety constraints at all times. As an advantage of model-based method, the sample complexity required at the second and third steps of the process is significantly lower than model-free methods and can enable online safe learning. We demonstrate the effectiveness of our methods in various continuous control problems and analyze the advantages over state-of-the-art approaches
Real-time and Probabilistic Temporal Logics: An Overview
Over the last two decades, there has been an extensive study on logical
formalisms for specifying and verifying real-time systems. Temporal logics have
been an important research subject within this direction. Although numerous
logics have been introduced for the formal specification of real-time and
complex systems, an up to date comprehensive analysis of these logics does not
exist in the literature. In this paper we analyse real-time and probabilistic
temporal logics which have been widely used in this field. We extrapolate the
notions of decidability, axiomatizability, expressiveness, model checking, etc.
for each logic analysed. We also provide a comparison of features of the
temporal logics discussed
Breaking Instance-Independent Symmetries In Exact Graph Coloring
Code optimization and high level synthesis can be posed as constraint
satisfaction and optimization problems, such as graph coloring used in register
allocation. Graph coloring is also used to model more traditional CSPs relevant
to AI, such as planning, time-tabling and scheduling. Provably optimal
solutions may be desirable for commercial and defense applications.
Additionally, for applications such as register allocation and code
optimization, naturally-occurring instances of graph coloring are often small
and can be solved optimally. A recent wave of improvements in algorithms for
Boolean satisfiability (SAT) and 0-1 Integer Linear Programming (ILP) suggests
generic problem-reduction methods, rather than problem-specific heuristics,
because (1) heuristics may be upset by new constraints, (2) heuristics tend to
ignore structure, and (3) many relevant problems are provably inapproximable.
Problem reductions often lead to highly symmetric SAT instances, and
symmetries are known to slow down SAT solvers. In this work, we compare several
avenues for symmetry breaking, in particular when certain kinds of symmetry are
present in all generated instances. Our focus on reducing CSPs to SAT allows us
to leverage recent dramatic improvement in SAT solvers and automatically
benefit from future progress. We can use a variety of black-box SAT solvers
without modifying their source code because our symmetry-breaking techniques
are static, i.e., we detect symmetries and add symmetry breaking predicates
(SBPs) during pre-processing.
An important result of our work is that among the types of
instance-independent SBPs we studied and their combinations, the simplest and
least complete constructions are the most effective. Our experiments also
clearly indicate that instance-independent symmetries should mostly be
processed together with instance-specific symmetries rather than at the
specification level, contrary to what has been suggested in the literature
Percentile Queries in Multi-Dimensional Markov Decision Processes
Markov decision processes (MDPs) with multi-dimensional weights are useful to
analyze systems with multiple objectives that may be conflicting and require
the analysis of trade-offs. We study the complexity of percentile queries in
such MDPs and give algorithms to synthesize strategies that enforce such
constraints. Given a multi-dimensional weighted MDP and a quantitative payoff
function , thresholds (one per dimension), and probability thresholds
, we show how to compute a single strategy to enforce that for all
dimensions , the probability of outcomes satisfying is at least . We consider classical quantitative payoffs from
the literature (sup, inf, lim sup, lim inf, mean-payoff, truncated sum,
discounted sum). Our work extends to the quantitative case the multi-objective
model checking problem studied by Etessami et al. in unweighted MDPs.Comment: Extended version of CAV 2015 pape
Composing Music in Constrained Search Environments
Composing music with computers in constrained search environments adds complexities and problems not present in the traditional problem domain of generative music. The traditional and well researched mechanisms of Markov chains, genetic algorithms and data driven rule based systems do not directly map to a problem domain in which pitch choice and rhythm choice are likely to be highly limited.
We therefore explore several possible solutions to generating rhythms in extremely constrained environments with the goal of generating music that adheres to user specified constraints and is aesthetically pleasing
- âŠ