5 research outputs found
Probabilities on Sentences in an Expressive Logic
Automated reasoning about uncertain knowledge has many applications. One
difficulty when developing such systems is the lack of a completely
satisfactory integration of logic and probability. We address this problem
directly. Expressive languages like higher-order logic are ideally suited for
representing and reasoning about structured knowledge. Uncertain knowledge can
be modeled by using graded probabilities rather than binary truth-values. The
main technical problem studied in this paper is the following: Given a set of
sentences, each having some probability of being true, what probability should
be ascribed to other (query) sentences? A natural wish-list, among others, is
that the probability distribution (i) is consistent with the knowledge base,
(ii) allows for a consistent inference procedure and in particular (iii)
reduces to deductive logic in the limit of probabilities being 0 and 1, (iv)
allows (Bayesian) inductive reasoning and (v) learning in the limit and in
particular (vi) allows confirmation of universally quantified
hypotheses/sentences. We translate this wish-list into technical requirements
for a prior probability and show that probabilities satisfying all our criteria
exist. We also give explicit constructions and several general
characterizations of probabilities that satisfy some or all of the criteria and
various (counter) examples. We also derive necessary and sufficient conditions
for extending beliefs about finitely many sentences to suitable probabilities
over all sentences, and in particular least dogmatic or least biased ones. We
conclude with a brief outlook on how the developed theory might be used and
approximated in autonomous reasoning agents. Our theory is a step towards a
globally consistent and empirically satisfactory unification of probability and
logic.Comment: 52 LaTeX pages, 64 definiton/theorems/etc, presented at conference
Progic 2011 in New Yor
Tree Based Hierarchical Reinforcement Learning
In this thesis we investigate methods for speeding up automatic control algorithms. Specifically, we provide new abstraction techniques for Reinforce-ment Learning and Semi-Markov Decision Processes (SMDPs). We introduce the use of policies as temporally abstract actions. This is different from pre-vious definitions of temporally abstract actions as we do not have termination criteria. We provide an approach for processing previously solved problems to extract these policies. We also contribute a method for using supplied or extracted policies to guide and speed up problem solving of new problems. We treat extracting policies as a supervised learning task and introduce the Lumberjack algorithm that extracts repeated sub-structure within a decision tree. We then introduce the TTree algorithm that combines state and temporal abstraction to increase problem solving speed on new problems. TTree solves SMDPs by using both user and machine supplied policies as temporally ab-stract actions while generating its own tree based abstract state representation
Tree Based Discretization for Continuous State Space Reinforcement Learning
Reinforcement learning is an effective technique for learning action policies in discrete stochastic environments, but its efficiency can decay exponentially with the size of the state space. In many situations significant portions of a large state space may be irrelevant to a specific goal and can be aggregated into a few, relevant, states. The U Tree algorithm generates a tree based state discretization that efficiently finds the relevant state chunks of large propositional domains. In this paper, we extend the U Tree algorithm to challenging domains with a continuous state space for which there is no initial discretization. This Continuous U Tree algorithm transfers traditional regression tree techniques to reinforcement learning. We have performed experiments in a variety of domains that show that Continuous U Tree effectively handles large continuous state spaces. In this paper, we report on results in two different domains, one gives a clear visualization of the algorithm and another empirically demonstrates an effective state discretization in a simple multi-agent environment