965 research outputs found
Volumetric Spanners: an Efficient Exploration Basis for Learning
Numerous machine learning problems require an exploration basis - a mechanism
to explore the action space. We define a novel geometric notion of exploration
basis with low variance, called volumetric spanners, and give efficient
algorithms to construct such a basis.
We show how efficient volumetric spanners give rise to the first efficient
and optimal regret algorithm for bandit linear optimization over general convex
sets. Previously such results were known only for specific convex sets, or
under special conditions such as the existence of an efficient self-concordant
barrier for the underlying set
von Neumann-Morgenstern and Savage Theorems for Causal Decision Making
Causal thinking and decision making under uncertainty are fundamental aspects
of intelligent reasoning. Decision making under uncertainty has been well
studied when information is considered at the associative (probabilistic)
level. The classical Theorems of von Neumann-Morgenstern and Savage provide a
formal criterion for rational choice using purely associative information.
Causal inference often yields uncertainty about the exact causal structure, so
we consider what kinds of decisions are possible in those conditions. In this
work, we consider decision problems in which available actions and consequences
are causally connected. After recalling a previous causal decision making
result, which relies on a known causal model, we consider the case in which the
causal mechanism that controls some environment is unknown to a rational
decision maker. In this setting we state and prove a causal version of Savage's
Theorem, which we then use to develop a notion of causal games with its
respective causal Nash equilibrium. These results highlight the importance of
causal models in decision making and the variety of potential applications.Comment: Submitted to Journal of Causal Inferenc
Adaptation-Based Programming in Haskell
We present an embedded DSL to support adaptation-based programming (ABP) in
Haskell. ABP is an abstract model for defining adaptive values, called
adaptives, which adapt in response to some associated feedback. We show how our
design choices in Haskell motivate higher-level combinators and constructs and
help us derive more complicated compositional adaptives.
We also show an important specialization of ABP is in support of
reinforcement learning constructs, which optimize adaptive values based on a
programmer-specified objective function. This permits ABP users to easily
define adaptive values that express uncertainty anywhere in their programs.
Over repeated executions, these adaptive values adjust to more efficient ones
and enable the user's programs to self optimize.
The design of our DSL depends significantly on the use of type classes. We
will illustrate, along with presenting our DSL, how the use of type classes can
support the gradual evolution of DSLs.Comment: In Proceedings DSL 2011, arXiv:1109.032
- …