5 research outputs found
Evaluating influence diagrams with decision circuits
Although a number of related algorithms have been developed to evaluate
influence diagrams, exploiting the conditional independence in the diagram, the
exact solution has remained intractable for many important problems. In this
paper we introduce decision circuits as a means to exploit the local structure
usually found in decision problems and to improve the performance of influence
diagram analysis. This work builds on the probabilistic inference algorithms
using arithmetic circuits to represent Bayesian belief networks [Darwiche,
2003]. Once compiled, these arithmetic circuits efficiently evaluate
probabilistic queries on the belief network, and methods have been developed to
exploit both the global and local structure of the network. We show that
decision circuits can be constructed in a similar fashion and promise similar
benefits.Comment: Appears in Proceedings of the Twenty-Third Conference on Uncertainty
in Artificial Intelligence (UAI2007
Three new sensitivity analysis methods for influence diagrams
Performing sensitivity analysis for influence diagrams using the decision
circuit framework is particularly convenient, since the partial derivatives
with respect to every parameter are readily available [Bhattacharjya and
Shachter, 2007; 2008]. In this paper we present three non-linear sensitivity
analysis methods that utilize this partial derivative information and therefore
do not require re-evaluating the decision situation multiple times.
Specifically, we show how to efficiently compare strategies in decision
situations, perform sensitivity to risk aversion and compute the value of
perfect hedging [Seyller, 2008].Comment: Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty
in Artificial Intelligence (UAI2010
Dynamic programming in in uence diagrams with decision circuits
Decision circuits perform efficient evaluation of influence diagrams,
building on the ad- vances in arithmetic circuits for belief net- work
inference [Darwiche, 2003; Bhattachar- jya and Shachter, 2007]. We show how
even more compact decision circuits can be con- structed for dynamic
programming in influ- ence diagrams with separable value functions and
conditionally independent subproblems. Once a decision circuit has been
constructed based on the diagram's "global" graphical structure, it can be
compiled to exploit "lo- cal" structure for efficient evaluation and sen-
sitivity analysis.Comment: Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty
in Artificial Intelligence (UAI2010
Recurrent Sum-Product-Max Networks for Decision Making in Perfectly-Observed Environments
Recent investigations into sum-product-max networks (SPMN) that generalize
sum-product networks (SPN) offer a data-driven alternative for decision making,
which has predominantly relied on handcrafted models. SPMNs computationally
represent a probabilistic decision-making problem whose solution scales
linearly in the size of the network. However, SPMNs are not well suited for
sequential decision making over multiple time steps. In this paper, we present
recurrent SPMNs (RSPMN) that learn from and model decision-making data over
time. RSPMNs utilize a template network that is unfolded as needed depending on
the length of the data sequence. This is significant as RSPMNs not only inherit
the benefits of SPMNs in being data driven and mostly tractable, they are also
well suited for sequential problems. We establish conditions on the template
network, which guarantee that the resulting SPMN is valid, and present a
structure learning algorithm to learn a sound template network. We demonstrate
that the RSPMNs learned on a testbed of sequential decision-making data sets
generate MEUs and policies that are close to the optimal on perfectly-observed
domains. They easily improve on a recent batch-constrained reinforcement
learning method, which is important because RSPMNs offer a new model-based
approach to offline reinforcement learning
BHATTACHARJYA & SHACHTER 9 Evaluating influence diagrams with decision circuits
Although a number of related algorithms have been developed to evaluate influence diagrams, exploiting the conditional independence in the diagram, the exact solution has remained intractable for many important problems. In this paper we introduce decision circuits as a means to exploit the local structure usually found in decision problems and to improve the performance of influence diagram analysis. This work builds on the probabilistic inference algorithms using arithmetic circuits to represent Bayesian belief networks [Darwiche, 2003]. Once compiled, these arithmetic circuits efficiently evaluate probabilistic queries on the belief network, and methods have been developed to exploit both the global and local structure of the network. We show that decision circuits can be constructed in a similar fashion and promise similar benefits.