Skip to main content
Article thumbnail
Location of Repository

Sequential Decision Making For Choice Functions On Gambles

By NATHAN HUNTLEY

Abstract

Choice functions on gambles (uncertain rewards) provide a framework for studying diverse preference and uncertainty models. For single decisions, applying a choice function is straightforward. In sequential problems, where the subject has multiple decision points, it is less easy. One possibility, called a normal form solution, is to list all available strategies (specifications of acts to take in all eventualities). This reduces the problem to a single choice between gambles. \ud \ud We primarily investigate three appealing behaviours of these solutions. The first, subtree perfectness, requires that the solution of a sequential problem, when restricted to a sub-problem, yields the solution to that sub-problem. The second, backward induction, requires that the solution of the problem can be found by working backwards from the final stage of the problem, removing everything judged non-optimal at any stage. The third, locality, applies only to special problems such as Markov decision processes, and requires that the optimal choice at each stage (considered separately from the rest of the problem) forms an optimal strategy.\ud \ud For these behaviours, we find necessary and sufficient conditions on the choice function. Showing that these hold is much easier than proving the behaviour from first principles. It also leads to answers to related questions, such as the relationship between the normal form and another popular form of solution, the extensive form. To demonstrate how these properties can be checked for particular choice functions, and how the theory can be easily extended to special cases, we investigate common choice functions from the theory of coherent lower previsions

Topics: Sequential decision making, choice functions, coherent lower previsions, backward induction
Year: 2011
OAI identifier: oai:etheses.dur.ac.uk:923
Provided by: Durham e-Theses

Suggested articles

Citations

  1. (2004). A contrast between two decision rules for use with (convex) sets of probabilities: -maximin doi
  2. (1983). A generalization of the quasilinear mean with applications to the measurement of income inequality and decision theory resolving the allais paradox. doi
  3. (1970). A Markovian model for hospital admission scheduling. doi
  4. (1976). A Mathematical Theory of Evidence. doi
  5. (1971). A study of lexicographic expected utility. doi
  6. (2008). A survey of the theory of coherent lower previsions. doi
  7. A theory of anticipated utility. doi
  8. (2008). An ecient normal form solution to decision trees with lower previsions. doi
  9. An investigation of the laws of thought on which are founded the mathematical theories of logic and probabilities. Walton and Maberly, doi
  10. (1961). and the savage axioms. doi
  11. (2005). Application of multicriteria decision analysis in environmental decision making. doi
  12. (2000). Applications of Markov decision processes in communication networks: a survey. doi
  13. (1961). Applied Statistical Decision Theory. doi
  14. (1976). Changing tastes and coherent dynamic choice. doi
  15. Characterizing factuality in normal form sequential decision making.
  16. (1988). Consequentialist foundations for expected utility. doi
  17. (1961). Consistency in statistical inference and decision.
  18. (2005). de Campos. Partially ordered preferences in decision trees: Computing strategies with imprecision in probabilities.
  19. (2007). Decision making under uncertainty using imprecise probabilities. doi
  20. (1985). Decision Making Under Uncertainty|The Case of State Dependent Preferences. doi
  21. (1988). Decision theory without `independence' or without `ordering': What is the dierence? doi
  22. (1971). Decision-making studies in patient management. doi
  23. (1995). Decision-making under environmental uncertainty. doi
  24. (1987). Does rolling back decision trees really require the independence axiom? doi
  25. (1989). Dynamic consistency and non-expected utility models of choice under uncertainty.
  26. (1990). Dynamic programming and in uence diagrams. doi
  27. (2005). Dynamic programming for deterministic discrete-time systems with uncertain gain. doi
  28. (1957). Dynamic Programming. doi
  29. (1987). Equivalent decision trees and their associated strategy sets. Theory and Decision, doi
  30. (1785). Essai sur l'Application de l'Analyse doi
  31. (1986). Evaluating in uence diagrams. doi
  32. (1982). Expected utility" analysis without the independence axiom. doi
  33. Exposition of the theory of choice under conditions of uncertainty. doi
  34. (2010). Filho. Sequential decision processes under act-state independence with arbitrary choice functions. In doi
  35. (2009). Finite approximations to coherent choice.
  36. (1975). Finitely additive conditional probabilities, conglomerability and disintegrations. doi
  37. (2004). Focussed processing of MDPs for path planning. doi
  38. (1957). Games and Decisions: introduction and critical survery.
  39. (2002). Generalizing Markov decision processes to imprecise probabilities. doi
  40. (1973). Independence of irrelevant alternatives. doi
  41. (1996). Inferences from multinomial data: Learning about a bag of marbles.
  42. (2001). Info-Gap Decision Theory: Decisions Under Severe Uncertainty. doi
  43. (1979). Introduction to Dynamic Systems. doi
  44. (1953). Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l' ecole Am ericaine. doi
  45. (2004). Learning and optimal control of imprecise Markov decision processes by dynamic programming using the imprecise Dirichlet model. doi
  46. (2006). Maintenance optimization of equipment by linear programming. doi
  47. (1985). Making Decisions. doi
  48. (2001). Making Hard Decisions. doi
  49. (1973). Markovian decision processes with uncertain transition probabilities. doi
  50. (2002). Markovian inventory policy with application to the paper industry. doi
  51. (1958). Modern moral philosophy. doi
  52. (1954). Multidimensional utilities. In
  53. (1954). Note on some proposed decision criteria.
  54. (2007). Notes on conditional previsions. doi
  55. (2001). On decision making under ambiguous prior and sampling information.
  56. (1974). On indeterminate probabilities. doi
  57. (1973). Path independence, rationality, and social choice. doi
  58. (1995). Planning under time constraints in stochastic domains.
  59. (2001). Probability and Finance: It's Only a Game! doi
  60. (1979). Prospect theory: An analysis of decision under risk. doi
  61. (1959). Purchasing raw material on a uctuating market. doi
  62. (1983). Rational belief. doi
  63. (1959). Rational choice functions and orderings. doi
  64. (1999). Rational decision making with imprecise probabilities.
  65. (1990). Rationality and Dynamic Choice: Foundational Explorations. doi
  66. (1975). Reexamination of the perfectness concept for equilibrium points in extensive games. doi
  67. (1998). Reinforcement Learning: An Introduction. doi
  68. (1986). Rolling back decision trees requires the independence axiom! doi
  69. (1977). Social choice theory: A re-examination. doi
  70. (1990). State-dependent utilities. doi
  71. (1985). Statistical decision theory and Bayesian analysis. doi
  72. (1991). Statistical Reasoning with Imprecise Probabilities. doi
  73. (1978). Subjectively weighted utility: A descriptive extension of the expected utility model. Organizational Behavior and Human Performance, doi
  74. (1984). The economics of optimism and pessimism: A de and some applications. doi
  75. (1980). The Enterprise of Knowledge.
  76. (1972). The Foundations of Statistics. doi
  77. (1959). Theories of decision-making in economics and behavioral science.
  78. Theory of games and economic behavior. doi
  79. (1974). Theory of Probability: A Critical Introductory Treatment. doi
  80. (1931). Truth and probability. In doi
  81. (1967). Upper and lower probabilities induced by a multivalued mapping. doi
  82. (1986). When is it optimal to kill o the market for used durable goods? doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.