3 research outputs found
Speeding Up the Convergence of Value Iteration in Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) have recently become
popular among many AI researchers because they serve as a natural model for
planning under uncertainty. Value iteration is a well-known algorithm for
finding optimal policies for POMDPs. It typically takes a large number of
iterations to converge. This paper proposes a method for accelerating the
convergence of value iteration. The method has been evaluated on an array of
benchmark problems and was found to be very effective: It enabled value
iteration to converge after only a few iterations on all the test problems
A Method for Speeding Up Value Iteration in Partially Observable Markov Decision Processes
We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incorporated into any existing POMDP value iteration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called incremental pruning. We find that the technique can make incremental pruning run several orders of magnitude faster