Constrained partially observable Markov decision processes (CPOMDPs) have
been used to model various real-world phenomena. However, they are notoriously
difficult to solve to optimality, and there exist only a few approximation
methods for obtaining high-quality solutions. In this study, grid-based
approximations are used in combination with linear programming (LP) models to
generate approximate policies for CPOMDPs. A detailed numerical study is
conducted with six CPOMDP problem instances considering both their finite and
infinite horizon formulations. The quality of approximation algorithms for
solving unconstrained POMDP problems is established through a comparative
analysis with exact solution methods. Then, the performance of the LP-based
CPOMDP solution approaches for varying budget levels is evaluated. Finally, the
flexibility of LP-based approaches is demonstrated by applying deterministic
policy constraints, and a detailed investigation into their impact on rewards
and CPU run time is provided. For most of the finite horizon problems,
deterministic policy constraints are found to have little impact on expected
reward, but they introduce a significant increase to CPU run time. For infinite
horizon problems, the reverse is observed: deterministic policies tend to yield
lower expected total rewards than their stochastic counterparts, but the impact
of deterministic constraints on CPU run time is negligible in this case.
Overall, these results demonstrate that LP models can effectively generate
approximate policies for both finite and infinite horizon problems while
providing the flexibility to incorporate various additional constraints into
the underlying model.Comment: 42 pages, 8 figure