Restricted Value Iteration: Theory and Algorithms

Value iteration is a popular algorithm for finding near optimal policies for POMDPs. It is inefficient due to the need to account for the entire belief space, which necessitates the solution of large numbers of linear programs. In this paper, we study value iteration restricted to belief subsets. We show that, together with properly chosen belief subsets, restricted value iteration yields near-optimal policies and we give a condition for determining whether a given belief subset would bring about savings in space and time. We also apply restricted value iteration to two interesting classes of POMDPs, namely informative POMDPs and near-discernible POMDPs

Experimental investigations in combining primal dual interior point method and simplex based LP solvers

The use of a primal dual interior point method (PD) based optimizer as a robust linear programming (LP) solver is now well established. Instead of replacing the sparse simplex algorithm (SSX), the PD is increasingly seen as complementing it. The progress of PD iterations is not hindered by the degeneracy or the stalling problem of the SSX, indeed it reaches the 'near optimum' solution very quickly. The SSX algorithm, in contrast, is not affected by the boundary conditions which slow down the convergence of the PD. If the solution to the LP problem is non unique, the PD algorithm converges to an interior point of the solution set while the SSX algorithm finds an extreme point solution. To take advantage of the attractive properties of both the PD and the SSX, we have designed a hybrid framework whereby cross over from PD to SSX can take place at any stage of the PD optimization run. The cross over to SSX involves the partition of the PD solution set to active and dormant variables. In this paper we examine the practical difficulties in partitioning the solution set, we discuss the reliability of predicting the solution set partition before optimality is reached and report the results of combining exact and inexact prediction with SSX basis recovery

Mathematical programs with complementarity constraints: convergence properties of a smoothing method

In this paper, optimization problems $P$ with complementarity constraints are considered. Characterizations for local minimizers $\bar{x}$ of $P$ of Orders 1 and 2 are presented. We analyze a parametric smoothing approach for solving these programs in which $P$ is replaced by a perturbed problem $P_{\tau}$ depending on a (small) parameter $\tau$. We are interested in the convergence behavior of the feasible set $\cal{F}_{\tau}$ and the convergence of the solutions $\bar{x}_{\tau}$ of $P_{\tau}$ for $\tau\to 0.$ In particular, it is shown that, under generic assumptions, the solutions $\bar{x}_{\tau}$ are unique and converge to a solution $\bar{x}$ of $P$ with a rate $\cal{O}(\sqrt{\tau})$. Moreover, the convergence for the Hausdorff distance $d(\cal{F}_{\tau}$, $\cal{F})$ between the feasible sets of $P_{\tau}$ and $P$ is of order $\cal{O}(\sqrt{\tau})$

Block designs for experiments with non-normal response

Many experiments measure a response that cannot be adequately described by a linear model withnormally distributed errors and are often run in blocks of homogeneous experimental units. Wedevelop the first methods of obtaining efficient block designs for experiments with an exponentialfamily response described by a marginal model fitted via Generalized Estimating Equations. Thismethodology is appropriate when the blocking factor is a nuisance variable as, for example, occursin industrial experiments. A D-optimality criterion is developed for finding designs robust to thevalues of the marginal model parameters and applied using three strategies: unrestricted algorithmicsearch, use of minimum-support designs, and blocking of an optimal design for the correspondingGeneralized Linear Model. Designs obtained from each strategy are critically compared and shownto be much more efficient than designs that ignore the blocking structure. The designs are comparedfor a range of values of the intra-block working correlation and for exchangeable, autoregressive andnearest neighbor structures. An analysis strategy is developed for a binomial response that allows es-timation from experiments with sparse data, and its efectiveness demonstrated. The design strategiesare motivated and demonstrated through the planning of an experiment from the aeronautics industr