Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/epsilon), where epsilon > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs. These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state MDPs which arise as extensions of classic, and heavily studied, purely stochastic processes. These include both the problem of maximizing and mininizing the termination (extinction) probability of multi-type branching MDPs, stochastic context-free MDPs, and 1-exit Recursive MDPs. Furthermore, we also show that we can compute in P-time an epsilon-optimal policy for both maximizing and minimizing branching, context-free, and 1-exit-Recursive MDPs, for any given desired epsilon > 0. This is despite the fact that actually computing optimal strategies is Sqrt-Sum-hard and PosSLP-hard in this setting. We also derive, as an easy consequence of these results, an FNP upper bound on the complexity of computing the value (within arbitrary desired precision) of branching simple stochastic games (BSSGs)

Stability and convergence in discrete convex monotone dynamical systems

We study the stable behaviour of discrete dynamical systems where the map is convex and monotone with respect to the standard positive cone. The notion of tangential stability for fixed points and periodic points is introduced, which is weaker than Lyapunov stability. Among others we show that the set of tangentially stable fixed points is isomorphic to a convex inf-semilattice, and a criterion is given for the existence of a unique tangentially stable fixed point. We also show that periods of tangentially stable periodic points are orders of permutations on $n$ letters, where $n$ is the dimension of the underlying space, and a sufficient condition for global convergence to periodic orbits is presented.Comment: 36 pages, 1 fugur

Eating Christmas Cookies, Whole-wheat Bread and Frozen Chicken in the Kindergarten: Doing Pedagogy by Other Means

The study presented here explores eating as a pedagogical practice by paying attention to arrangements of things such as Christmas cookies, whole-wheat and white bread, frozen chicken, plates, chairs, tables, and freezers. Through a series of ethnographic research examples from German and Brazilian preschools, it investigates how eating in the kindergarten can be a sensual pleasure, a health risk, an ethnic custom, or a civil right within different local histories. Through specific arrangements of foods and other things, young children are educated to eat with moderation, to change their ethnic dietary habits, or to be "modern citizens". Pedagogy can thus consist of doing public health, doing ethnic identity, or doing citizenship. Eating is an important way of doing pedagogy in early childhood education and care settings. © 2013 Springer Fachmedien Wiesbaden

Greatest Fixed Points of Probabilistic Min/Max Polynomial Equations, and Reachability for Branching Markov Decision Processes?

We give polynomial time algorithms for quantitative (and qualitative) reachability analysis for Branching Markov Decision Processes (BMDPs). Specifically, given a BMDP, and given an initial population, where the objective of the controller is to maximize (or minimize) the probability of eventually reaching a population that contains an object of a desired (or undesired) type, we give algorithms for approximating the supremum (infimum) reachability probability, within desired precision epsilon > 0, in time polynomial in the encoding size of the BMDP and in log(1/epsilon). We furthermore give P-time algorithms for computing epsilon-optimal strategies for both maximization and minimization of reachability probabilities. We also give P-time algorithms for all associated qualitative analysis problems, namely: deciding whether the optimal (supremum or infimum) reachability probabilities are 0 or 1. Prior to this paper, approximation of optimal reachability probabilities for BMDPs was not even known to be decidable. Our algorithms exploit the following basic fact: we show that for any BMDP, its maximum (minimum) non-reachability probabilities are given by the greatest fixed point (GFP) solution g* in [0,1]^n of a corresponding monotone max (min) Probabilistic Polynomial System of equations (max/min-PPS), x=P(x), which are the Bellman optimality equations for a BMDP with non-reachability objectives. We show how to compute the GFP of max/min PPSs to desired precision in P-time. We also study more general Branching Simple Stochastic Games (BSSGs) with (non-)reachability objectives. We show that: (1) the value of these games is captured by the GFP of a corresponding max-minPPS; (2) the quantitative problem of approximating the value is in TFNP; and (3) the qualitative problems associated with the value are all solvable in P-time

Block diagonalization

summary:We study block diagonalization of matrices induced by resolutions of the unit matrix into the sum of idempotent matrices. We show that the block diagonal matrices have disjoint spectra if and only if each idempotent matrix in the inducing resolution double commutes with the given matrix. Applications include a new characterization of an eigenprojection and of the Drazin inverse of a given matrix

Consecutive optimizers for a partitioning problem with applications to optimal inventory groupings for joint replenishment

We consider several subclasses of the problem of grouping n items (indexed 1, 2,.., n) into m subsets so as to minimize the function g(S 1,.., S,). In general, these problems are very difficult to solve to optimality, even for the case m = 2. We provide several sufficient conditions on g(') that guarantee that there is an optimum partition in which each subset consists of consecutive integers (or else the partition S,,-, S,, satisfies a more general condition called semiconsecutiveness"). Moreover, by restricting attention to 'consecutive" (or serniconsecutive " ) partitions, we can solve the partition problem in polynomial time for small values of m. If, in addition, g is symmetric, then the partition problem is solvable in purely polynomial time. We apply these results to generalizations of a problem in inventory groupings considered by the authors in a previous paper. We also relate the results to the Neyman-Pearson lemma in statistical hypothesis testing and to a graph partitioning problem of Barnes and Hoffman. C · lg · · ·II CIL ·I D···1C- ·------- · 111-ET a, , a and b,-, b be real numbers ordered so that for some integer 0 r n, b, *.., b, are negative, b,+,.., b are nonnegative and al ar-- c.-- and tbi I b ar+l an br+i- bn For b, = 0, we consider adb, to be +cc or- according to a> 0 or a, < 0. If ai = bi = 0, al/b1 is defined arbitrarily so that inequality (1) holds. As usual, we let a and b denote the vectors whose coordinates are a, and bi, respectively. Subject clasification: 334 partitioning items into subgroups, 625 optimal inventory groupings