Should one compute the Temporal Difference fix point or minimize the Bellman Residual? The unified oblique projection view

We investigate projection methods, for evaluating a linear approximation of the value function of a policy in a Markov Decision Process context. We consider two popular approaches, the one-step Temporal Difference fix-point computation (TD(0)) and the Bellman Residual (BR) minimization. We describe examples, where each method outperforms the other. We highlight a simple relation between the objective function they minimize, and show that while BR enjoys a performance guarantee, TD(0) does not in general. We then propose a unified view in terms of oblique projections of the Bellman equation, which substantially simplifies and extends the characterization of (schoknecht,2002) and the recent analysis of (Yu & Bertsekas, 2008). Eventually, we describe some simulations that suggest that if the TD(0) solution is usually slightly better than the BR solution, its inherent numerical instability makes it very bad in some cases, and thus worse on average

Singular value estimates of oblique projections

Let W and M be two finite dimensional subspaces of a Hilbert space H such that H = W ⊕ M⊥, and let PW {norm of matrix} M⊥ denote the oblique projection with range W and nullspace M⊥. In this article we get the following formula for the singular values of PW {norm of matrix} M⊥2 (sk (PW {norm of matrix} M⊥) - 1) = under(min, (F, H) ∈ X (W, M))2,where the minimum is taken over the set of all operator pairs (F, H) on H such that R (F) = W, R (H) = M and FH* = PW {norm of matrix} M⊥, and k ∈ {1, ..., dim W}. We also characterize all the pairs where the minimum is attained.Fil: Antezana, Jorge Abel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; Argentina. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Departamento de Matemáticas; ArgentinaFil: Corach, Gustavo. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Saavedra 15. Instituto Argentino de Matemática Alberto Calderón; Argentina. Universidad de Buenos Aires; Argentin

On condition numbers of symmetric and nonsymmetric domain decomposition methods

Using oblique projections and angles between subspaces we write condition number estimates for abstract nonsymmetric domain decomposition methods. In particular, we consider a restricted additive method for the Poisson equation and write a bound for the condition number of the preconditioned operator. We also obtain the non-negativity of the preconditioned operator. Condition number estimates are not enough for the convergence of iterative methods such as GMRES but these bounds may lead to further understanding of nonsymmetric domain decomposition methods

Banach space projections and Petrov-Galerkin estimates

We sharpen the classic a priori error estimate of Babuska for Petrov-Galerkin methods on a Banach space. In particular, we do so by (i) introducing a new constant, called the Banach-Mazur constant, to describe the geometry of a normed vector space; (ii) showing that, for a nontrivial projection $P$, it is possible to use the Banach-Mazur constant to improve upon the naive estimate $\| I - P \| \leq 1 + \| P \|$; and (iii) applying that improved estimate to the Petrov-Galerkin projection operator. This generalizes and extends a 2003 result of Xu and Zikatanov for the special case of Hilbert spaces.Comment: 9 pages; v2: added new section on application to Lp and Sobolev space

Weighted projections and Riesz frames

Let $\mathcal{H}$ be a (separable) Hilbert space and $\{e_k\}_{k\geq 1}$ a fixed orthonormal basis of $\mathcal{H}$. Motivated by many papers on scaled projections, angles of subspaces and oblique projections, we define and study the notion of compatibility between a subspace and the abelian algebra of diagonal operators in the given basis. This is used to refine previous work on scaled projections, and to obtain a new characterization of Riesz frames.Comment: 23 pages, to appear in Linear Algebra and its Application