An optimality system for finite average Markov decision chains under risk-aversion

summary:This work concerns controlled Markov chains with finite state space and compact action sets. The decision maker is risk-averse with constant risk-sensitivity, and the performance of a control policy is measured by the long-run average cost criterion. Under standard continuity-compactness conditions, it is shown that the (possibly non-constant) optimal value function is characterized by a system of optimality equations which allows to obtain an optimal stationary policy. Also, it is shown that the optimal superior and inferior limit average cost functions coincide

Eficiencia terminal en 10 años del posgrado en enseñanza de las ciencias UANL

La educación como eje de desarrollo de toda nación, es una estrategia insoslayable ante los retos que la modernidad, excelencia académica y competitividad nos marca en todos los niveles educativos. Esto indiscutiblemente se incorpora a la educación superior. Por lo anterior, la Universidad Autónoma de Nuevo León a través de la Secretaría Académica, convocó a cinco Facultades de la propia Universidad para desarrollar un proyecto compartido, el primero en su género, con el afán de impulsar los cuadros académicos para la enseñanza de las Ciencias en Biología, Física, Matemáticas y Química, consolidando un Programa de Maestría en Enseñanza de las Ciencias con las especialidades señaladas, con tronco común en la Facultad de Filosofía y Letras. Dicho programa registró un alto nivel en su eficiencia terminal, durante su vigencia, de 1997 al 2007, cumpliendo en su momento con el objetivo de ”Consolidar la infraestructura docente de las áreas básicas de Biología, Física, Matemáticas y Química de nuestra Universidad, mediante un programa de maestría congruente con nuestros planes de desarrollo y competitividad internacional”

Ecuaciones de optimalidad para el criterio del costo promedio sensible al riesgo en procesos de decisión markovianos sobre un espacio finito.

This work concerns Markov decision chains endowed with the risk-sensitive average cost criterion, and the main goals are to characterize the optimal value function and to determine an optimal stationary policy. The exposition begins in Chapter 1 where the notion of Markov decision chain is introduced, and the ideas of risk-aversion and risk–sensitivity coefficient are briefly discussed. After this point, the risk-sensitive average cost criterion is formulated, and the main objectives are formally stated. Next, in Chapter 2 a fundamental theorem by Howard and Matheson (1972), as well as a recent extension on the characterization of the optimal average cost in terms of a single optimality equation are analyzed; such results require that, under the action of any stationary policy, every state can be visited with positive probability regardless of the initial sate, and the arguments used in this work emphasize the central role of that communication property. The presentation continues in Chapter 3 studying a recent theorem on the existence of solutions to the optimality equation for ‘small’values of the risk-sensitivity coefficient, which was derived under the assumption that there exists a state that can be always reached with positive probability under the action of any stationary policy; the derivation presented in this work highlights the fundamental role of such an accessibility condition. The conclusions in Chapters 2 and 3 provide the motivation to pursue the main objective of this thesis, namely, to establish a characterization of the optimal risk-sensitive average cost without imposing any condition on the structure of the transition law of the model. This goal is achieved in Chapter 4, where the optimal risk-sensitive average cost function is characterized for general controlled Markov chains with finite state space and compact action sets, a result that is the main contribution of this thesis. It is supposed that the decision maker is risk-averse with constant risk-sensitivity coefficient and, under standard continuity–compactness conditions, it is proved that the (possibly non-constant) optimal value function is characterized by a nested system of equations, generalizing the conclusions s presented in the previous chapters, which require communication conditions on the transition law; moreover, it is shown that an optimal stationary policy can be derived form a solution of that system, and that the optimal superior and inferior limit average cost functions coincide. The approach used to obtain the main conclusions relies on the discounted method which, roughly, consists in using a family of contractive operators whose fixed points are used to approximate the optimal average index, to partition the state space in a family of equivalence classes, to determine a class of admissible actions at each state, and to construct a solution of a ‘reduced’ optimality equation on each equivalence class; the presentation of these results is based on the recent paper Alanís Durán and Cavazos-Cadena (2012). Finally, the exposition concludes in Chapter 5 with a retrospective view of the material presented in this work, and with the statement of two open problems concerning the extension of some of the conclusions in this work to models with denumerable state space