A note on the policy iteration algorithm for discounted Markov decision processes for a class of semicontinuous models

The standard version of the policy iteration (PI) algorithm fails for semicontinuous models, that is, for models with lower semicontinuous one-step costs and weakly continuous transition law. This is due to the lack of continuity properties of the discounted cost for stationary policies, thus appearing a measurability problem in the improvement step. The present work proposes an alternative version of PI algorithm which performs an smoothing step to avoid the measurability problem. Assuming that the model satisfies a Lyapunov growth conditions and also some standard continuity-compactness properties, it is shown the linear convergence of the policy iteration functions to the optimal value function. Strengthening the continuity conditions, in a second result, it is shown that among the improvement policies there is one with the best possible improvement and whose cost function is continuous.Comment: Fourteen pages page

A note on compact and {\sigma}-compact subsets of probability measures on metric spaces with an application to the distribution free newsvendor problem

This note identifies compact and {\sigma}-compact subsets of probability measures on a class of metric spaces with respect to the weak convergence topology. Moreover, it is shown by an example, that the space of probability measures on a {\sigma}-compact metric spaces not need to be {\sigma}-compact space, even though the converse statement holds true for metric spaces. The results are applied to an extended form of the distribution free newsvendor problem.Comment: Sixteen page

Empirical approximation in Markov games under unbounded payoff: discounted and average criteria

summary:This work deals with a class of discrete-time zero-sum Markov games whose state process $\left\{ x_{t}\right\}$ evolves according to the equation $x_{t+1}=F(x_{t},a_{t},b_{t},\xi _{t}),$ where $a_{t}$ and $b_{t}$ represent the actions of player 1 and 2, respectively, and $\left\{ \xi _{t}\right\}$ is a sequence of independent and identically distributed random variables with unknown distribution $\theta$. Assuming possibly unbounded payoff, and using the empirical distribution to estimate $\theta$, we introduce approximation schemes for the value of the game as well as for optimal strategies considering both, discounted and average criteria

Iteration Algorithms in Markov Decision Processes with State-Action-Dependent Discount Factors and Unbounded Costs

This chapter concerns discrete time Markov decision processes under a discounted optimality criterion with state-action-dependent discount factors, possibly unbounded costs, and noncompact admissible action sets. Under mild conditions, we show the existence of stationary optimal policies and we introduce the value iteration and the policy iteration algorithms to approximate the value function

Manejo Multidisciplinario del Adenocarcinoma de Páncreas: Guía de Práctica Clínica AUNA

Introduction: This article provides recommendations for the Multidisciplinary Management of Pancreatic Adenocarcinoma in the RED AUNA. Methods: A systematic search of clinical practice guidelines (CPG) similar to topics of interest was developed, it was assessed with the AGREE II instrument, a list of questions was elaborated under the PICO structure, a de novo search was carried out prioritizing reviews systematic with or without meta-analysis, followed by primary studies, the elaboration of the evidence tables and the evaluation of the global quality for the outcomes of the clinical questions was carried out following the GRADE methodology. Results: 5 PICO questions corresponding to initial management and systemic management were formulated with 18 recommendations regarding the most effective method for pathological diagnosis, biliary drainage and the most effective and safe systemic treatment in the neoadjuvant, adjuvant and metastatic setting. Conclusions: This article summarizes the methodology and evidence-based recommendations of the CPG for the multidisciplinary management of pancreatic adenocarcinoma of the AUNA Clinic Network.Introducción: Este artículo brinda recomendaciones para el Manejo Multidisciplinario del Adenocarcinoma de Páncreas en la RED AUNA. Métodos: Se desarrolló una búsqueda sistemática de guías de práctica clínica (GPC) similares al tópico de interés, se valoró con el instrumento AGREE II, se elaboró un listado de preguntas bajo la estructura PICO, se realizó una búsqueda de novo priorizando revisiones sistemáticas con o sin meta-análisis, seguida de estudios primarios, la elaboración de las tablas de evidencia y la evaluación de la calidad global para los desenlaces de las preguntas clínicas se realizó siguiendo la metodología GRADE. Resultados: Se formularon 5 preguntas PICO correspondientes al manejo inicial y manejo sistémico con 18 recomendaciones respecto al método más efectivo para el diagnóstico patológico, el drenaje biliar y el tratamiento sistémico más efectivo y seguro en el escenario neoadyuvante, adyuvante y metastásico. Conclusiones: El presente artículo resume la metodología y las recomendaciones basadas en evidencia de la GPC para el manejo multidisciplinario del Adenocarcinoma de páncreas de la Red de Clínicas AUNA

Semi-Markov control models with average costs

This paper studies semi-Markov control models with Borel state and control spaces, and unbounded cost functions, under the average cost criterion. Conditions are given for (i) the existence of a solution to the average cost optimality equation, and for (ii) the existence of strong optimal control policies. These conditions are illustrated with a semi-Markov replacement model

Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times

We deal with semi-Markov control processes (SMCPs) on Borel spaces with unbounded cost and mean holding time. Under suitable growth conditions on the cost function and the mean holding time, together with stability properties of the embedded Markov chains, we show the equivalence of several average cost criteria as well as the existence of stationary optimal policies with respect to each of these criteria

Implicaciones del ARN no codificante en biología y evolución: desde los primeros homínidos hasta los humanos modernos - Revisión

Massive genomic/transcriptomic sequencing has revealed a shocking paradox: pervasive or spurious transcription. Although such event is unwanted in principle, some of such transcripts may scape degradation, being further selected by evolution, with fascinating consequences on biology, including our brain development and what made us humans. Indeed, non-coding RNA are involved in many regulatory processes, across the central dogma of molecular biology, and even epigenetics events. Interestingly, that is partially accomplished regulating the expression and function of small RNA, like miRNA. More strikingly, non-coding RNA are involved in neuron physiology and brain neurogenesis, including outgrowth or neuron projections, synaptic functions and translation in synapses. Besides, non-coding RNA can be exported-imported between cells, through exosome vesicles. Surprisingly, some non-coding RNA are indeed translated into micropeptides, which may be involved in brain development. All that allows the remarkable cognitive power of the human brain. Unfortunately, this exquisite development, that made us humans, is specially prone to internal and external perturbations. They may generate neurodevelopmental, neurodegenerative and neuropsychiatric disorders, to which humans are more prone than other primates.La secuenciación genómica/transcriptómica masiva ha revelado una paradoja impactante: la transcripción generalizada o espuria. Aunque tal evento es no deseado en principio, algunos de tales transcritos pueden escapar de la degradación, siendo seleccionados por la evolución, con consecuencias fascinantes en biología, incluido el desarrollo de nuestro cerebro y lo que nos hizo humanos. De hecho, el ARN no codificante está involucrado en muchos procesos reguladores, a través de todo el dogma central de la biología molecular, e incluso en eventos epigenéticos. Curiosamente, ello se logra parcialmente regulando la expresión y la función de ARN pequeños, como los miARN. Más sorprendentemente, el ARN no codificante está involucrado en la fisiología de las neuronas y la neurogénesis cerebral, incluyendo las excrecencias o proyecciones neuronales, funciones sinápticas y traducción en sinapsis. Además, el ARN no codificante puede exportarse-importarse entre células, a través de vesículas de exosomas. Sorprendentemente, algunos ARN no codificantes se traducen en micropéptidos, que pueden estar involucrados en el desarrollo del cerebro. Todo eso permite el notable poder cognitivo del cerebro humano. Desafortunadamente, este desarrollo exquisito, que nos hizo humanos, es especialmente propenso a perturbaciones internas y externas. Así, pueden generarse trastornos del neurodesarrollo, neurodegenerativos y neuropsiquiátricos, a los cuales los humanos somos más propensos que otros primates