Average optimality for continuous-time Markov decision processes in polish spaces

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequalities we prove the existence of optimal (deterministic) stationary policies by using the Dynkin formula. Moreover, we present a ``semimartingale characterization'' of an optimal stationary policy. Finally, we use a generalized Potlach process with control to illustrate the difference between our conditions and those in the previous literature, and then further apply our results to average optimal control problems of generalized birth--death systems, upwardly skip-free processes and two queueing systems. The approach developed in this paper is slightly different from the ``optimality inequality approach'' widely used in the previous literature.Comment: Published at http://dx.doi.org/10.1214/105051606000000105 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Portfolio optimization with Markov-modulated stock prices and interest rates

A financial market with one bond and one stock is considered where the risk free interest rate, the appreciation rate of the stock and the volatility of the stock depend on an external finite state Markov chain. We investigate the problem of maximizing the expected utility from terminal wealth and solve it by stochastic control methods for different utility functions. Due to explicit solutions it is possible to compare the value function of the problem to one where we have constant (average) market data. The case of benchmark optimization is also considered

Portfolio optimization with unobservable Markov-modulated drift process

We study portfolio optimization problems in which the drift rate of the stock is Markov modulated and the driving factors cannot be observed by the investor. Using results from filter theory, we reduce this problem to one with complete observation. In the cases of logarithmic and power utility, we solve the problem explicitly with the help of stochastic control methods. It turns out that the value function is a classical solution of the corresponding Hamilton-Jacobi-Bellman equation. As a special case, we investigate the so-called Bayesian case, i.e. where the drift rate is unknown but does not change over time. In this case, we prove a number of interesting properties of the optimal portfolio strategy. In particular, using the likelihood-ratio ordering, we can compare the optimal investment in the case of observable drift rate to that in the case of unobservable drift rate. Thus, we also obtain the sign of the drift risk

Are reprogrammed cells a useful tool for studying dopamine dysfunction in psychotic disorders? A review of the current evidence

Since 2006, reprogrammed cells have increasingly been used as a biomedical research technique in addition to neuro-psychiatric methods. These rapidly evolving techniques allow for the generation of neuronal sub-populations, and have sparked interest not only in monogenetic neuro-psychiatric diseases, but also in poly-genetic and poly-aetiological disorders such as schizophrenia (SCZ) and bipolar disorder (BPD). This review provides a summary of 19 publications on reprogrammed adult somatic cells derived from patients with SCZ, and five publications using this technique in patients with BPD. As both disorders are complex and heterogeneous, there is a plurality of hypotheses to be tested in vitro. In SCZ, data on alterations of dopaminergic transmission in vitro are sparse, despite the great explanatory power of the so-called DA hypothesis of SCZ. Some findings correspond to perturbations of cell energy metabolism, and observations in reprogrammed cells suggest neuro-developmental alterations. Some studies also report on the efficacy of medicinal compounds to revert alterations observed in cellular models. However, due to the paucity of replication studies, no comprehensive conclusions can be drawn from studies using reprogrammed cells at the present time. In the future, findings from cell culture methods need to be integrated with clinical, epidemiological, pharmacological and imaging data in order to generate a more comprehensive picture of SCZ and BPD