Ergodicity, Decisions, and Partial Information

In the simplest sequential decision problem for an ergodic stochastic process X, at each time n a decision u_n is made as a function of past observations X_0,...,X_{n-1}, and a loss l(u_n,X_n) is incurred. In this setting, it is known that one may choose (under a mild integrability assumption) a decision strategy whose pathwise time-average loss is asymptotically smaller than that of any other strategy. The corresponding problem in the case of partial information proves to be much more delicate, however: if the process X is not observable, but decisions must be based on the observation of a different process Y, the existence of pathwise optimal strategies is not guaranteed. The aim of this paper is to exhibit connections between pathwise optimal strategies and notions from ergodic theory. The sequential decision problem is developed in the general setting of an ergodic dynamical system (\Omega,B,P,T) with partial information Y\subseteq B. The existence of pathwise optimal strategies grounded in two basic properties: the conditional ergodic theory of the dynamical system, and the complexity of the loss function. When the loss function is not too complex, a general sufficient condition for the existence of pathwise optimal strategies is that the dynamical system is a conditional K-automorphism relative to the past observations \bigvee_n T^n Y. If the conditional ergodicity assumption is strengthened, the complexity assumption can be weakened. Several examples demonstrate the interplay between complexity and ergodicity, which does not arise in the case of full information. Our results also yield a decision-theoretic characterization of weak mixing in ergodic theory, and establish pathwise optimality of ergodic nonlinear filters.Comment: 45 page

On the exchange of intersection and supremum of sigma-fields in filtering theory

We construct a stationary Markov process with trivial tail sigma-field and a nondegenerate observation process such that the corresponding nonlinear filtering process is not uniquely ergodic. This settles in the negative a conjecture of the author in the ergodic theory of nonlinear filters arising from an erroneous proof in the classic paper of H. Kunita (1971), wherein an exchange of intersection and supremum of sigma-fields is taken for granted.Comment: 20 page

A discrete invitation to quantum filtering and feedback control

The engineering and control of devices at the quantum-mechanical level--such as those consisting of small numbers of atoms and photons--is a delicate business. The fundamental uncertainty that is inherently present at this scale manifests itself in the unavoidable presence of noise, making this a novel field of application for stochastic estimation and control theory. In this expository paper we demonstrate estimation and feedback control of quantum mechanical systems in what is essentially a noncommutative version of the binomial model that is popular in mathematical finance. The model is extremely rich and allows a full development of the theory, while remaining completely within the setting of finite-dimensional Hilbert spaces (thus avoiding the technical complications of the continuous theory). We introduce discretized models of an atom in interaction with the electromagnetic field, obtain filtering equations for photon counting and homodyne detection, and solve a stochastic control problem using dynamic programming and Lyapunov function methods.Comment: 76 pages, 12 figures. A PDF file with high resolution figures can be found at http://minty.caltech.edu/papers.ph

Deterministic Dicke state preparation with continuous measurement and control

We characterize the long-time projective behavior of the stochastic master equation describing a continuous, collective spin measurement of an atomic ensemble both analytically and numerically. By adding state based feedback, we show that it is possible to prepare highly entangled Dicke states deterministically.Comment: Additional information is available at http://minty.caltech.edu/Ensemble

Effect of Short Term Exercise and High Fat Diet on Skeletal Muscle miR133a

Micro RNAs (miR) are small non-coding RNA that regulate gene expression at the post-transcriptional level. miR133a is abundant in cardiac and skeletal muscle. In skeletal muscle, miR133a is best known for its regulatory role in myogenesis and differentiation. Nie (2016) found that muscle miR133a expression increased after acute exercise and with 12w of treadmill exercise training in mice. Knockdown of miR133a in transgenic mice resulted in blunted skeletal muscle mitochondrial biogenesis and function in response to exercise training (Nie, 2016) suggesting a role for miR133a in regulating the normal skeletal muscle metabolic adaptive response to exercise. Among other miR, skeletal muscle miR133a is reported as downregulated in insulin-resistant muscle. Insulin resistance in mice fed a high-fat diet is detectable after 3 days on diet (Lee, 2011). In this study, voluntary, rather than forced, exercise was employed to test whether miR133a expression is regulated early in the adoption of increased daily physical activity

Quantum filter reduction for measurement-feedback control via unsupervised manifold learning

We derive simple models for the dynamics of a single atom coupled to a cavity field mode in the absorptive bistable parameter regime by projecting the time evolution of the state of the system onto a suitably chosen nonlinear low-dimensional manifold, which is found by use of local tangent space alignment. The output field from the cavity is detected with a homodyne detector allowing observation of quantum jumps of the system between states with different average numbers of photons in the cavity. We find that the models, which are significantly faster to integrate numerically than the full stochastic master equation, largely reproduce the dynamics of the system, and we demonstrate that they are sufficiently accurate to facilitate feedback control of the state of the system based on the predictions of the models alone.Comment: 15 pages, 6 figure

Haemogenic endocardium contributes to transient definitive haematopoiesis.

Haematopoietic cells arise from spatiotemporally restricted domains in the developing embryo. Although studies of non-mammalian animal and in vitro embryonic stem cell models suggest a close relationship among cardiac, endocardial and haematopoietic lineages, it remains unknown whether the mammalian heart tube serves as a haemogenic organ akin to the dorsal aorta. Here we examine the haemogenic activity of the developing endocardium. Mouse heart explants generate myeloid and erythroid colonies in the absence of circulation. Haemogenic activity arises from a subset of endocardial cells in the outflow cushion and atria earlier than in the aorta-gonad-mesonephros region, and is transient and definitive in nature. Interestingly, key cardiac transcription factors, Nkx2-5 and Isl1, are expressed in and required for the haemogenic population of the endocardium. Together, these data suggest that a subset of endocardial/endothelial cells serve as a de novo source for transient definitive haematopoietic progenitors

The impact of low erythrocyte density in human blood on the fitness and energetic reserves of the African malaria vector Anopheles gambiae

Background Anaemia is a common health problem in the developing world. This condition is characterized by a reduction in erythrocyte density, primarily from malnutrition and/or infectious diseases such as malaria. As red blood cells are the primary source of protein for haematophagous mosquitoes, any reduction could impede the ability of mosquito vectors to transmit malaria by influencing their fitness or that of the parasites they transmit. The aim of this study was to determine the impact of differences in the density of red blood cells in human blood on malaria vector (Anopheles gambiae sensu stricto) fitness. The hypotheses tested are that mosquito vector energetic reserves and fitness are negatively influenced by reductions in the red cell density of host human blood meals commensurate with those expected from severe anaemia. Methods Mosquitoes (An. gambiae s.s.) were offered blood meals of different packed cell volume(PCV) of human blood consistent with those arising from severe anaemia (15%) and normalPCV (50%). Associations between mosquito energetic reserves (lipid, glucose and glycogen)and fitness measures (reproduction and survival) and blood meal PCV were investigated. Results The amount of protein that malaria vectors acquired from blood feeding (indexed by haematin excretion) was significantly reduced at low blood PCV. However, mosquitoes feeding on blood of low PCV had the same oviposition rates as those feeding on blood of normal PCV, and showed an increase in egg production of around 15%. The long-term survival of An. gambiae s.s was reduced after feeding on low PCV blood, but PCV had no significant impact on the proportion of mosquitoes surviving through the minimal period required to develop and transmit malaria parasites (estimated as 14 days post-blood feeding). The impact of blood PCV on the energetic reserves of mosquitoes was relatively minor. Conclusions These results suggest that feeding on human hosts whose PCV has been depleted due to severe anaemia does not significantly reduce the fitness or transmission potential of malaria vectors, and indicates that mosquitoes may be able exploit resources for reproduction more efficiently from blood of low rather than normal PCV

A Quantum Langevin Formulation of Risk-Sensitive Optimal Control

In this paper we formulate a risk-sensitive optimal control problem for continuously monitored open quantum systems modelled by quantum Langevin equations. The optimal controller is expressed in terms of a modified conditional state, which we call a risk-sensitive state, that represents measurement knowledge tempered by the control purpose. One of the two components of the optimal controller is dynamic, a filter that computes the risk-sensitive state. The second component is an optimal control feedback function that is found by solving the dynamic programming equation. The optimal controller can be implemented using classical electronics. The ideas are illustrated using an example of feedback control of a two-level atom