Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. In order to obtain the optimal control function of ML-POSC, a system of the forward Fokker-Planck (FP) equation and the backward Hamilton-Jacobi-Bellman (HJB) equation needs to be solved. In this work, we firstly show that the system of HJB-FP equations can be interpreted via the Pontryagin's minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) to ML-POSC, which has been used in the Pontryagin's minimum principle. FBSM is an algorithm to compute the forward FP equation and the backward HJB equation alternately. Although the convergence of FBSM is generally not guaranteed, it is guaranteed in ML-POSC because the coupling of HJB-FP equations is limited to the optimal control function in ML-POSC

Mean-Field Control Approach to Decentralized Stochastic Control with Finite-Dimensional Memories

Decentralized stochastic control (DSC) considers the optimal control problem of a multi-agent system. However, DSC cannot be solved except in the special cases because the estimation among the agents is generally intractable. In this work, we propose memory-limited DSC (ML-DSC), in which each agent compresses the observation history into the finite-dimensional memory. Because this compression simplifies the estimation among the agents, ML-DSC can be solved in more general cases based on the mean-field control theory. We demonstrate ML-DSC in the general LQG problem. Because estimation and control are not clearly separated in the general LQG problem, the Riccati equation is modified to the decentralized Riccati equation, which improves estimation as well as control. Our numerical experiment shows that the decentralized Riccati equation is superior to the conventional Riccati equation.Comment: arXiv admin note: text overlap with arXiv:2203.1068

Oligonucleotide Microarray Analysis of Dietary-Induced Hyperlipidemia Gene Expression Profiles in Miniature Pigs

BACKGROUND: Hyperlipidemia animal models have been established, but complete gene expression profiles of the transition from normal lipid levels have not been obtained. Miniature pigs are useful model animals for gene expression studies on dietary-induced hyperlipidemia because they have a similar anatomy and digestive physiology to humans, and blood samples can be obtained from them repeatedly. METHODOLOGY: Two typical dietary treatments were used for dietary-induced hyperlipidemia models, by using specific pathogen-free (SPF) Clawn miniature pigs. One was a high-fat and high-cholesterol diet (HFCD) and the other was a high-fat, high-cholesterol, and high-sucrose diet (HFCSD). Microarray analyses were conducted from whole blood samples during the dietary period and from white blood cells at the end of the dietary period to evaluate the transition of expression profiles of the two dietary models. PRINCIPAL FINDINGS: Variations in whole blood gene expression intensity within the HFCD or the HFCSD group were in the same range as the controls provide with normal diet at all periods. This indicates uniformity of dietary-induced hyperlipidemia for our dietary protocols. Gene ontology- (GO) based functional analyses revealed that characteristics of the common changes between HFCD and HFCSD were involved in inflammatory responses and reproduction. The correlation coefficient between whole blood and white blood cell expression profiles at 27 weeks with the HFCSD diet was significantly lower than that of the control and HFCD diet groups. This may be due to the effects of RNA originating from the tissues and/or organs. CONCLUSIONS: No statistically significant differences in fasting plasma lipids and glucose levels between the HFCD and HFCSD groups were observed. However, blood RNA analyses revealed different characteristics corresponding to the dietary protocols. In this study, whole blood RNA analyses proved to be a useful tool to evaluate transitions in dietary-induced hyperlipidemia gene expression profiles in miniature pigs

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker–Planck (FP) equation and the backward Hamilton–Jacobi–Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin’s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin’s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker–Planck (FP) equation and the backward Hamilton–Jacobi–Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin’s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin’s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC

Subject body weights.

<p><b>•</b> represents control, <b>⧫</b> represents HFCD, and ▴ represents HFCSD. Values correspond to means (SD).</p

The ratio of neutrophils to white blood cells (%).

<p>Values are the mean ± SD. NS; not significant.</p>†<p><i>P</i> values were calculated using a one-way factorial ANOVA.</p