Stabilizing reinforcement learning control: A modular framework for optimizing over all stable behavior

We propose a framework for the design of feedback controllers that combines the optimization-driven and model-free advantages of deep reinforcement learning with the stability guarantees provided by using the Youla-Kucera parameterization to define the search domain. Recent advances in behavioral systems allow us to construct a data-driven internal model; this enables an alternative realization of the Youla-Kucera parameterization based entirely on input-output exploration data. Perhaps of independent interest, we formulate and analyze the stability of such data-driven models in the presence of noise. The Youla-Kucera approach requires a stable "parameter" for controller design. For the training of reinforcement learning agents, the set of all stable linear operators is given explicitly through a matrix factorization approach. Moreover, a nonlinear extension is given using a neural network to express a parameterized set of stable operators, which enables seamless integration with standard deep learning libraries. Finally, we show how these ideas can also be applied to tune fixed-structure controllers.Comment: Preprint; 18 pages. arXiv admin note: text overlap with arXiv:2304.0342

Reinforcement Learning with Partial Parametric Model Knowledge

We adapt reinforcement learning (RL) methods for continuous control to bridge the gap between complete ignorance and perfect knowledge of the environment. Our method, Partial Knowledge Least Squares Policy Iteration (PLSPI), takes inspiration from both model-free RL and model-based control. It uses incomplete information from a partial model and retains RL's data-driven adaption towards optimal performance. The linear quadratic regulator provides a case study; numerical experiments demonstrate the effectiveness and resulting benefits of the proposed method.Comment: IFAC World Congress 202

Successful β cells islet regeneration in streptozotocin-induced diabetic baboons using ultrasound-targeted microbubble gene therapy with cyclinD2/CDK4/GLP1

Both major forms of diabetes mellitus (DM) involve β-cell destruction and dysfunction. New treatment strategies have focused on replenishing the deficiency of β-cell mass common to both major forms of diabetes by islet transplantation or β-cell regeneration. The pancreas, not the liver, is the ideal organ for islet regeneration, because it is the natural milieu for islets. Since islet mass is known to increase during obesity and pregnancy, the concept of stimulating pancreatic islet regeneration in vivo is both rational and physiologic. This paper proposes a novel approach in which non-viral gene therapy is targeted to pancreatic islets using ultrasound targeted microbubble destruction (UTMD) in a non-human primate model (NHP), the baboon. Treated baboons received a gene cocktail comprised of cyclinD2, CDK, and GLP1, which in rats results in robust and durable islet regeneration with normalization of blood glucose, insulin, and C-peptide levels. We were able to generate important preliminary data indicating that gene therapy by UTMD can achieve in vivo normalization of the intravenous (IV) glucose tolerance test (IVGTT) curves in STZ hyperglycemic-induced conscious tethered baboons. Immunohistochemistry clearly demonstrated evidence of islet regeneration and restoration of β-cell mass

Interfacial Electron-Phonon Coupling Constants Extracted from Intrinsic Replica Bands in Monolayer FeSe/SrTiO3_3

The observation of replica bands by angle-resolved photoemission spectroscopy has ignited interest in the study of electron-phonon coupling at low carrier densities, particularly in monolayer FeSe/SrTiO$_3$, where the appearance of replica bands has motivated theoretical work suggesting that the interfacial coupling of electrons in the FeSe layer to optical phonons in the SrTiO$_3$ substrate might contribute to the enhanced superconducting pairing temperature. Alternatively, it has also been recently proposed that such replica bands might instead originate from extrinsic final state losses associated with the photoemission process. Here, we perform a quantitative examination of replica bands in monolayer FeSe/SrTiO$_3$, where we are able to conclusively demonstrate that the replica bands are indeed signatures of intrinsic electron-boson coupling, and not associated with final state effects. A detailed analysis of the energy splittings between the higher-order replicas, as well as other self-energy effects, allow us to determine that the interfacial electron-phonon coupling in the system corresponds to a value of $\lambda = 0.19 \pm 0.02$.Comment: 5 pages, 4 figure

Successful pharmaceutical-grade streptozotocin (STZ)-induced hyperglycemia in a conscious tethered baboon (Papio hamadryas) model

Background Non-human primate (NHP) diabetic models using chemical ablation of b-cells with STZ have been achieved by several research groups. Chemotherapeutic STZ could lead to serious adverse events including nephrotoxicity, hepatotoxicity, and mortality. Methods We implemented a comprehensive therapeutic strategy that included the tether system, permanent indwelling catheter implants, an aggressive hydration protocol, management for pain with IV nubain and anxiety with IV midazolam, moment-by-moment monitoring of glucose levels post-STZ administration, and continuous intravenous insulin therapy. Results A triphasic response in blood glucose after STZ administration was fully characterized. A dangerous hypoglycemic phase was also detected in all baboons. Other significant findings were hyperglycemia associated with low levels of plasma leptin, insulin and C-peptide concentrations, hyperglucagonemia, and elevated non-esterified fatty acids (NEFA) concentrations. Conclusions We successfully induced frank diabetes by IV administering a single dose of pharmaceutical-grade STZ safely and without adverse events in conscious tethered baboons

Convergent organization of aberrant MYB complex controls oncogenic gene expression in acute myeloid leukemia.

Dysregulated gene expression contributes to most prevalent features in human cancers. Here, we show that most subtypes of acute myeloid leukemia (AML) depend on the aberrant assembly of MYB transcriptional co-activator complex. By rapid and selective peptidomimetic interference with the binding of CBP/P300 to MYB, but not CREB or MLL1, we find that the leukemic functions of MYB are mediated by CBP/P300 co-activation of a distinct set of transcription factor complexes. These MYB complexes assemble aberrantly with LYL1, E2A, C/EBP family members, LMO2, and SATB1. They are organized convergently in genetically diverse subtypes of AML and are at least in part associated with inappropriate transcription factor co-expression. Peptidomimetic remodeling of oncogenic MYB complexes is accompanied by specific proteolysis and dynamic redistribution of CBP/P300 with alternative transcription factors such as RUNX1 to induce myeloid differentiation and apoptosis. Thus, aberrant assembly and sequestration of MYB:CBP/P300 complexes provide a unifying mechanism of oncogenic gene expression in AML. This work establishes a compelling strategy for their pharmacologic reprogramming and therapeutic targeting for diverse leukemias and possibly other human cancers caused by dysregulated gene control