Controlling Level of Unconsciousness by Titrating Propofol with Deep Reinforcement Learning

Reinforcement Learning (RL) can be used to fit a mapping from patient state to a medication regimen. Prior studies have used deterministic and value-based tabular learning to learn a propofol dose from an observed anesthetic state. Deep RL replaces the table with a deep neural network and has been used to learn medication regimens from registry databases. Here we perform the first application of deep RL to closed-loop control of anesthetic dosing in a simulated environment. We use the cross-entropy method to train a deep neural network to map an observed anesthetic state to a probability of infusing a fixed propofol dosage. During testing, we implement a deterministic policy that transforms the probability of infusion to a continuous infusion rate. The model is trained and tested on simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The deep RL agent significantly outperformed a proportional-integral-derivative controller (median absolute performance error 1.7% +/- 0.6 and 3.4% +/- 1.2). Modeling continuous input variables instead of a table affords more robust pattern recognition and utilizes our prior domain knowledge. Deep RL learned a smooth policy with a natural interpretation to data scientists and anesthesia care providers alike.Comment: International Conference on Artificial Intelligence in Medicine 202

Diabetic autonomic neuropathy: evidence for apoptosis in situ in the rat

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/75065/1/j.1365-2982.2004.00524.x.pd

Quantum value indefiniteness

The indeterministic outcome of a measurement of an individual quantum is certified by the impossibility of the simultaneous, definite, deterministic pre-existence of all conceivable observables from physical conditions of that quantum alone. We discuss possible interpretations and consequences for quantum oracles.Comment: 19 pages, 2 tables, 2 figures; contribution to PC0

Thyroid function tests in patients taking thyroid medication in Germany: Results from the population-based Study of Health in Pomerania (SHIP)

<p>Abstract</p> <p>Background</p> <p>Studies from iodine-sufficient areas have shown that a high proportion of patients taking medication for thyroid diseases have thyroid stimulating hormone (TSH) levels outside the reference range. Next to patient compliance, inadequate dosing adjustment resulting in under- and over-treatment of thyroid disease is a major cause of poor therapy outcomes. Using thyroid function tests, we aim to measure the proportions of subjects, who are under- or over-treated with thyroid medication in a previously iodine-deficient area.</p> <p>Findings</p> <p>Data from 266 subjects participating in the population-based Study of Health in Pomerania (SHIP) were analysed. All subjects were taking thyroid medication. Serum TSH levels were measured using immunochemiluminescent procedures. TSH levels of < 0.27 or > 2.15 mIU/L in subjects younger than 50 years and < 0.19 or > 2.09 mIU/L in subjects 50 years and older, were defined as decreased or elevated, according to the established reference range for the specific study area. Our analysis revealed that 56 of 190 (29.5%) subjects treated with thyroxine had TSH levels outside the reference range (10.0% elevated, 19.5% decreased). Of the 31 subjects taking antithyroid drugs, 12 (38.7%) had TSH levels outside the reference range (9.7% elevated, 29.0% decreased). These proportions were lower in the 45 subjects receiving iodine supplementation (2.2% elevated, 8.9% decreased). Among the 3,974 SHIP participants not taking thyroid medication, TSH levels outside the reference range (2.8% elevated, 5.9% decreased) were less frequent.</p> <p>Conclusion</p> <p>In concordance with previous studies in iodine-sufficient areas, our results indicate that a considerable number of patients taking thyroid medication are either under- or over-treated. Improved monitoring of these patients' TSH levels, compared to the local reference range, is recommended.</p

Properties of Graphene: A Theoretical Perspective

In this review, we provide an in-depth description of the physics of monolayer and bilayer graphene from a theorist's perspective. We discuss the physical properties of graphene in an external magnetic field, reflecting the chiral nature of the quasiparticles near the Dirac point with a Landau level at zero energy. We address the unique integer quantum Hall effects, the role of electron correlations, and the recent observation of the fractional quantum Hall effect in the monolayer graphene. The quantum Hall effect in bilayer graphene is fundamentally different from that of a monolayer, reflecting the unique band structure of this system. The theory of transport in the absence of an external magnetic field is discussed in detail, along with the role of disorder studied in various theoretical models. We highlight the differences and similarities between monolayer and bilayer graphene, and focus on thermodynamic properties such as the compressibility, the plasmon spectra, the weak localization correction, quantum Hall effect, and optical properties. Confinement of electrons in graphene is nontrivial due to Klein tunneling. We review various theoretical and experimental studies of quantum confined structures made from graphene. The band structure of graphene nanoribbons and the role of the sublattice symmetry, edge geometry and the size of the nanoribbon on the electronic and magnetic properties are very active areas of research, and a detailed review of these topics is presented. Also, the effects of substrate interactions, adsorbed atoms, lattice defects and doping on the band structure of finite-sized graphene systems are discussed. We also include a brief description of graphane -- gapped material obtained from graphene by attaching hydrogen atoms to each carbon atom in the lattice.Comment: 189 pages. submitted in Advances in Physic

Biological, clinical and population relevance of 95 loci for blood lipids.

Plasma concentrations of total cholesterol, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol and triglycerides are among the most important risk factors for coronary artery disease (CAD) and are targets for therapeutic intervention. We screened the genome for common variants associated with plasma lipids in &gt;100,000 individuals of European ancestry. Here we report 95 significantly associated loci (P &lt; 5 x 10(-8)), with 59 showing genome-wide significant association with lipid traits for the first time. The newly reported associations include single nucleotide polymorphisms (SNPs) near known lipid regulators (for example, CYP7A1, NPC1L1 and SCARB1) as well as in scores of loci not previously implicated in lipoprotein metabolism. The 95 loci contribute not only to normal variation in lipid traits but also to extreme lipid phenotypes and have an impact on lipid traits in three non-European populations (East Asians, South Asians and African Americans). Our results identify several novel loci associated with plasma lipids that are also associated with CAD. Finally, we validated three of the novel genes-GALNT2, PPP1R3B and TTC39B-with experiments in mouse models. Taken together, our findings provide the foundation to develop a broader biological understanding of lipoprotein metabolism and to identify new therapeutic opportunities for the prevention of CAD

Toward human-in-the-loop PID control based on CACLA reinforcement learning

A self-tuning PID control strategy using a reinforcement learning method, called CACLA (Continuous Actor-critic Learning Automata) is proposed in this paper with the example application of humanin-the-loop physical assistive control. An advantage of using reinforcement learning is that it can be done in an online manner. Moreover, since human is a time-variant system. The demonstration also shows that the reinforcement learning framework would be beneficial to give semi-supervision signal to reinforce the positive learning performance in any time-step

Detection of Gamma-Ray Emission from the Starburst Galaxies M82 and NGC 253 with the Large Area Telescope on Fermi

We report the detection of high-energy gamma-ray emission from two starburst galaxies using data obtained with the Large Area Telescope on board the Fermi Gamma-ray Space Telescope. Steady point-like emission above 200 MeV has been detected at significance levels of 6.8 sigma and 4.8 sigma respectively, from sources positionally coincident with locations of the starburst galaxies M82 and NGC 253. The total fluxes of the sources are consistent with gamma-ray emission originating from the interaction of cosmic rays with local interstellar gas and radiation fields and constitute evidence for a link between massive star formation and gamma-ray emission in star-forming galaxies.Comment: Submitted to ApJ Letter

Fermi Gamma-ray Imaging of a Radio Galaxy

The Fermi Gamma-ray Space Telescope has detected the gamma-ray glow emanating from the giant radio lobes of the radio galaxy Centaurus A. The resolved gamma-ray image shows the lobes clearly separated from the central active source. In contrast to all other active galaxies detected so far in high-energy gamma-rays, the lobe flux constitutes a considerable portion (>1/2) of the total source emission. The gamma-ray emission from the lobes is interpreted as inverse Compton scattered relic radiation from the cosmic microwave background (CMB), with additional contribution at higher energies from the infrared-to-optical extragalactic background light (EBL). These measurements provide gamma-ray constraints on the magnetic field and particle energy content in radio galaxy lobes, and a promising method to probe the cosmic relic photon fields.Comment: 27 pages, includes Supplementary Online Material; corresponding authors: C.C. Cheung, Y. Fukazawa, J. Knodlseder, L. Stawar