Distral: Robust Multitask Reinforcement Learning

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tasks. In practice, however, this is not usually observed, because gradients from different tasks can interfere negatively, making learning unstable and sometimes even less data efficient. Another issue is the different reward schemes between tasks, which can easily lead to one task dominating the learning of a shared model. We propose a new approach for joint training of multiple tasks, which we refer to as Distral (Distill & transfer learning). Instead of sharing parameters between the different workers, we propose to share a "distilled" policy that captures common behaviour across tasks. Each worker is trained to solve its own task while constrained to stay close to the shared policy, while the shared policy is trained by distillation to be the centroid of all task policies. Both aspects of the learning process are derived by optimizing a joint objective function. We show that our approach supports efficient transfer on complex 3D environments, outperforming several related methods. Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning

The relationship between in-hospital location and outcomes of care in patients of a large general medical service

Background: The discrepancy between the number of admissions and the allocation of hospital beds means many patients admitted under the care of a general medical service can be placed in other departments’ wards. These patients are called “outliers” and their outcomes are unknown. Aims: To examine the relation between the proportion of time each patient spent in their “home ward” during an index admission and the outcomes of that hospital stay. Methods: Data from Flinders Medical Centre’s (FMC) patient journey database were extracted and analysed. The analysis was carried out on the patient journeys of patients admitted under the General Medicine units. Results: Outlier patients’ length of stay (LOS) was significantly shorter than that of the inlier patients (110.7 hours cf 141.9 hours; p < 0.001).They had a reduced risk of readmission within 28 days of discharge from hospital. Outlier patients’ discharge summaries were less likely to be completed within a week (64.3% cf 78.0%; p < 0.001). Being an outlier patient increased the risk-adjusted risk of in-hospital mortality by over 40%. 50% of deaths in the outlier group occurred within 48 hours of admission. Outlier patients had spent longer in the Emergency Department (ED) waiting for a bed (6.3 hours cf 5.3 hours; p < 0.001) but duration of ED stay was not an independent predictor of mortality risk. Conclusion: Outlier patients had significantly shorter LOS in hospital, but significantly greater in-patient death rates. Surviving outlier patients had lower rates of readmission but lower rates of discharge summary completion

Identification of an env-defective HIV-1 mutant capable of spontaneous reversion to a wild-type phenotype in certain T-cell lines

BACKGROUND: Attempts to eradicate HIV from cellular reservoirs are vital but depend on a clear understanding of how viral variants are transmitted and survive in the different cell types that constitute such reservoirs. Mutations in the env gene of HIV may be able to exert a differential influence on viral transmission ability in regard to cell-free and cell-associated viral forms. METHODS: The ability of HIV containing an env G367R mutation in cell-free and cell-associated viruses to cause infection and to revert to wild-type was measured using several T cell lines. To determine factors that might potentially influence the reversion of G367R, we studied each of entry inhibitors, inhibitors of cellular endocytosis, and modulators of cell growth and activation. RESULTS: We demonstrate that an HIV-1 variant containing a G367R substitution within the CD4 binding site of gp120 was non-infectious as free virus in culture but was infectious when infected cells were co-cultured with certain T cell lines or when cells were transfected by a relevant proviral plasmid. Differences in viral infectivity by cell-associated G367R viruses were determined by the type of target cell employed, regardless which type of donor cell was used. Reversion was slowed or inhibited by entry inhibitors and by inhibitors of cellular endocytosis. Interleukin 2 was able to block G367R reversion in only one of the T cell lines studied but not in the other, while phorbol 12-myristate 13-acetate (PMA) inhibited G367R reversion in all the T cell lines. CONCLUSIONS: Env-defective HIV may have a different phenotype as cell-free versus cell-associated virus. The persistence of defective forms can potentially lead to the emergence of virulent forms. The heterogeneity of cell types that constitute the HIV reservoir can contribute to viral variability, even among similar types of cells. This is the first demonstration of a mutation in the HIV envelope, i.e. G367R, that can compromise infection by cell-free virus but less severely by cell-associated virus and that does so in a cell type-dependent manner

Path dependent scaling of geometric phase near a quantum multi-critical point

We study the geometric phase of the ground state in a one-dimensional transverse XY spin chain in the vicinity of a quantum multi-critical point. We approach the multi-critical point along different paths and estimate the geometric phase by applying a rotation in all spins about z-axis by an angle $\eta$. Although the geometric phase itself vanishes at the multi-critical point, the derivative with respect to the anisotropy parameter of the model shows peaks at different points on the ferromagnetic side close to it where the energy gap is a local minimum; we call these points `quasi-critical'. The value of the derivative at any quasi-critical point scales with the system size in a power-law fashion with the exponent varying continuously with the parameter $\alpha$ that defines a path, upto a critical value $\alpha = \alpha_{c}=2$. For $\alpha > \alpha_{c}$, or on the paramagnetic side no such peak is observed. Numerically obtained results are in perfect agreement with analytical predictions.Comment: 5 pages, 6 figure

Geometry and Mechanics

IASS-IACM 2008 Session: Geometry and Mechanics -- Session Organizers: Kai-Uwe BLETZINGER (TU Munich), Fehmi CIRAK (University of Cambridge) -- Keynote Lecture: "Modeling and computation of patient-specific vascular fluid-structure interaction using Isogeometric Analysis" by Yuri BAZILEVS , Victor M. CALO, Thomas J. R. HUGHES (University of Texas at Austin), Yongie ZHANG (Carnegie Mellon University) -- Keynote Lecture: "Optimal shapes of mechanically motivated surfaces" by Kai-Uwe BLETZINGER , Matthias FIRL, Johannes LINHARD, Roland WUCHNER (TU Munich) -- "Subdivision shells for nonsmooth and branching geometries" by Quan LONG, Fehmi CIRAK (University of Cambridge) -- "Water landing analyses with explicit finite element method" by John T. WANG (NASA Langley Research Center) -- "On a geometrically exact contact description for shells: From linear approximations for shells to high-order FEM" by Alexander KONYUKHOV, Karl SCHWEIZERHOF (University of Karlsruhe

Metabolic Disturbances Associated with Systemic Lupus Erythematosus

The metabolic disturbances that underlie systemic lupus erythematosus are currently unknown. A metabolomic study was executed, comparing the sera of 20 SLE patients against that of healthy controls, using LC/MS and GC/MS platforms. Validation of key differences was performed using an independent cohort of 38 SLE patients and orthogonal assays. SLE sera showed evidence of profoundly dampened glycolysis, Krebs cycle, fatty acid β oxidation and amino acid metabolism, alluding to reduced energy biogenesis from all sources. Whereas long-chain fatty acids, including the n3 and n6 essential fatty acids, were significantly reduced, medium chain fatty acids and serum free fatty acids were elevated. The SLE metabolome exhibited profound lipid peroxidation, reflective of oxidative damage. Deficiencies were noted in the cellular anti-oxidant, glutathione, and all methyl group donors, including cysteine, methionine, and choline, as well as phosphocholines. The best discriminators of SLE included elevated lipid peroxidation products, MDA, gamma-glutamyl peptides, GGT, leukotriene B4 and 5-HETE. Importantly, similar elevations were not observed in another chronic inflammatory autoimmune disease, rheumatoid arthritis. To sum, comprehensive profiling of the SLE metabolome reveals evidence of heightened oxidative stress, inflammation, reduced energy generation, altered lipid profiles and a pro-thrombotic state. Resetting the SLE metabolome, either by targeting selected molecules or by supplementing the diet with essential fatty acids, vitamins and methyl group donors offers novel opportunities for disease modulation in this disabling systemic autoimmune ailment

Respiratory viral pathogens among Singapore military servicemen 2009 - 2012: Epidemiology and clinical characteristics

10.1186/1471-2334-14-204BMC Infectious Diseases141-BIDM

Exhausted Cytotoxic Control of Epstein-Barr Virus in Human Lupus

Systemic Lupus Erythematosus (SLE) pathology has long been associated with an increased Epstein-Barr Virus (EBV) seropositivity, viremia and cross-reactive serum antibodies specific for both virus and self. It has therefore been postulated that EBV triggers SLE immunopathology, although the mechanism remains elusive. Here, we investigate whether frequent peaks of EBV viral load in SLE patients are a consequence of dysfunctional anti-EBV CD8+ T cell responses. Both inactive and active SLE patients (n = 76 and 42, respectively), have significantly elevated EBV viral loads (P = 0.003 and 0.002, respectively) compared to age- and sex-matched healthy controls (n = 29). Interestingly, less EBV-specific CD8+ T cells are able to secrete multiple cytokines (IFN-γ, TNF-α, IL-2 and MIP-1β) in inactive and active SLE patients compared to controls (P = 0.0003 and 0.0084, respectively). Moreover, EBV-specific CD8+ T cells are also less cytotoxic in SLE patients than in controls (CD107a expression: P = 0.0009, Granzyme B release: P = 0.0001). Importantly, cytomegalovirus (CMV)-specific responses were not found significantly altered in SLE patients. Furthermore, we demonstrate that EBV-specific CD8+ T cell impairment is a consequence of their Programmed Death 1 (PD-1) receptor up-regulation, as blocking this pathway reverses the dysfunctional phenotype. Finally, prospective monitoring of lupus patients revealed that disease flares precede EBV reactivation. In conclusion, EBV-specific CD8+ T cell responses in SLE patients are functionally impaired, but EBV reactivation appears to be an aggravating consequence rather than a cause of SLE immunopathology. We therefore propose that autoimmune B cell activation during flares drives frequent EBV reactivation, which contributes in a vicious circle to the perpetuation of immune activation in SLE patients