Human Apprenticeship Learning via Kernel-based Inverse Reinforcement Learning

It has been well demonstrated that inverse reinforcement learning (IRL) is an effective technique for teaching machines to perform tasks at human skill levels given human demonstrations (i.e., human to machine apprenticeship learning). This paper seeks to show that a similar application can be demonstrated with human learners. That is, given demonstrations from human experts inverse reinforcement learning techniques can be used to teach other humans to perform at higher skill levels (i.e., human to human apprenticeship learning). To show this two experiments were conducted using a simple, real-time web game where players were asked to touch targets in order to earn as many points as possible. For the experiment player performance was defined as the number of targets a player touched, irrespective of the points that a player actually earned. This allowed for in-game points to be modified and the effect of these alterations on performance measured. At no time were participants told the true performance metric. To determine the point modifications IRL was applied on demonstrations of human experts playing the game. The results of the experiment show with significance that performance improved over the control for select treatment groups. Finally, in addition to the experiment, we also detail the algorithmic challenges we faced when conducting the experiment and the techniques we used to overcome them.Comment: 31 pages, 23 figures, Submitted to Journal of Artificial Intelligence Research, "for source code, see https://github.com/mrucker/kpirl-kla

Gene expression profiling of mammary gland development reveals putative roles for death receptors and immune mediators in post-lactational regression

INTRODUCTION: In order to gain a better understanding of the molecular processes that underlie apoptosis and tissue regression in mammary gland, we undertook a large-scale analysis of transcriptional changes during the mouse mammary pregnancy cycle, with emphasis on the transition from lactation to involution. METHOD: Affymetrix microarrays, representing 8618 genes, were used to compare mammary tissue from 12 time points (one virgin, three gestation, three lactation and five involution stages). Six animals were used for each time point. Common patterns of gene expression across all time points were identified and related to biological function. RESULTS: The majority of significantly induced genes in involution were also differentially regulated at earlier stages in the pregnancy cycle. This included a marked increase in inflammatory mediators during involution and at parturition, which correlated with leukaemia inhibitory factor–Stat3 (signal transducer and activator of signalling-3) signalling. Before involution, expected increases in cell proliferation, biosynthesis and metabolism-related genes were observed. During involution, the first 24 hours after weaning was characterized by a transient increase in expression of components of the death receptor pathways of apoptosis, inflammatory cytokines and acute phase response genes. After 24 hours, regulators of intrinsic apoptosis were induced in conjunction with markers of phagocyte activity, matrix proteases, suppressors of neutrophils and soluble components of specific and innate immunity. CONCLUSION: We provide a resource of mouse mammary gene expression data for download or online analysis. Here we highlight the sequential induction of distinct apoptosis pathways in involution and the stimulation of immunomodulatory signals, which probably suppress the potentially damaging effects of a cellular inflammatory response while maintaining an appropriate antimicrobial and phagocytic environment

Sugar sensing responses to low and high light in leaves of the C4 model grass Setaria viridis

Although sugar regulate photosynthesis, the signalling pathways underlying this process remain elusive, especially for C4 crops. To address this knowledge gap and identify potential candidate genes, we treated Setaria viridis (C4 model) plants acclimated to medium light intensity (ML, 500 µmol m-2 s-1) with low (LL, 50 µmol m-2 s-1) or high (HL, 1000 µmol m-2 s-1) light for 4 days and observed the consequences on carbon metabolism and the transcriptome of source leaves. LL impaired photosynthesis and reduced leaf content of signalling sugars (glucose, sucrose and trehalose-6-phosphate). Contrastingly, HL strongly induced sugar accumulation without repressing photosynthesis. LL more profoundly impacted leaf transcriptome, including photosynthetic genes. LL and HL contrastingly altered the expression of HXK and SnRK1 sugar sensors and trehalose pathway genes. The expression of key target genes of HXK and SnRK1 were affected by LL and sugar depletion, while surprisingly HL and strong sugar accumulation only slightly repressed the SnRK1 signalling pathway. In conclusion, we demonstrate that LL profoundly impacted photosynthesis and the transcriptome of S. viridis source leaves, while HL altered sugar levels more than LL. We also present the first evidence that sugar signalling pathways in C4 source leaves may respond to light intensity and sugar accumulation differently to C3 source leaves

The clinical effectiveness of transurethral incision of the prostate : a systematic review of randomised controlled trials

The original publication is available at www.springerlink.com.Peer reviewedPostprin

Catalytic enantioselective synthesis of heterocyclic vicinal fluoroamines using asymmetric protonation : a method development and mechanistic study

Funding: Leverhulme Trust (Grant Number(s): RPG-2015-308), University of St Andrews, University of Manchester.A catalytic enantioselective synthesis of heterocyclic vicinal fluoroamines is reported. A chiral Brønsted acid promotes aza‐Michael addition to fluoroalkenyl heterocycles to give a prochiral enamine intermediate that undergoes asymmetric protonation upon rearomatization. The reaction accommodates a range of azaheterocycles and nucleophiles, generating the C−F stereocentre in high enantioselectivity, and is also amenable to stereogenic C−CF3 bonds. Extensive DFT calculations provided evidence for stereocontrolled proton transfer from catalyst to substrate as the rate‐determining step, and showed the importance of steric interactions from the catalyst's alkyl groups in enforcing the high enantioselectivity. Crystal structure data show the dominance of noncovalent interactions in the core structure conformation, enabling modulation of the conformational landscape. Ramachandran‐type analysis of conformer distribution and Protein Data Bank mining indicated that benzylic fluorination by this approach has the potential to improve the potency of several marketed drugs.Publisher PDFPeer reviewe

Hearing of malaria mosquitoes is modulated by a beta-adrenergic-like octopamine receptor which serves as insecticide target

Malaria mosquitoes acoustically detect their mating partners within large swarms that form transiently at dusk. Indeed, male malaria mosquitoes preferably respond to female flight tones during swarm time. This phenomenon implies a sophisticated context- and time-dependent modulation of mosquito audition, the mechanisms of which are largely unknown. Using transcriptomics, we identify a complex network of candidate neuromodulators regulating mosquito hearing in the species Anopheles gambiae. Among them, octopamine stands out as an auditory modulator during swarm time. In-depth analysis of octopamine auditory function shows that it affects the mosquito ear on multiple levels: it modulates the tuning and stiffness of the flagellar sound receiver and controls the erection of antennal fibrillae. We show that two α- and β-adrenergic-like octopamine receptors drive octopamine's auditory roles and demonstrate that the octopaminergic auditory control system can be targeted by insecticides. Our findings highlight octopamine as key for mosquito hearing and mating partner detection and as a potential novel target for mosquito control

Consensus Recommendations for Sick Day Medication Guidance for People With Diabetes, Kidney, or Cardiovascular Disease:A Modified Delphi Process

Rationale &amp; Objective: Sick day medication guidance (SDMG) involves withholding or adjusting specific medications in the setting of acute illnesses that could contribute to complications such as hypotension, acute kidney injury (AKI), or hypoglycemia. We sought to achieve consensus among clinical experts on recommendations for SDMG that could be studied in future intervention studies. Study Design: A modified Delphi process following guidelines for conducting and reporting Delphi studies. Setting &amp; Participants: An international group of clinicians with expertise relevant to SDMG was recruited through purposive and snowball sampling. A scoping review of the literature was presented, followed by 3 sequential rounds of development, refinement, and voting on recommendations. Meetings were held virtually and structured to allow the participants to provide their input and rapidly prioritize and refine ideas.Outcome: Opinions of participants were measured as the percentage who agreed with each recommendation, whereas consensus was defined as &gt;75% agreement. Analytical Approach: Quantitative data were summarized using counts and percentages. A qualitative content analysis was performed to capture the context of the discussion around recommendations and any additional considerations brought forward by participants. Results: The final panel included 26 clinician participants from 4 countries and 10 clinical disciplines. Participants reached a consensus on 42 specific recommendations: 5 regarding the signs and symptoms accompanying volume depletion that should trigger SDMG; 6 regarding signs that should prompt urgent contact with a health care provider (including a reduced level of consciousness, severe vomiting, low blood pressure, presence of ketones, tachycardia, and fever); and 14 related to scenarios and strategies for patient self-management (including frequent glucose monitoring, checking ketones, fluid intake, and consumption of food to prevent hypoglycemia). There was consensus that renin-angiotensin system inhibitors, diuretics, nonsteroidal anti-inflammatory drugs, sodium/glucose cotransporter 2 inhibitors, and metformin should be temporarily stopped. Participants recommended that insulin, sulfonylureas, and meglitinides be held only if blood glucose was low and that basal and bolus insulin be increased by 10%-20% if blood glucose was elevated. There was consensus on 6 recommendations related to the resumption of medications within 24-48 hours of the resolution of symptoms and the presence of normal patterns of eating and drinking. Limitations: Participants were from high-income countries, predominantly Canada. Findings may not be generalizable to implementation in other settings. Conclusions: A multidisciplinary panel of clinicians reached a consensus on recommendations for SDMG in the presence of signs and symptoms of volume depletion, as well as self-management strategies and medication instructions in this setting. These recommendations may inform the design of future trials of SDMG strategies.</p