Learning Rewards from Linguistic Feedback

We explore unconstrained natural language feedback as a learning signal for artificial agents. Humans use rich and varied language to teach, yet most prior work on interactive learning from language assumes a particular form of input (e.g., commands). We propose a general framework which does not make this assumption, using aspect-based sentiment analysis to decompose feedback into sentiment about the features of a Markov decision process. We then perform an analogue of inverse reinforcement learning, regressing the sentiment on the features to infer the teacher's latent reward function. To evaluate our approach, we first collect a corpus of teaching behavior in a cooperative task where both teacher and learner are human. We implement three artificial learners: sentiment-based "literal" and "pragmatic" models, and an inference network trained end-to-end to predict latent rewards. We then repeat our initial experiment and pair them with human teachers. All three successfully learn from interactive human feedback. The sentiment models outperform the inference network, with the "pragmatic" model approaching human performance. Our work thus provides insight into the information structure of naturalistic linguistic feedback as well as methods to leverage it for reinforcement learning.Comment: 9 pages, 4 figures. AAAI '2

Irbesartan in Marfan syndrome (AIMS): a double-blind, placebo-controlled randomised trial

BACKGROUND: Irbesartan, a long acting selective angiotensin-1 receptor inhibitor, in Marfan syndrome might reduce aortic dilatation, which is associated with dissection and rupture. We aimed to determine the effects of irbesartan on the rate of aortic dilatation in children and adults with Marfan syndrome. METHODS: We did a placebo-controlled, double-blind randomised trial at 22 centres in the UK. Individuals aged 6-40 years with clinically confirmed Marfan syndrome were eligible for inclusion. Study participants were all given 75 mg open label irbesartan once daily, then randomly assigned to 150 mg of irbesartan (increased to 300 mg as tolerated) or matching placebo. Aortic diameter was measured by echocardiography at baseline and then annually. All images were analysed by a core laboratory blinded to treatment allocation. The primary endpoint was the rate of aortic root dilatation. This trial is registered with ISRCTN, number ISRCTN90011794. FINDINGS: Between March 14, 2012, and May 1, 2015, 192 participants were recruited and randomly assigned to irbesartan (n=104) or placebo (n=88), and all were followed for up to 5 years. Median age at recruitment was 18 years (IQR 12-28), 99 (52%) were female, mean blood pressure was 110/65 mm Hg (SDs 16 and 12), and 108 (56%) were taking β blockers. Mean baseline aortic root diameter was 34·4 mm in the irbesartan group (SD 5·8) and placebo group (5·5). The mean rate of aortic root dilatation was 0·53 mm per year (95% CI 0·39 to 0·67) in the irbesartan group compared with 0·74 mm per year (0·60 to 0·89) in the placebo group, with a difference in means of -0·22 mm per year (-0·41 to -0·02, p=0·030). The rate of change in aortic Z score was also reduced by irbesartan (difference in means -0·10 per year, 95% CI -0·19 to -0·01, p=0·035). Irbesartan was well tolerated with no observed differences in rates of serious adverse events. INTERPRETATION: Irbesartan is associated with a reduction in the rate of aortic dilatation in children and young adults with Marfan syndrome and could reduce the incidence of aortic complications

Six-Year Study of the Incidence of Herpes in Genital and Nongenital Cultures in a Central Kentucky Medical Center Patient Population

Herpes infections are among the most common sexually transmitted diseases and are the most common cause of genital ulcer disease in the United States. This study addresses the changing distribution of herpes simplex virus type 1 (HSV-1) and HSV-2 in patients presenting for evaluation of herpetic infections. Viral culture results from the University of Kentucky Clinical Microbiology Laboratory were reviewed for a 6-year period (1994 through 1999). Data were collected on patient sex, site of culture, and culture result. These data were analyzed statistically to identify yearly trends. Of the 4,498 cultures analyzed, nearly equal proportions of HSV-1 (13.3%) and HSV-2 (12.0%) were detected for an overall culture positivity rate of 25.3%. Approximately two-thirds of all positive cultures were from women. Although HSV-2 remained the predominant type of genital herpes, over the 6-year span of this study, there was a trend toward increasing proportions of HSV-1 genitalis, with 31.8% of male patients and 44.8% of female patients demonstrating HSV-1 genitalis by 1999. The majority of patients with HSV in nongenital sites grew HSV-1. Although there was significant yearly variation, HSV-2 was isolated from only 9.4% of patients with nongenital HSV for the entire 6-year period. This study therefore concludes that HSV-2 remains primarily a genital pathogen, while HSV-1 is taking on an increasingly important role in causing genital ulcer disease in addition to being the primary nongenital HSV