Multi-Task Policy Search for Robotics

© 2014 IEEE.Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics. Training individual policies for every single potential task is often impractical, especially for continuous task variations, requiring more principled approaches to share and transfer knowledge among similar tasks. We present a novel approach for learning a nonlinear feedback policy that generalizes across multiple tasks. The key idea is to define a parametrized policy as a function of both the state and the task, which allows learning a single policy that generalizes across multiple known and unknown tasks. Applications of our novel approach to reinforcement and imitation learning in realrobot experiments are shown

Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning

Over the last years, there has been substantial progress in robust manipulation in unstructured environments. The long-term goal of our work is to get away from precise, but very expensive robotic systems and to develop affordable, potentially imprecise, self-adaptive manipulator systems that can interactively perform tasks such as playing with children. In this paper, we demonstrate how a low-cost off-the-shelf robotic system can learn closed-loop policies for a stacking task in only a handful of trials-from scratch. Our manipulator is inaccurate and provides no pose feedback. For learning a controller in the work space of a Kinect-style depth camera, we use a model-based reinforcement learning technique. Our learning method is data efficient, reduces model bias, and deals with several noise sources in a principled way during long-term planning. We present a way of incorporating state-space constraints into the learning process and analyze the learning gain by exploiting the sequential structure of the stacking task

Multi-Task Policy Search

Learning policies that generalize across multiple tasks is an important and challenging research topic in reinforcement learning and robotics. Training individual policies for every single potential task is often impractical, especially for continuous task variations, requiring more principled approaches to share and transfer knowledge among similar tasks. We present a novel approach for learning a nonlinear feedback policy that generalizes across multiple tasks. The key idea is to define a parametrized policy as a function of both the state and the task, which allows learning a single policy that generalizes across multiple known and unknown tasks. Applications of our novel approach to reinforcement and imitation learning in real-robot experiments are shown

Do HIV treatment eligibility expansions crowd out the sickest? Evidence from rural South Africa

OBJECTIVE: The 2015 WHO recommendation to initiate all HIV patients on antiretroviral therapy (ART) at diagnosis could potentially overextend health systems and crowd out sicker patients, mitigating the policy's impact. We evaluate whether South Africa's prior eligibility expansion from CD4 ≤200 to CD4 ≤350 cells/μL reduced ART uptake in the sickest patients. METHODS: Using data on all patients presenting to the Hlabisa HIV Treatment and Care Program in KwaZulu-Natal from April 2010 - June 2012 (n=13,809), we assessed the impact of the August 2011 eligibility expansion on the number of patients seeking care, number initiating ART, and time from HIV diagnosis to ART initiation among patients always eligible (CD4 0-200), newly eligible (CD4 201-350), and not yet eligible by CD4 count (>350). We used interrupted time series methods to control for long-run trends and isolate the effect of the policy. RESULTS: Expanding ART eligibility led to an increased number of patients initiating ART per month [+95.5; 95% CI (-1.3; 192.3)]. Newly eligible patients (CD4 201-350) initiated treatment 47% faster than before (95% CI 19%; 82%), while the sickest patients (CD4 ≤200) saw no decline in the monthly number of patients initiating treatment or the rate of treatment uptake. CONCLUSION: The Hlabisa program successfully extended ART to patients with CD4 ≤350 cells/μL, while ensuring that the sickest patients did not experience delays in ART initiation. Treatment programs must be vigilant to maintain quality of care for the sickest as countries move to treat all patients irrespective of CD4 count. This article is protected by copyright. All rights reserved

Building governance and energy efficiency: Mapping the interdisciplinary challenge

Improving the energy efficiency of multi-owned properties (MoPs)—commonly known as apartment or condominium buildings—is central to the achievement of European energy targets. However, little work to date has focused on how to facilitate retrofit in this context. Drawing on interdisciplinary Social Sciences and Humanities expertise in academia, policy and practice, this chapter posits that decision-making processes within MoPs might provide a key to the retrofit challenge. Existing theories or models of decision-making, applied in the MoP context, might help to explain how collective retrofit decisions are taken—or overlooked. Insights from case studies and practitioners are also key. Theories of change might then be employed to develop strategies to facilitate positive retrofit decisions. The chapter maps the issues and sets an agenda for further interdisciplinary research in this novel area

Diagnosing Dementia in the Clinical Setting: Can Amyloid PET Provide Additional Value Over Cerebrospinal Fluid?

Cerebrospinal fluid (CSF) measures of amyloid and tau are the first-line Alzheimer's disease biomarkers in many clinical centers. We assessed if and when the addition of amyloid PET following CSF measurements provides added diagnostic value. Twenty patients from a cognitive clinic, who had undergone detailed assessment including CSF measures, went on to have amyloid PET. The treating neurologist's working diagnosis, and degree of diagnostic certainty, was assessed both before and after the PET. Amyloid PET changed the diagnosis in 7/20 cases. Amyloid PET can provide added diagnostic value, particularly in young-onset, atypical dementias, where CSF results are borderline and diagnostic uncertainty remains

Migrations and habitat use of the smooth hammerhead shark (Sphyrna zygaena) in the Atlantic Ocean

The smooth hammerhead shark, Sphyrna zygaena, is a cosmopolitan semipelagic shark captured as bycatch in pelagic oceanic fisheries, especially pelagic longlines targeting swordfish and/or tunas. From 2012 to 2016, eight smooth hammerheads were tagged with Pop-up Satellite Archival Tags in the inter-tropical region of the Northeast Atlantic Ocean, with successful transmissions received from seven tags (total of 319 tracking days). Results confirmed the smooth hammerhead is a highly mobile species, as the longest migration ever documented for this species (> 6600 km) was recorded. An absence of a diel vertical movement behavior was noted, with the sharks spending most of their time at surface waters (0-50 m) above 23 degrees C. The operating depth of the pelagic long-line gear was measured with Minilog Temperature and Depth Recorders, and the overlap with the species vertical distribution was calculated. The overlap is taking place mainly during the night and is higher for juveniles (similar to 40% of overlap time). The novel information presented can now be used to contribute to the provision of sustainable management tools and serve as input for Ecological Risk Assessments for smooth hammerheads caught in Atlantic pelagic longline fisheries.Oceanario de Lisboa through Project "SHARK-TAG: Migrations and habitat use of the smooth hammerhead shark in the Atlantic Ocean"; Investigador-FCT from the Portuguese Foundation for Science and Technology (FCT, Fundacao para a Ciencia e Tecnologia) [Ref: IF/00253/2014]; EU European Social Fund; Programa Operacional Potencial Human

Gender Differences in Mortality and CD4 Count Response Among Virally Suppressed HIV-Positive Patients

Treatment outcomes for antiretroviral therapy (ART) patients may vary by gender, but estimates from current evidence may be confounded by disease stage and adherence. We investigated the gender differences in treatment response among HIV-positive patients virally suppressed within 6 months of treatment initiation