Guided Domain Randomization With Meta Reinforcement Learning

Reinforcement learning policies often need to be trained in simulations of the real environments, since training directly on the real agents can either be not feasible or expensive. When transferring those trained policies to the real agents, they often either fail completely or degrade significantly in performance, due to modelling deficiencies of the used simulators and/or mismatch of parameter values between these two domains. Domain randomization methods used in literature have attempted to solve this problem by training policies in environments whose parameters (or a subset of them) are sampled from ranges during training, usually resulting in robust policies. Despite these policies performing better upon transfer, robustness may still come at some cost of performance when compared to policies that could adapt to the current context. Therefore, this thesis proposes Meta Guided Domain Randomization methods where domain randomization techniques are combined with meta reinforcement learning algorithms instead of standard reinforcement learning ones, in order to train adaptive policies. This thesis also presents an attempt to analyze and improve the Active Domain Randomization algorithm - one of the popular guided domain randomization methods from the literature. The improved Active Domain Randomization algorithm is then compared to its proposed meta guided domain randomization counterpart in sim-to-sim experiments on one of MuJoCo® environments, and is shown to offer an improvement in average performance over the entire space of the sampled environments’ parameters

Individual heat map assessments demonstrate vestronidase alfa treatment response in a highly heterogeneous mucopolysaccharidosis VII study population.

Mucopolysaccharidosis (MPS) VII is an ultra-rare, progressively debilitating, life-threatening lysosomal disease caused by deficiency of the enzyme, β-glucuronidase. Vestronidase alfa is an approved enzyme replacement therapy for MPS VII. UX003-CL301 was a phase 3, randomized, placebo-controlled, blind-start study examining the efficacy and safety of vestronidase alfa 4 mg/kg intravenously administered every 2 weeks to 12 patients with MPS VII. Due to the rarity of disease, broad eligibility criteria resulted in a highly heterogeneous population with variable symptoms. For an integrated view of the diverse data, the changes from baseline (or randomization for the placebo period) in clinical endpoints were grouped into three functional domains (mobility, fatigue, and fine motor + self-care) and analyzed post-hoc as subject-level heat maps. Mobility assessments included the 6-minute walk test, 3-minute stair climb test, Bruininks-Oseretsky test (BOT-2) gross motor function subtests, and patient-reported outcome assessments (PROs) related to movement, pain, and ambulation. Fatigue assessments included the Pediatric Quality of Life Multidimensional Fatigue Scale and other fatigue-related PROs. Fine motor + self-care assessments included BOT-2 fine motor function subtests and PROs for eating, dressing, hygiene, and caregiver assistance. Most subjects showed improvement in at least one domain. Two subjects improved in two or more domains and two subjects did not show clear improvement in any domain. Both severely and mildly affected subjects improved with vestronidase alfa in clinical assessments, PRO results, or both. Heat map analysis demonstrates how subjects responded to treatment across multiple domains, providing a useful visual tool for studying rare diseases with variable symptoms

Efficacy and safety of secukinumab administration by autoinjector in patients with psoriatic arthritis: results from a randomized, placebo-controlled trial (FUTURE 3)

Background: The study aimed to assess 52-week efficacy and safety of secukinumab self-administration by autoinjector in patients with active psoriatic arthritis (PsA) in the FUTURE 3 study (ClinicalTrials.gov NCT01989468). Methods: Patients (≥ 18 years of age; N = 414) with active PsA were randomized 1:1:1 to subcutaneous (s.c.) secukinumab 300 mg, 150 mg, or placebo at baseline, weeks 1, 2, 3, and 4, and every 4 weeks thereafter. Per clinical response, placebo-treated patients were re-randomized to s.c. secukinumab 300 or 150 mg at week 16 (nonresponders) or week 24 (responders) and stratified at randomization by prior anti-tumor necrosis factor (TNF) therapy (anti-TNF-naïve, 68.1%; intolerant/inadequate response (anti-TNF-IR), 31.9%). The primary endpoint was the proportion of patients achieving at least 20% improvement in American College of Rheumatology response criteria (ACR20) at week 24. Autoinjector usability was evaluated by Self-Injection Assessment Questionnaire (SIAQ). Results: Overall, 92.1% (300 mg), 91.3% (150 mg), and 93.4% (placebo) of patients completed 24 weeks, and 84.9% (300 mg) and 79.7% (150 mg) completed 52 weeks. In the overall population (combined anti-TNF-naïve and anti-TNF-IR), ACR20 response rate at week 24 was significantly higher in secukinumab groups (300 mg, 48.2% (p < 0.0001); 150 mg, 42% (p < 0.0001); placebo, 16.1%) and was sustained through 52 weeks. SIAQ results showed that more than 93% of patients were satisfied/very satisfied with autoinjector usage. Secukinumab was well tolerated with no new or unexpected safety signals reported. Conclusions: Secukinumab provided sustained improvements in signs and symptoms in active PsA patients through 52 weeks. High acceptability of autoinjector was observed. The safety profile was consistent with that reported previously

Asymmetric Actor Critic for Image-Based Robot Learning

Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains. However, Robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator. While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator. In this work, we exploit the full state observability in the simulator to train better policies which take as input only partial observations (RGBD images). We do this by employing an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) gets rendered images as input. We show experimentally on a range of simulated tasks that using these asymmetric inputs significantly improves performance. Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block. We achieve this simulation to real world transfer without training on any real world data.Comment: Videos of experiments can be found at http://www.goo.gl/b57WT