Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the agents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols.Comment: 13 pages, 4 figures, version 2, accepted at ANTS 201

Effectiveness of lurasidone in schizophrenia or schizoaffective patients switched from other antipsychotics: a 6-month, open-label, extension study

Objective. To evaluate the long-term safety and tolerability of lurasidone in schizophrenia and schizoaffective disorder patients switched to lurasidone. Method. Patients in this multicenter, 6-month open-label, flexible-dose, extension study had completed a core 6-week randomized trial in which clinically stable, but symptomatic, outpatients with schizophrenia or schizoaffective disorder were switched to lurasidone. Patients started the extension study on treatment with the same dose of lurasidone taken at study endpoint of the 6-week core study; following this, lurasidone was flexibly dosed (40-120 mg/day), if clinically indicated, starting on Day 7 of the extension study. The primary safety endpoints were the proportion of patients with treatment emergent adverse events (AEs), serious AEs, or who discontinued due to AEs. Secondary endpoints included metabolic variables and measures of extrapyramidal symptoms and akathisia, as well as the Positive and Negative Syndrome Scale (PANSS), Clinical Global Impressions-Severity (CGI-S), and the Calgary Depression Scale for Schizophrenia (CDSS). The study was conducted from August 2010 to November 2011. Results. Of the 198 patients who completed the 6-week core study, 149 (75.3%) entered the extension study and 148 received study medication. A total of 98 patients (65.8%) completed the 6-month extension study. Lurasidone 40, 80, and 120 mg were the modal daily doses for 19 (12.8%), 65 (43.9%), and 64 (43.2%) of patients, respectively. Overall mean (SD) daily lurasidone dose was 102.0 mg (77.1). The most commonly reported AEs were insomnia (13 patients [8.8%]), nausea (13 patients [8.8%]), akathisia (12 patients [8.1%]), and anxiety (9 patients [6.1%]). A total of 16 patients (10.8%) had at least one AE leading to discontinuation from the study. Consistent with prior studies of lurasidone, there was no signal for clinically relevant adverse changes in body weight, lipids, glucose, insulin, or prolactin. Movement disorder rating scales did not demonstrate meaningful changes. Treatment failure (defined as any occurrence of discontinuation due to insufficient clinical response, exacerbation of underlying disease, or AE) was observed for 19 patients (12.8% of patients entering) and median time to treatment failure was 58 days (95% CI 22-86). The discontinuation rate due to any cause was 50/148 (33.8%), and median time to discontinuation was 62 days (95% CI 30-75). The mean PANSS total score, mean CGI-S score, and mean CDSS score decreased consistently from core study baseline across extension visits, indicating an improvement in overall condition. Conclusions. In this 6-month, open-label extension study, treatment with lurasidone was generally well-tolerated with sustained improvement in efficacy measures observed in outpatients with schizophrenia or schizoaffective disorder who had switched to lurasidone from a broad range of antipsychotic agents

Prevalence and correlates of antipsychotic polypharmacy in children and adolescents receiving antipsychotic treatment*

Antipsychotic polypharmacy (APP), which is common in adults with psychotic disorders, is of unproven efficacy and raises safety concerns. Although youth are increasingly prescribed antipsychotics, little is known about APP in this population. We performed a systematic PubMed search (last update 26 January 2013) of studies reporting the prevalence of APP in antipsychotic-treated youth. Summary statistics and statistical tests were calculated at the study level and not weighted by sample size. Fifteen studies (n=58041, range 68-23183) reported on APP in youth [mean age=13.4 +/- 1.7 yr, 67.1 +/- 10.2% male, 77.9 +/- 27.4% treated with second-generation antipsychotics (SGAs)]. Data collected in these studies covered 1993-2008. The most common diagnoses were attention-deficit hyperactivity disorder (ADHD; 39.9 +/- 23.5%) and conduct disorder/oppositional defiant disorder (CD/ODD; 33.6 +/- 24.8). In studies including predominantly children (mean age=yr, N=5), the most common diagnosis were ADHD (50.6 +/- 25.4%) and CD/ODD (39.5 +/- 27.5%); while in studies with predominantly adolescents (mean age=13yr, N=7) the most common diagnoses were schizophrenia-spectrum disorders (28.6 +/- 23.8%), anxiety disorders (26.9 +/- 14.9%) and bipolar-spectrum disorders (26.6 +/- 7.0%), followed closely by CD/ODD (25.8 +/- 17.7). The prevalence of APP among antipsychotic-treated youth was 9.6 +/- 7.2% (5.9 +/- 4.5% in child studies, 12.0 +/- 7.9% in adolescent studies, p=0.15). Higher prevalence of APP was correlated with a bipolar disorder or schizophrenia diagnosis (p=0.019) and APP involving SGA+SGA combinations (p=0.0027). No correlation was found with APP definition [1d (N=10) vs. \u3e30-90d (N=5), p=0.88]. Despite lacking safety and efficacy data, APP in youth is not uncommon, even in samples predominantly consisting of non-psychotic patients. The duration, clinical motivations and effectiveness of this practice require further study

Emotional abuse interacts with borderline personality in adolescent inpatients with binge-purging eating disorders

Purpose Childhood abuse is associated with an increased risk of developing eating disorders (EDs) as well as personality disorders (PDs). However, their interaction is still uncertain, particularly in adolescents. This study investigates the correlations between childhood emotional neglect (CEN), childhood emotional abuse (CEA), and obsessive-compulsive and borderline personality styles in female adolescent inpatients with eating disorders (EDs). Methods One hundred and twenty-eight inpatients (ages 14-18) were assessed, 54 were diagnosed with restricting-type anorexia nervosa (AN-R) and 33 with a binge-purging ED [BP-ED; comprising patients with binge-purging type anorexia nervosa (AN-BP), n = 15, and bulimia nervosa (BN), n = 18]. Fifty healthy participants made up the control group (CG). CEN and CEA were assessed with the Childhood Trauma Questionnaire, while the Personality Style and Disorder Inventory was implemented to determine personality styles. Results A MANOVA revealed a significant main effect of CEA on spontaneous-borderline personality style [F(8,119) = 17.1, p < 0.001, η2 = 0.126], as well as a main effect of ED group on spontaneous-borderline [F(2,119) = 3.1, p = 0.048, η2 = 0.050]. A significant interaction between ED group, CEA, and spontaneous-borderline was found [F(2,119) = 3.5, p = 0.034, η2 = 0.055] with BP-ED showing significantly higher scores in CEA (9.3 ± 4.0) and in spontaneous-borderline (14.2 ± 6.2). Conclusions Considering CEA and borderline personality style in adolescent inpatients with BN or AN-BP may help improve the understanding of the etiology and maintenance of BP-ED and provide more effective treatment targets. Level of evidence Level III, case–control analytic study