Hierarchies of Planning and Reinforcement Learning for Robot Navigation

Solving robotic navigation tasks via reinforcement learning (RL) is challenging due to their sparse reward and long decision horizon nature. However, in many navigation tasks, high-level (HL) task representations, like a rough floor plan, are available. Previous work has demonstrated efficient learning by hierarchal approaches consisting of path planning in the HL representation and using sub-goals derived from the plan to guide the RL policy in the source task. However, these approaches usually neglect the complex dynamics and sub-optimal sub-goal-reaching capabilities of the robot during planning. This work overcomes these limitations by proposing a novel hierarchical framework that utilizes a trainable planning policy for the HL representation. Thereby robot capabilities and environment conditions can be learned utilizing collected rollout data. We specifically introduce a planning policy based on value iteration with a learned transition model (VI-RL). In simulated robotic navigation tasks, VI-RL results in consistent strong improvement over vanilla RL, is on par with vanilla hierarchal RL on single layouts but more broadly applicable to multiple layouts, and is on par with trainable HL path planning baselines except for a parking task with difficult non-holonomic dynamics where it shows marked improvements.Comment: 7 pages, 5 figures, 2021 IEEE International Conference on Robotics and Automation (ICRA), v2: DOI number adde

Congenital syndactyly in cattle: four novel mutations in the low density lipoprotein receptor-related protein 4 gene (LRP4)

BACKGROUND: Isolated syndactyly in cattle, also known as mulefoot, is inherited as an autosomal recessive trait with variable penetrance in different cattle breeds. Recently, two independent mutations in the bovine LRP4 gene have been reported as the primary cause of syndactyly in the Holstein and Angus cattle breeds. RESULTS: We confirmed the previously described LRP4 exon 33 two nucleotide substitution in most of the affected Holstein calves and revealed additional evidence for allelic heterogeneity by the identification of four new LRP4 non-synonymous point mutations co-segregating in Holstein, German Simmental and Simmental-Charolais families. CONCLUSION: We confirmed a significant role of LRP4 mutations in the pathogenesis of congenital syndactyly in cattle. The newly detected missense mutations in the LRP4 gene represent independent mutations affecting different conserved protein domains. However, the four newly described LRP4 mutations do still not explain all analyzed cases of syndactyly

Learning Hierarchical Planning-Based Policies from Offline Data

Hierarchical policy architectures incorporating some planning component into the top-level have shown superior performance and generalization in agent navigation tasks. Cost or safety reasons may, however, prevent training in an online (RL) fashion with continuous environment interaction. We therefore propose HORIBLe-VRN, an algorithm to learn a hierarchical policy with a top-level planning-based module from pre-collected data. A key challenge is to deal with the unknown, latent high-level (HL) actions. Our algorithm features an EM-style hierarchical imitation learning stage, incorporating HL action inference, and a subsequent offline RL refinement stage for the top-level policy. We empirically evaluate HORIBLe-VRN in a long horizon, sparse reward agent navigation task, investigating performance, generalization capabilities, and robustness with respect to sub-optimal demonstration data

Childhood Maltreatment, Borderline Personality Features, and Coping as Predictors of Intimate Partner Violence

Intimate partner violence (IPV) is a serious mental and physical health concern worldwide. Although previous research suggests that childhood maltreatment increases the risk for IPV, the underlying psychological mechanisms of this relationship are not yet entirely understood. Borderline personality (BP) features may play an important role in the cycle of violence, being associated with interpersonal violence in both childhood and adult relationships. The present study investigated whether BP features mediate the relationship between childhood maltreatment and IPV, differentiating between perpetration and victimization, and taking maladaptive stress coping and gender into account. Self-reports on IPV, childhood trauma, BP features, and maladaptive stress coping were collected in a mixed (nonclinical and clinical) sample of 703 adults (n = 537 female, n = 166 male), using an online survey. A serial mediation analysis (PROCESS) was performed to quantify the direct effect of childhood maltreatment on IPV and its indirect effects through BP features and maladaptive coping. Childhood maltreatment severity significantly positively predicted IPV perpetration as well as victimization. BP features, but not coping, partially mediated this relationship. Follow-up analyses suggest that affective instability and interpersonal disturbances (e.g., separation concerns) play an important role in IPV perpetration, while interpersonal and identity disturbances may mediate the effect of childhood maltreatment on IPV victimization. In clinical practice, attention should be paid not only to histories of childhood abuse and neglect but also to BP features, which may be possible risk factors for IPV.Stress and Psychopatholog