Exploiting Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning

Swarm systems constitute a challenging problem for reinforcement learning (RL) as the algorithm needs to learn decentralized control policies that can cope with limited local sensing and communication abilities of the agents. While it is often difficult to directly define the behavior of the agents, simple communication protocols can be defined more easily using prior knowledge about the given task. In this paper, we propose a number of simple communication protocols that can be exploited by deep reinforcement learning to find decentralized control policies in a multi-robot swarm environment. The protocols are based on histograms that encode the local neighborhood relations of the gents and can also transmit task-specific information, such as the shortest distance and direction to a desired target. In our framework, we use an adaptation of Trust Region Policy Optimization to learn complex collaborative tasks, such as formation building and building a communication link. We evaluate our findings in a simulated 2D-physics environment, and compare the implications of different communication protocols

A Novel Mechanism of Soluble HLA-G Mediated Immune Modulation: Downregulation of T Cell Chemokine Receptor Expression and Impairment of Chemotaxis

BACKGROUND: In recent years, many immunoregulatory functions have been ascribed to soluble HLA-G (sHLA-G). Since chemotaxis is crucial for an efficient immune response, we have investigated for the first time the effects of sHLA-G on chemokine receptor expression and function in different human T cell populations. METHODOLOGY/PRINCIPAL FINDINGS: T cell populations isolated from peripheral blood were stimulated in the presence or absence of sHLA-G. Chemokine receptors expression was evaluated by flow cytometry. sHLA-G downregulated expression of i) CCR2, CXCR3 and CXCR5 in CD4(+) T cells, ii) CXCR3 in CD8(+) T cells, iii) CXCR3 in Th1 clones iv) CXCR3 in TCR Vdelta2gamma9 T cells, and upregulated CXCR4 expression in TCR Vdelta2gamma9 T cells. sHLA-G inhibited in vitro chemotaxis of i) CD4(+) T cells towards CCL2, CCL8, CXCL10 and CXCL11, ii) CD8(+) T cells towards CXCL10 and CXCL11, iii) Th1 clones towards CXCL10, and iv) TCR Vdelta2gamma9 T cells towards CXCL10 and CXCL11. Downregulation of CXCR3 expression on CD4+ T cells by sHLA-G was partially reverted by adding a blocking antibody against ILT2/CD85j, a receptor for sHLA-G, suggesting that sHLA-G downregulated chemokine receptor expression mainly through the interaction with ILT2/CD85j. Follicular helper T cells (T(FH)) were isolated from human tonsils and stimulated as described above. sHLA-G impaired CXCR5 expression in T(FH) and chemotaxis of the latter cells towards CXCL13. Moreover, sHLA-G expression was detected in tonsils by immunohistochemistry, suggesting a role of sHLA-G in local control of T(FH) cell chemotaxis. Intracellular pathways were investigated by Western Blot analysis on total extracts from CD4+ T cells. Phosphorylation of Stat5, p70 s6k, beta-arrestin and SHP2 was modulated by sHLA-G treatment. CONCLUSIONS/SIGNIFICANCE: Our data demonstrated that sHLA-G impairs expression and functionality of different chemokine receptors in T cells. These findings delineate a novel mechanism whereby sHLA-G modulates T cell recruitment in physiological and pathological conditions

Illuminating the life of GPCRs

The investigation of biological systems highly depends on the possibilities that allow scientists to visualize and quantify biomolecules and their related activities in real-time and non-invasively. G-protein coupled receptors represent a family of very dynamic and highly regulated transmembrane proteins that are involved in various important physiological processes. Since their localization is not confined to the cell surface they have been a very attractive "moving target" and the understanding of their intracellular pathways as well as the identified protein-protein-interactions has had implications for therapeutic interventions. Recent and ongoing advances in both the establishment of a variety of labeling methods and the improvement of measuring and analyzing instrumentation, have made fluorescence techniques to an indispensable tool for GPCR imaging. The illumination of their complex life cycle, which includes receptor biosynthesis, membrane targeting, ligand binding, signaling, internalization, recycling and degradation, will provide new insights into the relationship between spatial receptor distribution and function. This review covers the existing technologies to track GPCRs in living cells. Fluorescent ligands, antibodies, auto-fluorescent proteins as well as the evolving technologies for chemical labeling with peptide- and protein-tags are described and their major applications concerning the GPCR life cycle are presented

The Art of Gate-Crashing Bringing HRI into users' homes

Special purpose service robots have already entered the market and their users homes. Also the idea of the general purpose service robot or personal robot companion is increasingly discussed and investigated. To probe human-robot interaction with a mobile robot in arbitrary domestic settings, we conducted a study in eight different homes. Based on previous results from laboratory studies we identified particular interaction situations which should be studied thoroughly in real home settings. Based upon the collected sensory data from the robot we found that the different environments influenced the spatial management observable during our subjects' interaction with the robot. We also validated empirically that the concept of spatial prompting can aid spatial management and communication, and assume this concept to be helpful for Human-Robot Interaction (HRI) design. In this article we report on our exploratory field study and our findings regarding, in particular, the spatial management observed during show episodes and movement through narrow passages

Interaction awareness for joint environment exploration

An important goal for research on service robots is the cooperation of a human and a robot as team. A service robot in a domestic environment needs to build a representation of its future workspace that corresponds to the human user's understanding of these surroundings. But it also needs to apply this model about the "where" and "what" in its current interaction to allow communication about objects and places in a human-adequate way. In this paper we present the integration of a hierarchical robotic mapping system into an interactive framework controlled by a dialog system. The goal is to use interactively acquired environment models to implement a robot with interaction aware behaviors. A major contribution of this work is a three-level hierarchy of spatial representation affecting three different communication dimensions. This hierarchy is consequently applied in the design of the grounding-based dialog, laser-based topological mapping, and an objects attention system. We demonstrate the benefits of this integration for learning and tour guiding in a humancomprehensible interaction between a robot and its user in a home-tour scenario. The enhanced interaction capabilities are crucial for developing a new generation of robots that will be accepted not only as service robots but also as robot companions