Learning robust policies for object manipulation with robot swarms

Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly. Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source. In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution. Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm. These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots

Robust learning of object assembly tasks with an invariant representation of robot swarms

— Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly. Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source. In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution. Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm. These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots

Slow relaxation in the Ising model on a small-world network with strong long-range interactions

We consider the Ising model on a small-world network, where the long-range interaction strength $J_2$ is in general different from the local interaction strength $J_1$, and examine its relaxation behaviors as well as phase transitions. As $J_2/J_1$ is raised from zero, the critical temperature also increases, manifesting contributions of long-range interactions to ordering. However, it becomes saturated eventually at large values of $J_2/J_1$ and the system is found to display very slow relaxation, revealing that ordering dynamics is inhibited rather than facilitated by strong long-range interactions. To circumvent this problem, we propose a modified updating algorithm in Monte Carlo simulations, assisting the system to reach equilibrium quickly.Comment: 5 pages, 5 figure

Solution of Abel's integral equation using Tikhonov regularization

Peer reviewed: YesNRC publication: Ye

Estimating the welfare loss to households from natural disasters in developing countries: a contingent valuation study of flooding in Vietnam

Background: Natural disasters have severe impacts on the health and well-being of affected households. However, we find evidence that official damage cost assessments for floods and other natural disasters in Vietnam, where households have little or no insurance, clearly underestimate the total economic damage costs of these events as they do not include the welfare loss from mortality, morbidity and reduced well-being experienced by the households affected by the floods. This should send a message to the local communities and national authorities that higher investments in flood alleviation, reduction and adaptive measures can be justified since the social benefits of these measures in terms of avoided damage costs are higher than previously thought. Methods: We pioneer the use of the contingent valuation (CV) approach of willingness-to-contribute (WTC) labour to a flood prevention program, as a measure of the welfare loss experienced by household due to a flooding event. In a face-to-face household survey of 706 households in the Quang Nam province in Central Vietnam, we applied this approach together with reported direct physical damage in order to shed light of the welfare loss experienced by the households. We asked about households’ WTC labour and multiplied their WTC person-days of labour by an estimate for their opportunity cost of time in order to estimate the welfare loss to households from the 2007 floods. Results: The results showed that this contingent valuation (CV) approach of asking about willingness-to-pay in-kind avoided the main problems associated with applying CV in developing countries. Conclusion: Thus, the CV approach of WTC labour instead of money is promising in terms of capturing the total welfare loss of natural disasters to households, and promising in terms of further application in other developing countries and for other types of natural disasters

Summary of personal neutron dosemeter results obtained within the EVIDOS project

Within the EC project EVIDOS (‘Evaluation of Individual Dosimetry in Mixed Neutron and Photon Radiation Fields'), different types of active neutron personal dosemeters (and some passive ones) were tested in workplace fields at nuclear installations in Europe. The results of the measurements which have been performed up to now are summarised and compared to our currently best estimates of the personal dose equivalent Hp(10). Under- and over-readings by more than a factor of two for the same dosemeter in different workplace fields indicate that in most cases the use of field-specific correction factors is require

Application of workplace correction factors to dosemeter results for the assessment of personal doses at nuclear facilities

Ratios of Hp(10) and H*(10) were determined with reference instruments in a number of workplace fields within the nuclear industry and used to derive workplace-specific correction factors. When commercial survey meter results together with these factors were applied to the results of the locally used personal dosemeters their results improved and became within 0.7 and 1.7 of the reference values or better depending on the response of the survey meter. A similar result was obtained when a correction was determined with a prototype reference instrument for Hp(10) after adjustment of its response. Commercially available survey instruments both for photon and neutron H*(10) measurements agreed with the reference instruments in most cases to within 0.5-1.5. Those conclusions are derived from results reported within the EC supported EVIDOS contrac

Combined In Silico, In Vivo, and In Vitro Studies Shed Insights into the Acute Inflammatory Response in Middle-Aged Mice

We combined in silico, in vivo, and in vitro studies to gain insights into age-dependent changes in acute inflammation in response to bacterial endotoxin (LPS). Time-course cytokine, chemokine, and NO2-/NO3- data from "middle-aged" (6-8 months old) C57BL/6 mice were used to re-parameterize a mechanistic mathematical model of acute inflammation originally calibrated for "young" (2-3 months old) mice. These studies suggested that macrophages from middle-aged mice are more susceptible to cell death, as well as producing higher levels of pro-inflammatory cytokines, vs. macrophages from young mice. In support of the in silico-derived hypotheses, resident peritoneal cells from endotoxemic middle-aged mice exhibited reduced viability and produced elevated levels of TNF-α, IL-6, IL-10, and KC/CXCL1 as compared to cells from young mice. Our studies demonstrate the utility of a combined in silico, in vivo, and in vitro approach to the study of acute inflammation in shock states, and suggest hypotheses with regard to the changes in the cytokine milieu that accompany aging. © 2013 Namas et al

Ensemble Models of Neutrophil Trafficking in Severe Sepsis

A hallmark of severe sepsis is systemic inflammation which activates leukocytes and can result in their misdirection. This leads to both impaired migration to the locus of infection and increased infiltration into healthy tissues. In order to better understand the pathophysiologic mechanisms involved, we developed a coarse-grained phenomenological model of the acute inflammatory response in CLP (cecal ligation and puncture)-induced sepsis in rats. This model incorporates distinct neutrophil kinetic responses to the inflammatory stimulus and the dynamic interactions between components of a compartmentalized inflammatory response. Ensembles of model parameter sets consistent with experimental observations were statistically generated using a Markov-Chain Monte Carlo sampling. Prediction uncertainty in the model states was quantified over the resulting ensemble parameter sets. Forward simulation of the parameter ensembles successfully captured experimental features and predicted that systemically activated circulating neutrophils display impaired migration to the tissue and neutrophil sequestration in the lung, consequently contributing to tissue damage and mortality. Principal component and multiple regression analyses of the parameter ensembles estimated from survivor and non-survivor cohorts provide insight into pathologic mechanisms dictating outcome in sepsis. Furthermore, the model was extended to incorporate hypothetical mechanisms by which immune modulation using extracorporeal blood purification results in improved outcome in septic rats. Simulations identified a sub-population (about of the treated population) that benefited from blood purification. Survivors displayed enhanced neutrophil migration to tissue and reduced sequestration of lung neutrophils, contributing to improved outcome. The model ensemble presented herein provides a platform for generating and testing hypotheses in silico, as well as motivating further experimental studies to advance understanding of the complex biological response to severe infection, a problem of growing magnitude in humans