Using humanoid robots to study human behavior

Our understanding of human behavior advances as our humanoid robotics work progresses-and vice versa. This team's work focuses on trajectory formation and planning, learning from demonstration, oculomotor control and interactive behaviors. They are programming robotic behavior based on how we humans “program” behavior in-or train-each other

Overcoming Exploration in Reinforcement Learning with Demonstrations

Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.Comment: 8 pages, ICRA 201

Putting Community First: A Promising Approach to Federal Collaboration for Environmental Improvement: An Evaluation of the Community Action for a Renewed Environment (CARE) Demonstration Program

This report is an independent evaluation of the Environmental Protection Agency's (EPA) Community Action for a Renewed Environment (CARE) Demonstration Program, a community-driven process that uses the best available data to help communities set priorities and take action on their greatest environmental risks. CARE fosters local partnerships that seek participation from business, government, organizations, residents and EPA staff. It also supports a public, transparent planning and implementation process based on collaborative decision-making and shared action.Key FindingsThe National Academy Panel overseeing this effort was impressed by the dedication of the EPA staff to this unique initiative and commended the EPA for its efforts to partner with communities in achieving important long-term and sustainable environmental improvements at the local level. Recommended actions for the CARE Program include: (1) develop and implement a multifaceted information sharing approach; (2) coordinate and refine internal program management activities; and (3) develop a strategic plan and a business plan for CARE

CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environments

In this paper we study a new reinforcement learning setting where the environment is non-rewarding, contains several possibly related objects of various controllability, and where an apt agent Bob acts independently, with non-observable intentions. We argue that this setting defines a realistic scenario and we present a generic discrete-state discrete-action model of such environments. To learn in this environment, we propose an unsupervised reinforcement learning agent called CLIC for Curriculum Learning and Imitation for Control. CLIC learns to control individual objects in its environment, and imitates Bob's interactions with these objects. It selects objects to focus on when training and imitating by maximizing its learning progress. We show that CLIC is an effective baseline in our new setting. It can effectively observe Bob to gain control of objects faster, even if Bob is not explicitly teaching. It can also follow Bob when he acts as a mentor and provides ordered demonstrations. Finally, when Bob controls objects that the agent cannot, or in presence of a hierarchy between objects in the environment, we show that CLIC ignores non-reproducible and already mastered interactions with objects, resulting in a greater benefit from imitation

Prediction of intent in robotics and multi-agent systems.

Moving beyond the stimulus contained in observable agent behaviour, i.e. understanding the underlying intent of the observed agent is of immense interest in a variety of domains that involve collaborative and competitive scenarios, for example assistive robotics, computer games, robot-human interaction, decision support and intelligent tutoring. This review paper examines approaches for performing action recognition and prediction of intent from a multi-disciplinary perspective, in both single robot and multi-agent scenarios, and analyses the underlying challenges, focusing mainly on generative approaches

Intelligent Agents for Disaster Management

ALADDIN [1] is a multi-disciplinary project that is developing novel techniques, architectures, and mechanisms for multi-agent systems in uncertain and dynamic environments. The application focus of the project is disaster management. Research within a number of themes is being pursued and this is considering different aspects of the interaction between autonomous agents and the decentralised system architectures that support those interactions. The aim of the research is to contribute to building more robust multi-agent systems for future applications in disaster management and other similar domains