Human-to-Robot Imitation in the Wild

We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real world, they are either sample inefficient or are constrained to lab settings. Meanwhile, there has been a lot of success in processing passive, unstructured human data. We propose tackling this problem via an efficient one-shot robot learning algorithm, centered around learning from a third-person perspective. We call our method WHIRL: In-the-Wild Human Imitating Robot Learning. WHIRL extracts a prior over the intent of the human demonstrator, using it to initialize our agent's policy. We introduce an efficient real-world policy learning scheme that improves using interactions. Our key contributions are a simple sampling-based policy optimization approach, a novel objective function for aligning human and robot videos as well as an exploration method to boost sample efficiency. We show one-shot generalization and success in real-world settings, including 20 different manipulation tasks in the wild. Videos and talk at https://human2robot.github.ioComment: Published at RSS 2022. Demos at https://human2robot.github.i

Structured World Models from Human Videos

We tackle the problem of learning complex, general behaviors directly in the real world. We propose an approach for robots to efficiently learn manipulation skills using only a handful of real-world interaction trajectories from many different settings. Inspired by the success of learning from large-scale datasets in the fields of computer vision and natural language, our belief is that in order to efficiently learn, a robot must be able to leverage internet-scale, human video data. Humans interact with the world in many interesting ways, which can allow a robot to not only build an understanding of useful actions and affordances but also how these actions affect the world for manipulation. Our approach builds a structured, human-centric action space grounded in visual affordances learned from human videos. Further, we train a world model on human videos and fine-tune on a small amount of robot interaction data without any task supervision. We show that this approach of affordance-space world models enables different robots to learn various manipulation skills in complex settings, in under 30 minutes of interaction. Videos can be found at https://human-world-model.github.ioComment: RSS 2023. Website at https://human-world-model.github.i

Efficient RL via Disentangled Environment and Agent Representations

Agents that are aware of the separation between themselves and their environments can leverage this understanding to form effective representations of visual input. We propose an approach for learning such structured representations for RL algorithms, using visual knowledge of the agent, such as its shape or mask, which is often inexpensive to obtain. This is incorporated into the RL objective using a simple auxiliary loss. We show that our method, Structured Environment-Agent Representations, outperforms state-of-the-art model-free approaches over 18 different challenging visual simulation environments spanning 5 different robots. Website at https://sear-rl.github.io/Comment: ICML 2023. Website at https://sear-rl.github.io

Impact on inequities in health indicators: effect of implementing the integrated management of neonatal and childhood illness programme in Haryana, India

Background: A trial to evaluate the Integrated Management of Neonatal and Childhood Illness (IMNCI) strategy showed that the intervention resulted in lower infant mortality and improved infant care practices. In this paper, we present the results of a secondary analysis to examine the effect of the IMNCI strategy on inequities in health indicators. Methods: The trial was a cluster-randomized controlled trial in 18 primary health centre areas. For this analysis, the population was divided into subgroups by wealth status (using Principal Component Analysis), religion and caste, education of mother and sex of the infant. Multiple linear regression analysis was used to examine inequity gradients in neonatal and post-neonatal mortality, care practices and care seeking, and the differences in these gradients between intervention and control clusters. Findings: Inequity in post-neonatal infant mortality by wealth status was lower in the intervention as compared to control clusters (adjusted difference in gradients 2.2 per 1000, 95% confidence interval (CI) 0 to 4.4 per 1000, P = 0.053). The intervention had no effect on inequities in neonatal mortality. The intervention resulted in a larger effect on breastfeeding within one hour of birth in poorer families (difference in inequity gradients 3.0%, CI 1.5 to 4.5, P < 0.001), in lower caste and minorities families, and in infants of mothers with fewer years of schooling. The intervention also reduced gender inequity in care seeking for severe neonatal illness from an appropriate provider (difference in inequity gradients 9.3%, CI 0.4 to 18.2, P = 0.042). Conclusions: Implementation of IMNCI reduced inequities in post-neonatal mortality, and newborn care practices (particularly starting breastfeeding within an hour of birth) and health care-seeking for severe illness. In spite of the intervention substantial inequities remained in the intervention group and therefore further efforts to ensure that health programs reach the vulnerable population subgroups are required