Concurrent Constrained Optimization of Unknown Rewards for Multi-Robot Task Allocation

Task allocation can enable effective coordination of multi-robot teams to accomplish tasks that are intractable for individual robots. However, existing approaches to task allocation often assume that task requirements or reward functions are known and explicitly specified by the user. In this work, we consider the challenge of forming effective coalitions for a given heterogeneous multi-robot team when task reward functions are unknown. To this end, we first formulate a new class of problems, dubbed COncurrent Constrained Online optimization of Allocation (COCOA). The COCOA problem requires online optimization of coalitions such that the unknown rewards of all the tasks are simultaneously maximized using a given multi-robot team with constrained resources. To address the COCOA problem, we introduce an online optimization algorithm, named Concurrent Multi-Task Adaptive Bandits (CMTAB), that leverages and builds upon continuum-armed bandit algorithms. Experiments involving detailed numerical simulations and a simulated emergency response task reveal that CMTAB can effectively trade-off exploration and exploitation to simultaneously and efficiently optimize the unknown task rewards while respecting the team's resource constraints.Comment: 9 pages, 5 figures, to be published in RSS 202

On the Utility of Koopman Operator Theory in Learning Dexterous Manipulation Skills

Despite impressive dexterous manipulation capabilities enabled by learning-based approaches, we are yet to witness widespread adoption beyond well-resourced laboratories. This is likely due to practical limitations, such as significant computational burden, inscrutable learned behaviors, sensitivity to initialization, and the considerable technical expertise required for implementation. In this work, we investigate the utility of Koopman operator theory in alleviating these limitations. Koopman operators are simple yet powerful control-theoretic structures to represent complex nonlinear dynamics as linear systems in higher dimensions. Motivated by the fact that complex nonlinear dynamics underlie dexterous manipulation, we develop a Koopman operator-based imitation learning framework to learn the desired motions of both the robotic hand and the object simultaneously. We show that Koopman operators are surprisingly effective for dexterous manipulation and offer a number of unique benefits. Notably, policies can be learned analytically, drastically reducing computation burden and eliminating sensitivity to initialization and the need for painstaking hyperparameter optimization. Our experiments reveal that a Koopman operator-based approach can perform comparably to state-of-the-art imitation learning algorithms in terms of success rate and sample efficiency, while being an order of magnitude faster. Policy videos can be viewed at https://sites.google.com/view/kodex-corl.Comment: This work has been accepted for an oral presentation at CORL 202

MARBLER: An Open Platform for Standarized Evaluation of Multi-Robot Reinforcement Learning Algorithms

Multi-agent reinforcement learning (MARL) has enjoyed significant recent progress, thanks to deep learning. This is naturally starting to benefit multi-robot systems (MRS) in the form of multi-robot RL (MRRL). However, existing infrastructure to train and evaluate policies predominantly focus on challenges in coordinating virtual agents, and ignore characteristics important to robotic systems. Few platforms support realistic robot dynamics, and fewer still can evaluate Sim2Real performance of learned behavior. To address these issues, we contribute MARBLER: Multi-Agent RL Benchmark and Learning Environment for the Robotarium. MARBLER offers a robust and comprehensive evaluation platform for MRRL by marrying Georgia Tech's Robotarium (which enables rapid prototyping on physical MRS) and OpenAI's Gym framework (which facilitates standardized use of modern learning algorithms). MARBLER offers a highly controllable environment with realistic dynamics, including barrier certificate-based obstacle avoidance. It allows anyone across the world to train and deploy MRRL algorithms on a physical testbed with reproducibility. Further, we introduce five novel scenarios inspired by common challenges in MRS and provide support for new custom scenarios. Finally, we use MARBLER to evaluate popular MARL algorithms and provide insights into their suitability for MRRL. In summary, MARBLER can be a valuable tool to the MRS research community by facilitating comprehensive and standardized evaluation of learning algorithms on realistic simulations and physical hardware. Links to our open-source framework and the videos of real-world experiments can be found at https://shubhlohiya.github.io/MARBLER/.Comment: 7 pages, 3 figures, submitted to MRS 2023, for the associated website, see https://shubhlohiya.github.io/MARBLER

From Coexistence to Collaboration: Towards Reliable Collaborative Robots

Presented online October 14, 2020, 12:15 p.m.-1:15 p.m.IRIM Seminar Series: Session 4Dr. Harish Ravichandar is currently a Research Scientist in the School of Interactive Computing and a faculty member of the Institute of Intelligent Robots and Machines (IRIM) at Georgia Institute of Technology, where he joined as a Postdoctoral Fellow in 2018. He received his M.S. degree in Electrical and Computer Engineering from the University of Florida in 2014 and his Ph.D. in Electrical and Computer Engineering from the University of Connecticut in 2018. His current research interests span the areas of robot learning, human-robot interaction, and multi-agent systems. His work has been recognized by the ASME DSCC Best Student Robotics Paper Award (2015), IEEE CSS Video Contest Award (2015), UTC Institute for Advanced System Engineering Graduate Fellowship (2016-2018), and Georgia Tech's College of Computing Outstanding Post-Doctoral Research Award (2019) and Outstanding Research Scientist Award (2020).Runtime: 53:45 minutesThe field of robotics has made incredible progress over the past several decades. Indeed, we have built impressive robots capable of performing complex and intricate tasks in a variety of domains. Most modern robots, however, passively coexist with humans while performing pre-specified tasks in predictable environments. As robots become an increasingly integral part of our everyday lives -- from factory floors to our living rooms -- it is imperative that we build robots that can reliably operate and actively collaborate in unstructured environments. This talk will present three key aspects of collaborative robotics that will help us make progress toward this goal. Specifically, we will discuss algorithmic techniques that enable robots to i) consistently and reliably perform manipulation tasks, ii) understand and predict the behavior of other agents involved, and iii) effectively collaborate with other robots and humans

Structured Methods for an Unstructured World: Toward Reliable and Collaborative Robots

IRIM Research Symposium 2021, August 25, 2021 from 8:30 a.m-4:00 p.m. Klaus Advanced Computing Building Room 1116, Georgia Institute of Technology, Atlanta, GA.The Institute for Robotics and Intelligent Machines (IRIM) Annual Research Symposium is an interdisciplinary educational and networking event that presents new findings from IRIM faculty and encourages exchange between roboticists from across the Institute’s Schools, Colleges, and GTRI. External academics, IRIM Industry Partners and interested companies are welcome to attend the event.Harish Ravichandar is affiliated with the School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA.Runtime: 16:37 minute