Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment

The introduction of collaborative robots in industrial environments reinforces the need to provide these robots with better cognition to accomplish their tasks while fostering worker safety without entering into safety shutdowns that reduce workflow and production times. This paper presents a novel strategy that combines the execution of contact-rich tasks, namely disassembly, with real-time collision avoidance through machine learning for safe human-robot interaction. Specifically, a goal-conditioned reinforcement learning approach is proposed, in which the removal direction of a peg, of varying friction, tolerance, and orientation, is subject to the location of a human collaborator with respect to a 7-degree-of-freedom manipulator at each time step. For this purpose, the suitability of three state-of-the-art actor-critic algorithms is evaluated, and results from simulation and real-world experiments are presented. In reality, the policy’s deployment is achieved through a new scalable multi-control framework that allows a direct transfer of the control policy to the robot and reduces response times. The results show the effectiveness, generalization, and transferability of the proposed approach with two collaborative robots against static and dynamic obstacles, leveraging the set of available solutions in non-monotonic tasks to avoid a potential collision with the human worker

SOCIALGYM 2.0: Simulator for Multi-Agent Social Robot Navigation in Shared Human Spaces

We present SocialGym 2, a multi-agent navigation simulator for social robot research. Our simulator models multiple autonomous agents, replicating real-world dynamics in complex environments, including doorways, hallways, intersections, and roundabouts. Unlike traditional simulators that concentrate on single robots with basic kinematic constraints in open spaces, SocialGym 2 employs multi-agent reinforcement learning (MARL) to develop optimal navigation policies for multiple robots with diverse, dynamic constraints in complex environments. Built on the PettingZoo MARL library and Stable Baselines3 API, SocialGym 2 offers an accessible python interface that integrates with a navigation stack through ROS messaging. SocialGym 2 can be easily installed and is packaged in a docker container, and it provides the capability to swap and evaluate different MARL algorithms, as well as customize observation and reward functions. We also provide scripts to allow users to create their own environments and have conducted benchmarks using various social navigation algorithms, reporting a broad range of social navigation metrics. Projected hosted at: https://amrl.cs.utexas.edu/social_gym/index.htmlComment: Submitted to RSS 202

Deep Reinforcement Learning Controller for 3D Path-following and Collision Avoidance by Autonomous Underwater Vehicles

Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, in complex systems, such as autonomous underwater vehicles performing the dual objective of path-following and collision avoidance, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning (DRL) techniques, to develop autonomous agents capable of achieving this hybrid objective without having \`a priori knowledge about the goal or the environment. Our results demonstrate the viability of DRL in path-following and avoiding collisions toward achieving human-level decision making in autonomous vehicle systems within extreme obstacle configurations

Assessment of simulated and real-world autonomy performance with small-scale unmanned ground vehicles

Off-road autonomy is a challenging topic that requires robust systems to both understand and navigate complex environments. While on-road autonomy has seen a major expansion in recent years in the consumer space, off-road systems are mostly relegated to niche applications. However, these applications can provide safety and navigation to dangerous areas that are the most suited for autonomy tasks. Traversability analysis is at the core of many of the algorithms employed in these topics. In this thesis, a Clearpath Robotics Jackal vehicle is equipped with a 3D Ouster laser scanner to define and traverse off-road environments. The Mississippi State University Autonomous Vehicle Simulator (MAVS) and the Navigating All Terrains Using Robotic Exploration (NATURE) autonomy stack are used in conjunction with the small-scale vehicle platform to traverse uneven terrain and collect data. Additionally, the NATURE stack is used as a point of comparison between a MAVS simulated and physical Clearpath Robotics Jackal vehicle in testing

Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning

Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.acceptedVersionPeer reviewe

Collision avoidance control for Unmanned Autonomous Vehicles (UAV): Recent advancements and future prospects

The recent advances in collision avoidance technologies for unmanned vehicles such as UAVs, AUVs, AGVs, and USVs have greatly advanced the industry. Their lower cost and acceptability of high-risk missions have enabled the development of collision avoidance controllers for autonomous vehicles. These low-maintenance gadgets are also portable, need low maintenance, and enable continuous monitoring to occur near real-time. This may be said; however it would be incorrect, because collision avoidance controllers have been related with compromises that affect data dependability. Research on collision avoidance controls is quickly developing; therefore it is distributed throughout multiple papers, projects, and grey literature. This report critically reviews the recent relevant research on creating collision avoidance systems for autonomous vehicles. Typically, the assessment measures are dependent on the algorithm's use case and the platform's capabilities. The full evaluation of the benefits and drawbacks of the most prevalent approaches in the present state of the art is provided based on 7 metrics which are complexity, communication dependence, pre-mission planning, robustness, 3D compatibility, real-time performance and escape trajectories

Collision avoidance control for Unmanned Autonomous Vehicles (UAV): Recent advancements and future prospects

873-883The recent advances in collision avoidance technologies for unmanned vehicles such as UAVs, AUVs, AGVs, and USVs have greatly advanced the industry. Their lower cost and acceptability of high-risk missions have enabled the development of collision avoidance controllers for autonomous vehicles. These low-maintenance gadgets are also portable, need low maintenance, and enable continuous monitoring to occur near real-time. This may be said; however it would be incorrect, because collision avoidance controllers have been related with compromises that affect data dependability. Research on collision avoidance controls is quickly developing; therefore it is distributed throughout multiple papers, projects, and grey literature. This report critically reviews the recent relevant research on creating collision avoidance systems for autonomous vehicles. Typically, the assessment measures are dependent on the algorithm's use case and the platform's capabilities. The full evaluation of the benefits and drawbacks of the most prevalent approaches in the present state of the art is provided based on 7 metrics which are complexity, communication dependence, pre-mission planning, robustness, 3D compatibility, real-time performance and escape trajectories