Framework for deep reinforcement learning in Webots virtual environments

Abstract

Reinforcement learning (RL) algorithms, particularly deep reinforcement learning (DRL), have shown transformative potential in robotics by enabling adaptive behaviour in virtual environments. However, a comprehensive framework for efficiently testing, training, and deploying robots in these environments remains underexplored. This study introduces a standardized, open-source framework designed specifically for the Webots simulation environment. Supported by a robust methodology, the framework integrates innovative design patterns and the digital twin (DT) concept with three distinct design patterns for structuring agent-environment interaction, notably including a novel pattern aimed at improving sim-toreal transferability, to enhance RL workflows. The proposed framework is validated through experimental studies on both a model the inverted pendulum and a production-grade Pioneer 3-AT robotic platform. The experiments highlight the framework’s ability to bridge the gap between virtual training and real-world implementation. All resources, including the framework, methodology, and experimental configurations, are openly accessible on GitHub