Reinforcement Q-Learning using OpenAI Gym

Abstract. Q-Learning is an off-policy algorithm for reinforcement learning, that can be used to find optimal policies in Markovian domains. This thesis is about how Q-Learning can be applied to a test environment in the OpenAI Gym toolkit. The utility of testing the algorithm on a problem case is to find out how well it performs as well proving the practical utility of the algorithm. This thesis starts off with a general overview of reinforcement learning as well as the Markov decision process, both of which are crucial in understanding the theoretical groundwork that Q-Learning is based on. After that we move on to discussing the Q-Learning technique itself and dissect the algorithm in detail. We also go over OpenAI Gym toolkit and how it can be used to test the algorithm’s functionality. Finally, we introduce the problem case and apply the algorithm to solve it and analyse the results. The reasoning for this thesis is the rise of reinforcement learning and its increasing relevance in the future as technological progress allows for more and more complex and sophisticated applications of machine learning and artificial intelligence

CaiRL: A High-Performance Reinforcement Learning Environment Toolkit

This paper addresses the dire need for a platform that efficiently provides a framework for running reinforcement learning (RL) experiments. We propose the CaiRL Environment Toolkit as an efficient, compatible, and more sustainable alternative for training learning agents and propose methods to develop more efficient environment simulations. There is an increasing focus on developing sustainable artificial intelligence. However, little effort has been made to improve the efficiency of running environment simulations. The most popular development toolkit for reinforcement learning, OpenAI Gym, is built using Python, a powerful but slow programming language. We propose a toolkit written in C++ with the same flexibility level but works orders of magnitude faster to make up for Python's inefficiency. This would drastically cut climate emissions. CaiRL also presents the first reinforcement learning toolkit with a built-in JVM and Flash support for running legacy flash games for reinforcement learning research. We demonstrate the effectiveness of CaiRL in the classic control benchmark, comparing the execution speed to OpenAI Gym. Furthermore, we illustrate that CaiRL can act as a drop-in replacement for OpenAI Gym to leverage significantly faster training speeds because of the reduced environment computation time.Comment: Published in 2022 IEEE Conference on Games (CoG

The Optical RL-Gym: an open-source toolkit for applying reinforcement learning in optical networks

Reinforcement Learning (RL) is leading to important breakthroughs in several areas (e.g., self-driving vehicles, robotics, and network automation). Part of its success is due to the existence of toolkits (e.g., OpenAI Gym) to implement standard RL tasks. On the one hand, they allow for the quick implementation and testing of new ideas. On the other, these toolkits ensure easy reproducibility via quick and fair benchmarking. RL is also gaining traction in the optical networks research community, showing promising results while solving several use cases. However, there are many scenarios where the benefits of RL-based solutions remain still unclear. A possible reason for this is the steep learning curve required to tailor RL-based frameworks to each specific use case. This, in turn, might delay or even prevent the development of new ideas. This paper introduces the Optical Network Reinforcement-Learning-Gym (Optical RL-Gym), an open-source toolkit that can be used to apply RL to problems related to optical networks. The Optical RL-Gym follows the principles established by the OpenAI Gym, the de-facto standard for RL environments. Optical RL-Gym allows for the quick integration with existing RL agents, as well as the possibility to build upon several already available environments to implement and solve more elaborated use cases related to the optical networks research area. The capabilities and the benefits of the proposed toolkit are illustrated by using the Optical RL-Gym to solve two different service provisioning problems