Packet Routing Simulator for Multi-Agent Reinforcement Learning (PRISMA) (Version v0.1)

Abstract

PRISMA (Packet Routing Simulator for Multi-Agent Reinforcement Learning) is a network simulation playground for developing and testing Multi-Agent Reinforcement Learning (MARL) solutions for dynamic packet routing (DPR). This framework is based on the OpenAI Gym toolkit and the ns-3 simulator.The OpenAI Gym is a toolkit for RL widely used in research. The network simulator ns–3 is a standard library, which may provide useful simulation tools. It generates discrete events and provides several protocol implementations.Moreover, the NetSim implementation is based on ns3-gym, which integrates OpenAI Gym and ns-3.The main contributions of this framework: 1) A RL framework designed for specifically the DPR problem, serving as a playground where the community can easily validate their own RL approaches and compare them. 2) A more realistic modelling based on: (i) the well-known ns-3 network simulator, and (ii) a multi-threaded implementation for each agent. 3) A modular code design, which allows a researcher to test their own RL algorithm for the DPR problem, without needing to work on the implementation of the environment