A Java Reinforcement Learning Module for the Recursive Porous Agent Simulation Toolkit : facilitating study and experimentation with reinforcement learning in social science multi-agent simulations

This work details a machine learning tool developed to support computational, agent-based simulation research in the social sciences. Specifically, the Java Reinforcement Learning Module (JReLM) is a platform for implementing reinforcement learning algorithms for use in agent-based simulations. The module was designed for use with the Recursive Porous Agent Simulation Toolkit (Repast), an agent-based simulation platform popular in computational social science research. Background, architecture, and implementation of JReLM are discussed within. This includes explanation of pre-implemented tools and algorithms available for immediate use in Repast simulations. In addition, an account of JReLM\u27s use in an agent-based computational economics simulation is included as an illustrative application. Directions for further development and future use in ongoing agent-based computational economics work are discussed as well

High-Performance Computing and ABMS for High-Resolution COVID-19 Spreading Simulation

This paper presents an approach for the modeling and the simulation of the spreading of COVID-19 based on agent-based modeling and simulation (ABMS). Our goal is not only to support large-scale simulations but also to increase the simulation resolution. Moreover, we do not assume an underlying network of contacts, and the person-to-person contacts responsible for the spreading are modeled as a function of the geographical distance among the individuals. In particular, we defined a commuting mechanism combining radiation-based and gravity-based models and we exploited the commuting properties at different resolution levels (municipalities and provinces). Finally, we exploited the high-performance computing (HPC) facilities to simulate millions of concurrent agents, each mapping the individual’s behavior. To do such simulations, we developed a spreading simulator and validated it through the simulation of the spreading in two of the most populated Italian regions: Lombardy and Emilia-Romagna. Our main achievement consists of the effective modeling of 10 million of concurrent agents, each one mapping an individual behavior with a high-resolution in terms of social contacts, mobility and contribution to the virus spreading. Moreover, we analyzed the forecasting ability of our framework to predict the number of infections being initialized with only a few days of real data. We validated our model with the statistical data coming from the serological analysis conducted in Lombardy, and our model makes a smaller error than other state of the art models with a final root mean squared error equal to 56,009 simulating the entire first pandemic wave in spring 2020. On the other hand, for the Emilia-Romagna region, we simulated the second pandemic wave during autumn 2020, and we reached a final RMSE equal to 10,730.11

Simulator for Undergraduate Multi-Agent Systems

In recent years, Multi-Agent Systems (MAS) have for the first time begun to be accepted in mainstream computing. Software companies haave been founded focusing purely on MAS software, telecommunications companies now use agent-based technologies in cell phones, and there have even been two successful DARPA funded, military-grade defense projects in the past ten years. The growth in demand development tools available. The various development platforms focus on mobile devices, large-scale distributed systems, and specific research applications; however, these tools leave an important facet of MAS development unsatisfied--undergraduate research and teaching. Each of the solutions available is either too complex, too specific, or in some way infeasible to be used by students in what is possibly their first introduction to MAS. This research concentrates on creating a distributed, graphical MAS simulator in Java and an associated Application Program Interface (API) for developing agent-based systems at the undergraduate level. Whether in research or in the classroom, the well designed, easily extensible API allows students to create and immediately display their agents\u27 interactions in the simulation environment with minimal programming. The API provides agents with the capacity for perception, communication, memory, and action. Future undergraduate research and learning in the field of MAS will be greatly facilitated by this intuitive simulation platform. Students can learn MAS by observing agents visually, and student researchers can focus purely on programming and analyzing agent behavior

On the Simulation of Global Reputation Systems

Reputation systems evolve as a mechanism to build trust in virtual communities. In this paper we evaluate different metrics for computing reputation in multi-agent systems. We present a formal model for describing metrics in reputation systems and show how different well-known global reputation metrics are expressed by it. Based on the model a generic simulation framework for reputation metrics was implemented. We used our simulation framework to compare different global reputation systems to find their strengths and weaknesses. The strength of a metric is measured by its resistance against different threat-models, i.e. different types of hostile agents. Based on our results we propose a new metric for reputation systems.Reputation System, Trust, Formalization, Simulation

Developing sustainability pathways for social simulation tools and services

The use of cloud technologies to teach agent-based modelling and simulation (ABMS) is an interesting application of a nascent technological paradigm that has received very little attention in the literature. This report fills that gap and aims to help instructors, teachers and demonstrators to understand why and how cloud services are appropriate solutions to common problems they face delivering their study programmes, as well as outlining the many cloud options available. The report first introduces social simulation and considers how social simulation is taught. Following this factors affecting the implementation of agent-based models are explored, with attention focused primarily on the modelling and execution platforms currently available, the challenges associated with implementing agent-based models, and the technical architectures that can be used to support the modelling, simulation and teaching process. This sets the context for an extended discussion on cloud computing including service and deployment models, accessing cloud resources, the financial implications of adopting the cloud, and an introduction to the evaluation of cloud services within the context of developing, executing and teaching agent-based models

Network Partitioning in Distributed Agent-Based Models

Agent-Based Models (ABMs) are an emerging simulation paradigm for modeling complex systems, comprised of autonomous, possibly heterogeneous, interacting agents. The utility of ABMs lies in their ability to represent such complex systems as self-organizing networks of agents. Modeling and understanding the behavior of complex systems usually occurs at large and representative scales, and often obtaining and visualizing of simulation results in real-time is critical. The real-time requirement necessitates the use of in-memory computing, as it is difficult and challenging to handle the latency and unpredictability of disk accesses. Combining this observation with the scale requirement emphasizes the need to use parallel and distributed computing platforms, such as MPI-enabled CPU clusters. Consequently, the agent population must be partitioned across different CPUs in a cluster. Further, the typically high volume of interactions among agents can quickly become a significant bottleneck for real-time or large-scale simulations. The problem is exacerbated if the underlying ABM network is dynamic and the inter-process communication evolves over the course of the simulation. Therefore, it is critical to develop topology-aware partitioning mechanisms to support such large simulations. In this dissertation, we demonstrate that distributed agent-based model simulations benefit from the use of graph partitioning algorithms that involve a local, neighborhood-based perspective. Such methods do not rely on global accesses to the network and thus are more scalable. In addition, we propose two partitioning schemes that consider the bottom-up individual-centric nature of agent-based modeling. The First technique utilizes label-propagation community detection to partition the dynamic agent network of an ABM. We propose a latency-hiding, seamless integration of community detection in the dynamics of a distributed ABM. To achieve this integration, we exploit the similarity in the process flow patterns of a label-propagation community-detection algorithm and self-organizing ABMs. In the second partitioning scheme, we apply a combination of the Guided Local Search (GLS) and Fast Local Search (FLS) metaheuristics in the context of graph partitioning. The main driving principle of GLS is the dynamic modi?cation of the objective function to escape local optima. The algorithm augments the objective of a local search, thereby transforming the landscape structure and escaping a local optimum. FLS is a local search heuristic algorithm that is aimed at reducing the search space of the main search algorithm. It breaks down the space into sub-neighborhoods such that inactive sub-neighborhoods are removed from the search process. The combination of GLS and FLS allowed us to design a graph partitioning algorithm that is both scalable and sensitive to the inherent modularity of real-world networks

Programming agent-based demographic models with cross-state and message-exchange dependencies: A study with speculative PDES and automatic load-sharing

Agent-based modeling and simulation is a versatile and promising methodology to capture complex interactions among entities and their surrounding environment. A great advantage is its ability to model phenomena at a macro scale by exploiting simpler descriptions at a micro level. It has been proven effective in many fields, and it is rapidly becoming a de-facto standard in the study of population dynamics. In this article we study programmability and performance aspects of the last-generation ROOT-Sim speculative PDES environment for multi/many-core shared-memory architectures. ROOT-Sim transparently offers a programming model where interactions can be based on both explicit message passing and in-place state accesses. We introduce programming guidelines for systematic exploitation of these facilities in agent-based simulations, and we study the effects on performance of an innovative load-sharing policy targeting these types of dependencies. An experimental assessment with synthetic and real-world applications is provided, to assess the validity of our proposal

Modeling and simulation of the resistance of bacteria to antibiotics

The unnecessary use of antibiotics has given rise to antibiotic resistance and for this reason is a cause of growing concern in contemporary health care contexts. Antibiotic resistance means that an antibiotic is losing or has lost the ability to kill a given bacteria and/or to prevent it from reproducing. The result: an increase in the number of patients suffering from and even dying of infections. Resistant bacteria continue to increase in number, as they survive the antibiotic designed and used to kill them. The disease induced by the bacteria lasts longer, therefore, than would have been the case were the bacteria not antibiotic resistant. Thus, prolonged treatment and/or even death results together with an increase in cost associated with these outcomes. The purpose of this study is to investigate the interactions among the bacteria, immune system cells, and antibiotics in a Repast Simphony 2.1 agent-based simulation environment modeled to observe the effects of the antibiotic resistance in the infection process. According to our results, increased antibiotic resistance constitutes a serious threat to the success of established methods used in the treatment of bacterial infections