Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process

From Social Simulation to Integrative System Design

As the recent financial crisis showed, today there is a strong need to gain "ecological perspective" of all relevant interactions in socio-economic-techno-environmental systems. For this, we suggested to set-up a network of Centers for integrative systems design, which shall be able to run all potentially relevant scenarios, identify causality chains, explore feedback and cascading effects for a number of model variants, and determine the reliability of their implications (given the validity of the underlying models). They will be able to detect possible negative side effect of policy decisions, before they occur. The Centers belonging to this network of Integrative Systems Design Centers would be focused on a particular field, but they would be part of an attempt to eventually cover all relevant areas of society and economy and integrate them within a "Living Earth Simulator". The results of all research activities of such Centers would be turned into informative input for political Decision Arenas. For example, Crisis Observatories (for financial instabilities, shortages of resources, environmental change, conflict, spreading of diseases, etc.) would be connected with such Decision Arenas for the purpose of visualization, in order to make complex interdependencies understandable to scientists, decision-makers, and the general public.Comment: 34 pages, Visioneer White Paper, see http://www.visioneer.ethz.c

Agent Based Modeling and Simulation: An Informatics Perspective

The term computer simulation is related to the usage of a computational model in order to improve the understanding of a system's behavior and/or to evaluate strategies for its operation, in explanatory or predictive schemes. There are cases in which practical or ethical reasons make it impossible to realize direct observations: in these cases, the possibility of realizing 'in-machina' experiments may represent the only way to study, analyze and evaluate models of those realities. Different situations and systems are characterized by the presence of autonomous entities whose local behaviors (actions and interactions) determine the evolution of the overall system; agent-based models are particularly suited to support the definition of models of such systems, but also to support the design and implementation of simulators. Agent-Based models and Multi-Agent Systems (MAS) have been adopted to simulate very different kinds of complex systems, from the simulation of socio-economic systems to the elaboration of scenarios for logistics optimization, from biological systems to urban planning. This paper discusses the specific aspects of this approach to modeling and simulation from the perspective of Informatics, describing the typical elements of an agent-based simulation model and the relevant research.Multi-Agent Systems, Agent-Based Modeling and Simulation