Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning

Recent advances in combining deep neural network architectures with reinforcement learning techniques have shown promising potential results in solving complex control problems with high dimensional state and action spaces. Inspired by these successes, in this paper, we build two kinds of reinforcement learning algorithms: deep policy-gradient and value-function based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The policy-gradient based agent maps its observation directly to the control signal, however the value-function based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Our methods show promising results in a traffic network simulated in the SUMO traffic simulator, without suffering from instability issues during the training process

Spatial Dynamic Modeling and Urban Land Use Transformation:

Assessing the economic impacts of urban land use transformation has become complex and acrimonious. Although community planners are beginning to comprehend the economic trade-offs inherent in transforming the urban fringe, they find it increasingly difficult to analyze and assess the trade-offs expediently and in ways that can influence local decisionmaking. New and sophisticated spatial modeling techniques are now being applied to urban systems that can quickly assess the probable spatial outcomes of given communal policies. Applying an economic impact assessment to the probable spatial patterns can provide to planners the tools needed to quickly assess scenarios for policy formation that will ultimately help inform decision makers. This paper focuses on the theoretical underpinnings and practical application of an economic impact analysis submodel developed within the Land use Evolution and Impact Assessment Modeling (LEAM) environment. The conceptual framework of LEAM is described, followed by an application of the model to the assessment of the cost of urban sprawl in Kane County, Illinois. The results show the effectiveness of spatially explicit modeling from a theoretical and a practical point of view. The agent-based approach of spatial dynamic modeling with a high spatial resolution allows for discerning the macro-level implications of micro-level behaviors. These phenomena are highlighted in the economic submodel in the discussion of the implications of land use change decisions on individual and communal costs; low-density development patterns favoring individual behaviors at the expense of the broader community.

Agent-based modelling of air transport demand

Constraints such as opening hours or passenger capacities influence travel options that can be offered by an airport and by the connecting airlines. If infrastructure, policy or technological measures modify transport options, then the benefits do not only depend on the technology, but also on possibly heterogeneous user preferences such as desired arrival times or on the availability of alternative travel modes. This paper proposes an agent-based, iterative assignment procedure to model European air traffic and German passenger demand on a microscopic level, capturing individual passenger preferences. Air transport technology is simulated microscopically, i.e. each aircraft is represented as single unit with attached attributes such as departure time, flight duration or seat availability. Trip-chaining and delay propagation can be added. Microsimulation is used to verify and assess passengers’ choices of travel alternatives, where those choices improve over iterations until an agent-based stochastic user equilibrium is reached. This requires fast simulation models, thus, similar to other approaches in air traffic modelling a queue model is used. In contrast to those approaches, the queue model in this work is solved algorithmically. Overall, the approach is suited to analyze, forecast and evaluate the consequences of mid-distance transport measures