Integrating Pro-Environmental Behavior with Transportation Network Modeling: User and System Level Strategies, Implementation, and Evaluation

Personal transport is a leading contributor to fossil fuel consumption and greenhouse (GHG) emissions in the U.S. The U.S. Energy Information Administration (EIA) reports that light-duty vehicles (LDV) are responsible for 61\% of all transportation related energy consumption in 2012, which is equivalent to 8.4 million barrels of oil (fossil fuel) per day. The carbon content in fossil fuels is the primary source of GHG emissions that links to the challenge associated with climate change. Evidently, it is high time to develop actionable and innovative strategies to reduce fuel consumption and GHG emissions from the road transportation networks. This dissertation integrates the broader goal of minimizing energy and emissions into the transportation planning process using novel systems modeling approaches. This research aims to find, investigate, and evaluate strategies that minimize carbon-based fuel consumption and emissions for a transportation network. We propose user and system level strategies that can influence travel decisions and can reinforce pro-environmental attitudes of road users. Further, we develop strategies that system operators can implement to optimize traffic operations with emissions minimization goal. To complete the framework we develop an integrated traffic-emissions (EPA-MOVES) simulation framework that can assess the effectiveness of the strategies with computational efficiency and reasonable accuracy. ^ The dissertation begins with exploring the trade-off between emissions and travel time in context of daily travel decisions and its heterogeneous nature. Data are collected from a web-based survey and the trade-off values indicating the average additional travel minutes a person is willing to consider for reducing a lb. of GHG emissions are estimated from random parameter models. Results indicate that different trade-off values for male and female groups. Further, participants from high-income households are found to have higher trade-off values compared with other groups. Next, we propose personal mobility carbon allowance (PMCA) scheme to reduce emissions from personal travel. PMCA is a market-based scheme that allocates carbon credits to users at no cost based on the emissions reduction goal of the system. Users can spend carbon credits for travel and a market place exists where users can buy or sell credits. This dissertation addresses two primary dimensions: the change in travel behavior of the users and the impact at network level in terms of travel time and emissions when PMCA is implemented. To understand this process, a real-time experimental game tool is developed where players are asked to make travel decisions within the carbon budget set by PMCA and they are allowed to trade carbon credits in a market modeled as a double auction game. Random parameter models are estimated to examine the impact of PMCA on short-term travel decisions. Further, to assess the impact at system level, a multi-class dynamic user equilibrium model is formulated that captures the travel behavior under PMCA scheme. The equivalent variational inequality problem is solved using projection method. Results indicate that PMCA scheme is able to reduce GHG emissions from transportation networks. Individuals with high value of travel time (VOTT) are less sensitive to PMCA scheme in context of work trips. High and medium income users are more likely to have non-work trips with lower carbon cost (higher travel time) to save carbon credits for work trips. ^ Next, we focus on the strategies from the perspectives of system operators in transportation networks. Learning based signal control schemes are developed that can reduce emissions from signalized urban networks. The algorithms are implemented and tested in VISSIM micro simulator. Finally, an integrated emissions-traffic simulator framework is outlined that can be used to evaluate the effectiveness of the strategies. The integrated framework uses MOVES2010b as the emissions simulator. To estimate the emissions efficiently we propose a hierarchical clustering technique with dynamic time warping similarity measures (HC-DTW) to find the link driving schedules for MOVES2010b. Test results using the data from a five-intersection corridor show that HC-DTW technique can significantly reduce emissions estimation time without compromising the accuracy. The benefits are found to be most significant when the level of congestion variation is high. ^ In addition to finding novel strategies for reducing emissions from transportation networks, this dissertation has broader impacts on behavior based energy policy design and transportation network modeling research. The trade-off values can be a useful indicator to identify which policies are most effective to reinforce pro-environmental travel choices. For instance, the model can estimate the distribution of trade-off between emissions and travel time, and provide insights on the effectiveness of policies for New York City if we are able to collect data to construct a representative sample. The probability of route choice decisions vary across population groups and trip contexts. The probability as a function of travel and demographic attributes can be used as behavior rules for agents in an agent-based traffic simulation. Finally, the dynamic user equilibrium based network model provides a general framework for energy policies such carbon tax, tradable permit, and emissions credits system

Teaching Agents with Deep Apprenticeship Learning

As the field of robotic and humanoid systems expand, more research is being done on how to best control systems to perform complex, smart tasks. Many supervised learning and classification techniques require large datasets, and only result in the system mimicking what it was given. The sequential relationship within datasets used for task learning results in Markov decision problems that traditional classification algorithms cannot solve. Reinforcement learning helps to solve these types of problems using a reward/punishment and exploration/exploitation methodology without the need for datasets. While this works for simple systems, complex systems are more difficult to teach using traditional reinforcement learning. Often these systems have complex, non-linear, non-intuitive cost functions which make it near impossible to model. Inverse reinforcement learning, or apprenticeship learning algorithms, learn complex cost functions based on input from an expert system. Deep learning has also made a large impact in learning complex systems, and has achieved state of the art results in several applications. Using methods from apprenticeship learning and deep learning a system can be taught complex tasks from watching an expert. It is shown here how well these types of networks solve a specific task, and how well they generalize and understand the task through raw pixel data from an expert

Acta Universitatis Sapientiae - Electrical and Mechanical Engineering

Series Electrical and Mechanical Engineering publishes original papers and surveys in various fields of Electrical and Mechanical Engineering

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Apprentissage Intelligent des Robots Mobiles dans la Navigation Autonome

Modern robots are designed for assisting or replacing human beings to perform complicated planning and control operations, and the capability of autonomous navigation in a dynamic environment is an essential requirement for mobile robots. In order to alleviate the tedious task of manually programming a robot, this dissertation contributes to the design of intelligent robot control to endow mobile robots with a learning ability in autonomous navigation tasks. First, we consider the robot learning from expert demonstrations. A neural network framework is proposed as the inference mechanism to learn a policy offline from the dataset extracted from experts. Then we are interested in the robot self-learning ability without expert demonstrations. We apply reinforcement learning techniques to acquire and optimize a control strategy during the interaction process between the learning robot and the unknown environment. A neural network is also incorporated to allow a fast generalization, and it helps the learning to converge in a number of episodes that is greatly smaller than the traditional methods. Finally, we study the robot learning of the potential rewards underneath the states from optimal or suboptimal expert demonstrations. We propose an algorithm based on inverse reinforcement learning. A nonlinear policy representation is designed and the max-margin method is applied to refine the rewards and generate an optimal control policy. The three proposed methods have been successfully implemented on the autonomous navigation tasks for mobile robots in unknown and dynamic environments.Les robots modernes sont appelés à effectuer des opérations ou tâches complexes et la capacité de navigation autonome dans un environnement dynamique est un besoin essentiel pour les robots mobiles. Dans l’objectif de soulager de la fastidieuse tâche de préprogrammer un robot manuellement, cette thèse contribue à la conception de commande intelligente afin de réaliser l’apprentissage des robots mobiles durant la navigation autonome. D’abord, nous considérons l’apprentissage des robots via des démonstrations d’experts. Nous proposons d’utiliser un réseau de neurones pour apprendre hors-ligne une politique de commande à partir de données utiles extraites d’expertises. Ensuite, nous nous intéressons à l’apprentissage sans démonstrations d’experts. Nous utilisons l’apprentissage par renforcement afin que le robot puisse optimiser une stratégie de commande pendant le processus d’interaction avec l’environnement inconnu. Un réseau de neurones est également incorporé et une généralisation rapide permet à l’apprentissage de converger en un certain nombre d’épisodes inférieur à la littérature. Enfin, nous étudions l’apprentissage par fonction de récompenses potentielles compte rendu des démonstrations d’experts optimaux ou non-optimaux. Nous proposons un algorithme basé sur l’apprentissage inverse par renforcement. Une représentation non-linéaire de la politique est désignée et la méthode du max-margin est appliquée permettant d’affiner les récompenses et de générer la politique de commande. Les trois méthodes proposées sont évaluées sur des robots mobiles afin de leurs permettre d’acquérir les compétences de navigation autonome dans des environnements dynamiques et inconnu