Decentralized Multiagent Coordination for Connected and Autonomous Vehicle Routing in Congested Networks

In 2017, the cost of congestion in the United States was around 305 billion dollars, and city-dwellers, on average, lost 1400 dollars while sitting 42 hours in traffic jams. Aiming for better mobility and more efficient utilization of the transportation network, emerging connected and autonomous vehicle (CAV) technologies and their communication capabilities can produce well-coordinated and more efficient routing behavior to dissipate the traffic rather uniformly throughout the network, resulting in slower travel times. Vehicle routing is among the most critical and challenging, yet unsolved, tasks in CAV research. Current routing strategies either rely on a centralized control system which can fail in scaling, or employ decentralized schemes that yield sub-optimal coordination and poor system performance. In addition, it is of great importance for the deployment of CAV technologies to understand the transportation systems behavior in a mixed environment with various levels of communication complexity, where CAVs and Non-CAVs coexist and interoperate. The routing problem in a multiagent system resembles a competitive congestion game. The decisions of one agent (in this case, a CAV) directly impacts the performance of the others. When the number of agents traversing the same transportation facility at the same time exceeds a certain threshold, bottlenecks may occur, and thus, higher travel times. Therefore, coordination between CAVs is key to avoiding such circumstances. This dissertation answers how and to what extent different routing optimization algorithms, under various levels of autonomy and communication capabilities, can increase the mobility of the transportation system. This work designs this system in a decentralized manner that scales linearly in achieving a social and system-level optimum. To realistically analyze this system, we investigated the coordination behavior of CAVs under (1) No Communication, (2) Minimal Communication, and (3) Extensive Communication. In the absence of connectivity between the CAVs, a learning-based approach has been implemented where each CAV optimizes its own route using a reinforcement learning technique and based on its prior experiences. This competitive game quickly overwhelms the system as the market penetration of CAVs surpasses the critical threshold range (50% to 75%), where the mobility improvements are the most significant, and beyond which the system performance degrades. Under minimal communication level, we assumed the CAVs share information regarding their location and speed with the rest of the CAVs in their communication cluster through a multi-hop network. Then, a coordination scheme was implemented where each CAV minimizes its travel time based on the limited information it receives. Results showed that this application can reduce system travel time by up to 20%. Additionally, the emergence of mobility benefits are shown to correlate with the CAV network characteristics through the lens of percolation theory. The results revealed that, for the mobility benefits to surface, at least 70% of the CAVs are required to form a communication cluster. Under an extensive communication capability, where the CAVs not only share their location and speed but also their preferred path to their destination, a reduction of up to 40% in system travel time was achieved for high levels of CAV market penetration and communication radius. Moreover, the improvement in mobility was proved to be highly associated with the uniform dissipation of traffic onto the network. These findings provide solid support to create evidence-driven frameworks to guide future CAV development and deployment in a decentralized and coordinated manner

MFIRE-2: A Multi Agent System for Flow-based Intrusion Detection Using Stochastic Search

Detecting attacks targeted against military and commercial computer networks is a crucial element in the domain of cyberwarfare. The traditional method of signature-based intrusion detection is a primary mechanism to alert administrators to malicious activity. However, signature-based methods are not capable of detecting new or novel attacks. This research continues the development of a novel simulated, multiagent, flow-based intrusion detection system called MFIRE. Agents in the network are trained to recognize common attacks, and they share data with other agents to improve the overall effectiveness of the system. A Support Vector Machine (SVM) is the primary classifier with which agents determine an attack is occurring. Agents are prompted to move to different locations within the network to find better vantage points, and two methods for achieving this are developed. One uses a centralized reputation-based model, and the other uses a decentralized model optimized with stochastic search. The latter is tested for basic functionality. The reputation model is extensively tested in two configurations and results show that it is significantly superior to a system with non-moving agents. The resulting system, MFIRE-2, demonstrates exciting new network defense capabilities, and should be considered for implementation in future cyberwarfare applications

Alleviating Traffic Congestion: Developing and Evaluating Novel Multi-Agent Reinforcement Learning Traffic Light Coordination Techniques

Contract # 69A3551747111Traffic congestion costs American cities tens of billions of dollars per year, not to mention its negative impact on the environment or people\u2019s mental health. Novel Markov game models and advanced reinforcement learning algorithms hold the promise of drastically alleviating congestion through dynamic coordination of traffic signals and adaptive techniques to dynamically re-route traffic. This project involves a collaboration with Econolite, a leading provider of traffic management systems

A Multi Agent System for Flow-Based Intrusion Detection

The detection and elimination of threats to cyber security is essential for system functionality, protection of valuable information, and preventing costly destruction of assets. This thesis presents a Mobile Multi-Agent Flow-Based IDS called MFIREv3 that provides network anomaly detection of intrusions and automated defense. This version of the MFIRE system includes the development and testing of a Multi-Objective Evolutionary Algorithm (MOEA) for feature selection that provides agents with the optimal set of features for classifying the state of the network. Feature selection provides separable data points for the selected attacks: Worm, Distributed Denial of Service, Man-in-the-Middle, Scan, and Trojan. This investigation develops three techniques of self-organization for multiple distributed agents in an intrusion detection system: Reputation, Stochastic, and Maximum Cover. These three movement models are tested for effectiveness in locating good agent vantage points within the network to classify the state of the network. MFIREv3 also introduces the design of defensive measures to limit the effects of network attacks. Defensive measures included in this research are rate-limiting and elimination of infected nodes. The results of this research provide an optimistic outlook for flow-based multi-agent systems for cyber security. The impact of this research illustrates how feature selection in cooperation with movement models for multi agent systems provides excellent attack detection and classification