Multiple Stochastic Learning Automata for Vehicle Path Control in an Automated Highway System

This paper suggests an intelligent controller for an automated vehicle planning its own trajectory based on sensor and communication data. The intelligent controller is designed using the learning stochastic automata theory. Using the data received from on-board sensors, two automata (one for lateral actions, one for longitudinal actions) can learn the best possible action to avoid collisions. The system has the advantage of being able to work in unmodeled stochastic environments, unlike adaptive control methods or expert systems. Simulations for simultaneous lateral and longitudinal control of a vehicle provide encouraging result

Simulation Study of Learning Automata Games in Automated Highway Systems

One of the most important issues in Automated Highway System (AHS) deployment is intelligent vehicle control. While the technology to safely maneuver vehicles exists, the problem of making intelligent decisions to improve a single vehicle’s travel time and safety while optimizing the overall traffic flow is still a stumbling block. We propose an artificial intelligence technique called stochastic learning automata to design an intelligent vehicle path controller. Using the information obtained by on-board sensors and local communication modules, two automata are capable of learning the best possible (lateral and longitudinal) actions to avoid collisions. This learning method is capable of adapting to the automata environment resulting from an unmodeled physical environment. Although the learning approach taken is capable of providing a safe decision, optimization of the overall traffic flow is required. This is achieved by studying the interaction of the vehicles. The design of the adaptive vehicle path planner based on local information is extended to the interaction of multiple intelligent vehicles. By analyzing the situations consisting of conflicting desired vehicle paths, we extend our design by additional decision structures. The analysis of the situations and the design of the additional structures are made possible by treatment of the interacting reward-penalty mechanisms in individual vehicles as automata games. The definition of the physical environment of a vehicle as a series of discrete state transitions associated with a “stationary automata environment” is the key to this analysis and to the design of the intelligent vehicle path controller

Intelligent Control of Vehicles: Preliminary Results on the Application of Learning Automata Techniques to Automated Highway System

We suggest an intelligent controller for an automated vehicle to plan its own trajectory based on sensor and communication data received. Our intelligent controller is based on an artificial intelligence technique called learning stochastic automata. The automaton can learn the best possible action to avoid collisions using the data received from on-board sensors. The system has the advantage of being able to work in unmodeled stochastic environments. Simulations for the lateral control of a vehicle using this AI method provides encouraging results

Situational Awareness Enhancement for Connected and Automated Vehicle Systems

Recent developments in the area of Connected and Automated Vehicles (CAVs) have boosted the interest in Intelligent Transportation Systems (ITSs). While ITS is intended to resolve and mitigate serious traffic issues such as passenger and pedestrian fatalities, accidents, and traffic congestion; these goals are only achievable by vehicles that are fully aware of their situation and surroundings in real-time. Therefore, connected and automated vehicle systems heavily rely on communication technologies to create a real-time map of their surrounding environment and extend their range of situational awareness. In this dissertation, we propose novel approaches to enhance situational awareness, its applications, and effective sharing of information among vehicles.;The communication technology for CAVs is known as vehicle-to-everything (V2x) communication, in which vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) have been targeted for the first round of deployment based on dedicated short-range communication (DSRC) devices for vehicles and road-side transportation infrastructures. Wireless communication among these entities creates self-organizing networks, known as Vehicular Ad-hoc Networks (VANETs). Due to the mobile, rapidly changing, and intrinsically error-prone nature of VANETs, traditional network architectures are generally unsatisfactory to address VANETs fundamental performance requirements. Therefore, we first investigate imperfections of the vehicular communication channel and propose a new modeling scheme for large-scale and small-scale components of the communication channel in dense vehicular networks. Subsequently, we introduce an innovative method for a joint modeling of the situational awareness and networking components of CAVs in a single framework. Based on these two models, we propose a novel network-aware broadcast protocol for fast broadcasting of information over multiple hops to extend the range of situational awareness. Afterward, motivated by the most common and injury-prone pedestrian crash scenarios, we extend our work by proposing an end-to-end Vehicle-to-Pedestrian (V2P) framework to provide situational awareness and hazard detection for vulnerable road users. Finally, as humans are the most spontaneous and influential entity for transportation systems, we design a learning-based driver behavior model and integrate it into our situational awareness component. Consequently, higher accuracy of situational awareness and overall system performance are achieved by exchange of more useful information

Situational reasoning for road driving in an urban environment

Robot navigation in urban environments requires situational reasoning. Given the complexity of the environment and the behavior specified by traffic rules, it is necessary to recognize the current situation to impose the correct traffic rules. In an attempt to manage the complexity of the situational reasoning subsystem, this paper describes a finite state machine model to govern the situational reasoning process. The logic state machine and its interaction with the planning system are discussed. The approach was implemented on Alice, Team Caltech’s entry into the 2007 DARPA Urban Challenge. Results from the qualifying rounds are discussed. The approach is validated and the shortcomings of the implementation are identified

Application and Control Aware Communication Strategies for Transportation and Energy Cyber-Physical Systems

Cyber--Physical Systems (CPSs) are a generation of engineered systems in which computing, communication, and control components are tightly integrated. Some important application domains of CPS are transportation, energy, and medical systems. The dynamics of CPSs are complex, involving the stochastic nature of communication systems, discrete dynamics of computing systems, and continuous dynamics of control systems. The existence of communication between and among controllers of physical processes is one of the basic characteristics of CPSs. Under this situation, some fundamental questions are: 1) How does the network behavior (communication delay, packet loss, etc.) affect the stability of the system? 2) Under what conditions is a complex system stabilizable?;In cases where communication is a component of a control system, scalability of the system becomes a concern. Therefore, one of the first issues to consider is how information about a physical process should be communicated. For example, the timing for sampling and communication is one issue. The traditional approach is to sample the physical process periodically or at predetermined times. An alternative is to sample it when specific events occur. Event-based sampling requires continuous monitoring of the system to decide a sample needs to be communicated. The main contributions of this dissertation in energy cyber-physical system domain are designing and modeling of event-based (on-demand) communication mechanisms. We show that in the problem of tracking a dynamical system over a network, if message generation and communication have correlation with estimation error, the same performance as the periodic sampling and communication method can be reached using a significantly lower rate of data.;For more complex CPSs such as vehicle safety systems, additional considerations for the communication component are needed. Communication strategies that enable robust situational awareness are critical for the design of CPSs, in particular for transportation systems. In this dissertation, we utilize the recently introduced concept of model-based communication and propose a new communication strategy to address this need. Our approach to model behavior of remote vehicles mathematically is to describe the small-scale structure of the remote vehicle movement (e.g. braking, accelerating) by a set of dynamic models and represent the large-scale structure (e.g. free following, turning) by coupling these dynamic models together into a Markov chain. Assuming model-based communication approach, a novel stochastic model predictive method is proposed to achieve cruise control goals and investigate the effect of new methodology.;To evaluate the accuracy and robustness of a situational awareness methodology, it is essential to study the mutual effect of the components of a situational awareness subsystem, and their impact on the accuracy of situational awareness. The main components are estimation and networking processes. One possible approach in this task is to produce models that provide a clear view into the dynamics of these two components. These models should integrate continuous physical dynamics, expressed with ordinary differential equations, with the discrete behaviors of communication, expressed with finite automata or Markov chain. In this dissertation, a hybrid automata model is proposed to combine and model both networking and estimation components in a single framework and investigate their interactions.;In summary, contributions of this dissertation lie in designing and evaluating methods that utilize knowledge of the physical element of CPSs to optimize the behavior of communication subsystems. Employment of such methods yields significant overall system performance improvement without incurring additional communication deployment costs

Discretized Bayesian pursuit – A new scheme for reinforcement learning

The success of Learning Automata (LA)-based estimator algorithms over the classical, Linear Reward-Inaction ( L RI )-like schemes, can be explained by their ability to pursue the actions with the highest reward probability estimates. Without access to reward probability estimates, it makes sense for schemes like the L RI to first make large exploring steps, and then to gradually turn exploration into exploitation by making progressively smaller learning steps. However, this behavior becomes counter-intuitive when pursuing actions based on their estimated reward probabilities. Learning should then ideally proceed in progressively larger steps, as the reward probability estimates turn more accurate. This paper introduces a new estimator algorithm, the Discretized Bayesian Pursuit Algorithm (DBPA), that achieves this. The DBPA is implemented by linearly discretizing the action probability space of the Bayesian Pursuit Algorithm (BPA) [1]. The key innovation is that the linear discrete updating rules mitigate the counter-intuitive behavior of the corresponding linear continuous updating rules, by augmenting them with the reward probability estimates. Extensive experimental results show the superiority of DBPA over previous estimator algorithms. Indeed, the DBPA is probably the fastest reported LA to date