Modeling motorcycle maneuvering in urban scenarios using Markov decision process with a dynamical-discretized reward field

This paper proposes a novel MDP framework to deal with the accuracy of the motorcycle driving model. It proposes a weighted and unweighted Dynamical-Discretized Reward Field (DDRF) as a major contribution on modeling motorcycle maneuver in mixed traffic conditions. Other contributions of this work are the integration of a motorcycle trajectory maneuver model in the state transition function, derivation of probability functions, area of awareness (AoA) and its sectorization to perceive vehicles inside the AoA, which is used to determine actions. We conducted some simulations to evaluate the performance of the proposed model by comparing the data from the simulations with real data. In this study, we use 100 simulation data on motorcycle maneuvering, which consisted of two different scenarios, i.e., 50 data of motorcycle maneuvering to avoid other motorcycles and 50 data of motorcycle maneuvering to avoid cars. We adjusted the simulation setting to the real situation and measured the performance of the proposed model using root mean square error (RMSE). In general, the proposed method can properly model the maneuver of motorcycles in heterogeneous traffic with an RMSE value of around 0.74 meters. This model performs twice as good as an existing car-following model. Furthermore, the proposed reward function performs around 4~6% better than the reward function in previous studies

Modeling motorcycle's maneuver using Markov decision process with a dynamical discretized reward field in urban scenarios

This paper proposes a novel MDP framework to deal with the accuracy of the motorcycle driving model. It proposes a weighted and unweighted Dynamical-Discretized Reward Field (DDRF) as a major contribution on modeling motorcycle maneuver in mixed traffic conditions. Other contributions of this work are the integration of a motorcycle trajectory maneuver model in the state transition function, derivation of probability functions, area of awareness (AoA) and its sectorization to perceive vehicles inside the AoA, which is used to determine actions. We conducted some simulations to evaluate the performance of the proposed model by comparing the data from the simulations with real data. In this study, we use 100 simulation data on motorcycle maneuvering, which consisted of two different scenarios, i.e., 50 data of motorcycle maneuvering to avoid other motorcycles and 50 data of motorcycle maneuvering to avoid cars. We adjusted the simulation setting to the real situation and measured the performance of the proposed model using root mean square error (RMSE). In general, the proposed method can properly model the maneuver of motorcycles in heterogeneous traffic with an RMSE value of around 0.74 meters. This model performs twice as good as an existing car-following model. Furthermore, the proposed reward function performs around 4~6% better than the reward function in previous studies

A Point-based MDP for Robust Single-Lane Autonomous Driving Behavior under Uncertainties

Abstract—In this paper, a point-based Markov Decision Process (QMDP) algorithm is used for robust single-lane autonomous driving behavior control under uncertainties. Autonomous vehicle decision making is modeled as a Markov Decision Process (MDP), then extended to a QMDP framework. Based on MDP/QMDP, three kinds of uncertainties are taken into account: sensor noise, perception constraints and surrounding vehicles ’ behavior. In simulation, the QMDP-based reasoning framework makes the autonomous vehicle perform with differing levels of conservativeness corresponding to different perception confidence levels. Road tests also indicate that the proposed algorithm helps the vehicle in avoiding potentially unsafe situations under these uncertainties. In general, the results indicate that the proposed QMDP-based algorithm makes autonomous driving more robust to limited sensing ability and occasional sensor failures. I

Agent-Based Resilient Transportation Infrastructure with Surrogate Adaptive Networks

Connected autonomous intelligent agents (AIA) with enhanced decision making through machine learning can improve intersection performance and resilience for the transportation infrastructure. An agent is an autonomous decision maker whose decision making is determined internally but may be altered by interactions with the environment or other agents. Implementing agent-based modeling techniques to advance communication for more appropriate decision making will provide great benefits to autonomous vehicle technology. A new algorithm is proposed to improve the decision-making process of autonomous vehicles and intelligent traffic signals, specifically at city like intersections. This is completed by understanding vehicle to vehicle (V2V), vehicle to infrastructure (V2I), and infrastructure to infrastructure (I2I) communication and using gathered data to ensure these agents make more appropriate decisions given the circumstances. These vehicles and signals are modeled to adapt to the common traffic flow of the intersection and ultimately find an optimum flow that will decrease average vehicle time to ultimately reduce inefficiency through each intersection. Considering each light and vehicle as an agent and utilizing communication between these agents will enable opportunity for data transmission. Improving agent-based I2I communication and decision making will provide performance benefits to traffic flow capacities. Evaluations were completed comparing intersections with fixed, coordinated, and adaptive timing signals. A fixed timing signal is an intersection using a fixed maximum green light time with no opportunity for adjustment. The coordinated signals adapt and change light status based on the current light status of adjacent intersections. Adaptive signals add in a recognition of vehicle load in one direction and adjust their own status either based on the load at the individual intersection or a neighboring light status change with the intent to improve traffic flow. To compare these scenarios given a specific example of 160 total vehicles present on the road in a 2x2 intersection grid setup, inefficiency was reduced from 50% to 45% given the relationship between ideal average time compared to actual average time for vehicles proceeding through an intersection. Overall tests were run to compare the different light signal options based on the number of vehicles on the road and maximum green light time in one direction. The results were consistent and overall inefficiency was reduced using an adaptive traffic signal to recognize upcoming vehicles combined with the ability to adjust based on adjacent intersection light status changes

A Human Driver Model for Autonomous Lane Changing in Highways: Predictive Fuzzy Markov Game Driving Strategy

This study presents an integrated hybrid solution to mandatory lane changing problem to deal with accident avoidance by choosing a safe gap in highway driving. To manage this, a comprehensive treatment to a lane change active safety design is proposed from dynamics, control, and decision making aspects. My effort first goes on driver behaviors and relating human reasoning of threat in driving for modeling a decision making strategy. It consists of two main parts; threat assessment in traffic participants, (TV s) states, and decision making. The first part utilizes an complementary threat assessment of TV s, relative to the subject vehicle, SV , by evaluating the traffic quantities. Then I propose a decision strategy, which is based on Markov decision processes (MDPs) that abstract the traffic environment with a set of actions, transition probabilities, and corresponding utility rewards. Further, the interactions of the TV s are employed to set up a real traffic condition by using game theoretic approach. The question to be addressed here is that how an autonomous vehicle optimally interacts with the surrounding vehicles for a gap selection so that more effective performance of the overall traffic flow can be captured. Finding a safe gap is performed via maximizing an objective function among several candidates. A future prediction engine thus is embedded in the design, which simulates and seeks for a solution such that the objective function is maximized at each time step over a horizon. The combined system therefore forms a predictive fuzzy Markov game (FMG) since it is to perform a predictive interactive driving strategy to avoid accidents for a given traffic environment. I show the effect of interactions in decision making process by proposing both cooperative and non-cooperative Markov game strategies for enhanced traffic safety and mobility. This level is called the higher level controller. I further focus on generating a driver controller to complement the automated car’s safe driving. To compute this, model predictive controller (MPC) is utilized. The success of the combined decision process and trajectory generation is evaluated with a set of different traffic scenarios in dSPACE virtual driving environment. Next, I consider designing an active front steering (AFS) and direct yaw moment control (DYC) as the lower level controller that performs a lane change task with enhanced handling performance in the presence of varying front and rear cornering stiffnesses. I propose a new control scheme that integrates active front steering and the direct yaw moment control to enhance the vehicle handling and stability. I obtain the nonlinear tire forces with Pacejka model, and convert the nonlinear tire stiffnesses to parameter space to design a linear parameter varying controller (LPV) for combined AFS and DYC to perform a commanded lane change task. Further, the nonlinear vehicle lateral dynamics is modeled with Takagi-Sugeno (T-S) framework. A state-feedback fuzzy H∞ controller is designed for both stability and tracking reference. Simulation study confirms that the performance of the proposed methods is quite satisfactory