Self-Learning Neural controller for Hybrid Power Management using Neuro-Dynamic Programming

A supervisory controller strategy for a hybrid vehicle coordinates the operation of the two power sources onboard of a vehicle to maximize objectives like fuel economy. In the past, various control strategies have been developed using heuristics as well as optimal control theory. The Stochastic Dynamic Programming (SDP) has been previously applied to determine implementable optimal control policies for discrete time dynamic systems whose states evolve according to given transition probabilities. However, the approach is constrained by the curse of dimensionality, i.e. an exponential increase in computational effort with increase in system state space, faced by dynamic programming based algorithms. This paper proposes a novel approach capable of overcoming the curse of dimensionality and solving policy optimization for a system with very large design state space. We propose developing a supervisory controller for hybrid vehicles based on the principles of reinforcement learning and neuro-dynamic programming, whereby the cost-to-go function is approximated using a neural network. The controller learns and improves its performance over time. The simulation results obtained for a series hydraulic hybrid vehicle over a driving schedule demonstrate the effectiveness of the proposed technique.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/89874/1/draft_01.pd

A Reinforcement Learning Technique For Enhancing Human Behavior Models In A Context-based Architecture

A reinforcement-learning technique for enhancing human behavior models in a context-based learning architecture is presented. Prior to the introduction of this technique, human models built and developed in a Context-Based reasoning framework lacked learning capabilities. As such, their performance and quality of behavior was always limited by what the subject matter expert whose knowledge is modeled was able to articulate or demonstrate. Results from experiments performed show that subject matter experts are prone to making errors and at times they lack information on situations that are inherently necessary for the human models to behave appropriately and optimally in those situations. The benefits of the technique presented is two fold; 1) It shows how human models built in a context-based framework can be modified to correctly reflect the knowledge learnt in a simulator; and 2) It presents a way for subject matter experts to verify and validate the knowledge they share. The results obtained from this research show that behavior models built in a context-based framework can be enhanced by learning and reflecting the constraints in the environment. From the results obtained, it was shown that after the models are enhanced, the agents performed better based on the metrics evaluated. Furthermore, after learning, the agent was shown to recognize unknown situations and behave appropriately in previously unknown situations. The overall performance and quality of behavior of the agent improved significantly

Safety of autonomous vehicles: A survey on Model-based vs. AI-based approaches

The growing advancements in Autonomous Vehicles (AVs) have emphasized the critical need to prioritize the absolute safety of AV maneuvers, especially in dynamic and unpredictable environments or situations. This objective becomes even more challenging due to the uniqueness of every traffic situation/condition. To cope with all these very constrained and complex configurations, AVs must have appropriate control architectures with reliable and real-time Risk Assessment and Management Strategies (RAMS). These targeted RAMS must lead to reduce drastically the navigation risks. However, the lack of safety guarantees proves, which is one of the key challenges to be addressed, limit drastically the ambition to introduce more broadly AVs on our roads and restrict the use of AVs to very limited use cases. Therefore, the focus and the ambition of this paper is to survey research on autonomous vehicles while focusing on the important topic of safety guarantee of AVs. For this purpose, it is proposed to review research on relevant methods and concepts defining an overall control architecture for AVs, with an emphasis on the safety assessment and decision-making systems composing these architectures. Moreover, it is intended through this reviewing process to highlight researches that use either model-based methods or AI-based approaches. This is performed while emphasizing the strengths and weaknesses of each methodology and investigating the research that proposes a comprehensive multi-modal design that combines model-based and AI approaches. This paper ends with discussions on the methods used to guarantee the safety of AVs namely: safety verification techniques and the standardization/generalization of safety frameworks

Risk analysis of autonomous vehicle and its safety impact on mixed traffic stream

In 2016, more than 35,000 people died in traffic crashes, and human error was the reason for 94% of these deaths. Researchers and automobile companies are testing autonomous vehicles in mixed traffic streams to eliminate human error by removing the human driver behind the steering wheel. However, recent autonomous vehicle crashes while testing indicate the necessity for a more thorough risk analysis. The objectives of this study were (1) to perform a risk analysis of autonomous vehicles and (2) to evaluate the safety impact of these vehicles in a mixed traffic stream. The overall research was divided into two phases: (1) risk analysis and (2) simulation of autonomous vehicles. Risk analysis of autonomous vehicles was conducted using the fault tree method. Based on failure probabilities of system components, two fault tree models were developed and combined to predict overall system reliability. It was found that an autonomous vehicle system could fail 158 times per one-million miles of travel due to either malfunction in vehicular components or disruption from infrastructure components. The second phase of this research was the simulation of an autonomous vehicle, where change in crash frequency after autonomous vehicle deployment in a mixed traffic stream was assessed. It was found that average travel time could be reduced by about 50%, and 74% of conflicts, i.e., traffic crashes, could be avoided by replacing 90% of the human drivers with autonomous vehicles

Hybrid and Electric Vehicles Optimal Design and Real-time Control based on Artificial Intelligence

L'abstract è presente nell'allegato / the abstract is in the attachmen

Automated and intelligent hacking detection system

Dissertação de mestrado integrado em Informatics EngineeringThe Controller Area Network (CAN) is the backbone of automotive networking, connecting many Electronic ControlUnits (ECUs) that control virtually every vehicle function from fuel injection to parking sensors. It possesses,however, no security functionality such as message encryption or authentication by default. Attackers can easily inject or modify packets in the network, causing vehicle malfunction and endangering the driver and passengers. There is an increasing number of ECUs in modern vehicles, primarily driven by the consumer’s expectation of more features and comfort in their vehicles as well as ever-stricter government regulations on efficiency and emissions. Combined with vehicle connectivity to the exterior via Bluetooth, Wi-Fi, or cellular, this raises the risk of attacks. Traditional networks, such as Internet Protocol (IP), typically have an Intrusion Detection System (IDS) analysing traffic and signalling when an attack occurs. The system here proposed is an adaptation of the traditional IDS into the CAN bus using a One Class Support Vector Machine (OCSVM) trained with live, attack-free traffic. The system is capable of reliably detecting a variety of attacks, both known and unknown, without needing to understand payload syntax, which is largely proprietary and vehicle/model dependent. This allows it to be installed in any vehicle in a plug-and-play fashion while maintaining a large degree of accuracy with very few false positives.A Controller Area Network (CAN) é a principal tecnologia de comunicação interna automóvel, ligando muitas Electronic Control Units (ECUs) que controlam virtualmente todas as funções do veículo desde injeção de combustível até aos sensores de estacionamento. No entanto, não possui por defeito funcionalidades de segurança como cifragem ou autenticação. É possível aos atacantes facilmente injetarem ou modificarem pacotes na rede causando estragos e colocando em perigo tanto o condutor como os passageiros. Existe um número cada vez maior de ECUs nos veículos modernos, impulsionado principalmente pelas expectativas do consumidores quanto ao aumento do conforto nos seus veículos, e pelos cada vez mais exigentes regulamentos de eficiência e emissões. Isto, associada à conexão ao exterior através de tecnologias como o Bluetooth, Wi-Fi, ou redes móveis, aumenta o risco de ataques. Redes tradicionais, como a rede Internet Protocol (IP), tipicamente possuem um Intrusion Detection Systems (IDSs) que analiza o tráfego e assinala a presença de um ataque. O sistema aqui proposto é uma adaptação do IDS tradicional à rede CAN utilizando uma One Class Support Vector Machine (OCSVM) treinada com tráfego real e livre de ataques. O sistema é capaz de detetar com fiabilidade uma variedade de ataques, tanto conhecidos como desconhecidos, sem a necessidade de entender a sintaxe do campo de dados das mensagens, que é maioritariamente proprietária. Isto permite ao sistema ser instalado em qualquer veículo num modo plug-and-play enquanto mantém um elevado nível de desempenho com muito poucos falsos positivos

Comparison of Modern Controls and Reinforcement Learning for Robust Control of Autonomously Backing Up Tractor-Trailers to Loading Docks

Two controller performances are assessed for generalization in the path following task of autonomously backing up a tractor-trailer. Starting from random locations and orientations, paths are generated to loading docks with arbitrary pose using Dubins Curves. The combination vehicles can be varied in wheelbase, hitch length, weight distributions, and tire cornering stiffness. The closed form calculation of the gains for the Linear Quadratic Regulator (LQR) rely heavily on having an accurate model of the plant. However, real-world applications cannot expect to have an updated model for each new trailer. Finding alternative robust controllers when the trailer model is changed was the motivation of this research. Reinforcement learning, with neural networks as their function approximators, can allow for generalized control from its learned experience that is characterized by a scalar reward value. The Linear Quadratic Regulator and the Deep Deterministic Policy Gradient (DDPG) are compared for robust control when the trailer is changed. This investigation quantifies the capabilities and limitations of both controllers in simulation using a kinematic model. The controllers are evaluated for generalization by altering the kinematic model trailer wheelbase, hitch length, and velocity from the nominal case. In order to close the gap from simulation and reality, the control methods are also assessed with sensor noise and various controller frequencies. The root mean squared and maximum errors from the path are used as metrics, including the number of times the controllers cause the vehicle to jackknife or reach the goal. Considering the runs where the LQR did not cause the trailer to jackknife, the LQR tended to have slightly better precision. DDPG, however, controlled the trailer successfully on the paths where the LQR jackknifed. Reinforcement learning was found to sacrifice a short term reward, such as precision, to maximize the future expected reward like reaching the loading dock. The reinforcement learning agent learned a policy that imposed nonlinear constraints such that it never jackknifed, even when it wasn\u27t the trailer it trained on

Advances in Intelligent Vehicle Control

This book is a printed edition of the Special Issue Advances in Intelligent Vehicle Control that was published in the journal Sensors. It presents a collection of eleven papers that covers a range of topics, such as the development of intelligent control algorithms for active safety systems, smart sensors, and intelligent and efficient driving. The contributions presented in these papers can serve as useful tools for researchers who are interested in new vehicle technology and in the improvement of vehicle control systems

Performance and Safety Enhancement Strategies in Vehicle Dynamics and Ground Contact

Recent trends in vehicle engineering are testament to the great efforts that scientists and industries have made to seek solutions to enhance both the performance and safety of vehicular systems. This Special Issue aims to contribute to the study of modern vehicle dynamics, attracting recent experimental and in-simulation advances that are the basis for current technological growth and future mobility. The area involves research, studies, and projects derived from vehicle dynamics that aim to enhance vehicle performance in terms of handling, comfort, and adherence, and to examine safety optimization in the emerging contexts of smart, connected, and autonomous driving.This Special Issue focuses on new findings in the following topics:(1) Experimental and modelling activities that aim to investigate interaction phenomena from the macroscale, analyzing vehicle data, to the microscale, accounting for local contact mechanics; (2) Control strategies focused on vehicle performance enhancement, in terms of handling/grip, comfort and safety for passengers, motorsports, and future mobility scenarios; (3) Innovative technologies to improve the safety and performance of the vehicle and its subsystems; (4) Identification of vehicle and tire/wheel model parameters and status with innovative methodologies and algorithms; (5) Implementation of real-time software, logics, and models in onboard architectures and driving simulators; (6) Studies and analyses oriented toward the correlation among the factors affecting vehicle performance and safety; (7) Application use cases in road and off-road vehicles, e-bikes, motorcycles, buses, trucks, etc