Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station

An electric vehicle charging station (EVCS) infrastructure is the backbone of transportation electrification. However, the EVCS has myriads of exploitable vulnerabilities in software, hardware, supply chain, and incumbent legacy technologies such as network, communication, and control. These standalone or networked EVCS open up large attack surfaces for the local or state-funded adversaries. The state-of-the-art approaches are not agile and intelligent enough to defend against and mitigate advanced persistent threats (APT). We propose the data-driven model-free distributed intelligence based on multiagent Deep Reinforcement Learning (MADRL)-- Twin Delayed Deep Deterministic Policy Gradient (TD3) -- that efficiently learns the control policy to mitigate the cyberattacks on the controllers of EVCS. Also, we have proposed two additional mitigation methods: the manual/Bruteforce mitigation and the controller clone-based mitigation. The attack model considers the APT designed to malfunction the duty cycles of the EVCS controllers with Type-I low-frequency attack and Type-II constant attack. The proposed model restores the EVCS operation under threat incidence in any/all controllers by correcting the control signals generated by the legacy controllers. Also, the TD3 algorithm provides higher granularity by learning nonlinear control policies as compared to the other two mitigation methods. Index Terms: Cyberattack, Deep Reinforcement Learning(DRL), Electric Vehicle Charging Station, Mitigation.Comment: Submitted to IEEE Transactions on Smart Grid

Neuro-fuzzy chip to handle complex tasks with analog performance

This paper presents a mixed-signal neuro-fuzzy controller chip which, in terms of power consumption, input–output delay, and precision, performs as a fully analog implementation. However, it has much larger complexity than its purely analog counterparts. This combination of performance and complexity is achieved through the use of a mixed-signal architecture consisting of a programmable analog core of reduced complexity, and a strategy, and the associated mixed-signal circuitry, to cover the whole input space through the dynamic programming of this core. Since errors and delays are proportional to the reduced number of fuzzy rules included in the analog core, they are much smaller than in the case where the whole rule set is implemented by analog circuitry. Also, the area and the power consumption of the new architecture are smaller than those of its purely analog counterparts simply because most rules are implemented through programming. The Paper presents a set of building blocks associated to this architecture, and gives results for an exemplary prototype. This prototype, called multiplexing fuzzy controller (MFCON), has been realized in a CMOS 0.7 um standard technology. It has two inputs, implements 64 rules, and features 500 ns of input to output delay with 16-mW of power consumption. Results from the chip in a control application with a dc motor are also provided

Neuro-fuzzy chip to handle complex tasks with analog performance

This Paper presents a mixed-signal neuro-fuzzy controller chip which, in terms of power consumption, input-output delay and precision performs as a fully analog implementation. However, it has much larger complexity than its purely analog counterparts. This combination of performance and complexity is achieved through the use of a mixed-signal architecture consisting of a programmable analog core of reduced complexity, and a strategy, and the associated mixed-signal circuitry, to cover the whole input space through the dynamic programming of this core [1]. Since errors and delays are proportional to the reduced number of fuzzy rules included in the analog core, they are much smaller than in the case where the whole rule set is implemented by analog circuitry. Also, the area and the power consumption of the new architecture are smaller than those of its purely analog counterparts simply because most rules are implemented through programming. The Paper presents a set of building blocks associated to this architecture, and gives results for an exemplary prototype. This prototype, called MFCON, has been realized in a CMOS 0.7μm standard technology. It has two inputs, implements 64 rules and features 500ns of input to output delay with 16mW of power consumption. Results from the chip in a control application with a DC motor are also provided

Deep Learning -Powered Computational Intelligence for Cyber-Attacks Detection and Mitigation in 5G-Enabled Electric Vehicle Charging Station

An electric vehicle charging station (EVCS) infrastructure is the backbone of transportation electrification. However, the EVCS has various cyber-attack vulnerabilities in software, hardware, supply chain, and incumbent legacy technologies such as network, communication, and control. Therefore, proactively monitoring, detecting, and defending against these attacks is very important. The state-of-the-art approaches are not agile and intelligent enough to detect, mitigate, and defend against various cyber-physical attacks in the EVCS system. To overcome these limitations, this dissertation primarily designs, develops, implements, and tests the data-driven deep learning-powered computational intelligence to detect and mitigate cyber-physical attacks at the network and physical layers of 5G-enabled EVCS infrastructure. Also, the 5G slicing application to ensure the security and service level agreement (SLA) in the EVCS ecosystem has been studied. Various cyber-attacks such as distributed denial of services (DDoS), False data injection (FDI), advanced persistent threats (APT), and ransomware attacks on the network in a standalone 5G-enabled EVCS environment have been considered. Mathematical models for the mentioned cyber-attacks have been developed. The impact of cyber-attacks on the EVCS operation has been analyzed. Various deep learning-powered intrusion detection systems have been proposed to detect attacks using local electrical and network fingerprints. Furthermore, a novel detection framework has been designed and developed to deal with ransomware threats in high-speed, high-dimensional, multimodal data and assets from eccentric stakeholders of the connected automated vehicle (CAV) ecosystem. To mitigate the adverse effects of cyber-attacks on EVCS controllers, novel data-driven digital clones based on Twin Delayed Deep Deterministic Policy Gradient (TD3) Deep Reinforcement Learning (DRL) has been developed. Also, various Bruteforce, Controller clones-based methods have been devised and tested to aid the defense and mitigation of the impact of the attacks of the EVCS operation. The performance of the proposed mitigation method has been compared with that of a benchmark Deep Deterministic Policy Gradient (DDPG)-based digital clones approach. Simulation results obtained from the Python, Matlab/Simulink, and NetSim software demonstrate that the cyber-attacks are disruptive and detrimental to the operation of EVCS. The proposed detection and mitigation methods are effective and perform better than the conventional and benchmark techniques for the 5G-enabled EVCS

A predictive energy management strategy for multi-mode plug-in hybrid electric vehicles based on multi neural networks

Online optimal energy management of plug-in hybrid electric vehicles has been continually investigated for better fuel economy. This paper proposed a predictive energy management strategy based on multi neural networks for a multi-mode plug-in hybrid electric vehicle. To attain it, firstly, the offline optimal results prepared for knowledge learning are derived by dynamic programming and Pontryagin’s minimum principle. Then, the mode recognition neural network is trained based on the optimal results of dynamic programming and the recurrent neural network is firstly exploited to realize online co-state estimation application. Consequently, the velocity prediction-based online model predictive control framework is established with the co-state correction and slacked constraints to solve the real-time optimal control sequence. A series of numerical simulation results validate that the optimal performance yielded from global optimal strategy can be exploited online to attain the satisfied cost reduction, compared with equivalent consumption minimum strategy, with the assistance of estimated real time co-state and slacked reference. In addition, the computation duration of proposed algorithm decreases by 23.40%, compared with conventional Pontryagin’s minimum principle-based model predictive control scheme, thereby proving its online application potential

Battery State-of-Charge Estimation Using Neural Networks

This thesis proposes a way to augment the existing machine learning algorithm applied to state-of-charge estimation by introducing a form of pulse injection to the running battery cells. It is believed that the information contained in the pulse responses can be interpreted by a machine learning algorithm whereas other techniques are difficult to decode due to the nonlinearity. OCV Mapping is also applied in order to evaluate and compare the performances with feedforward neural network (FNN) -based approach. Coulomb-counting is selected as the basis of making comparison as it is capable of obtaining the SoC with an error less than 0.1%. The detailed system layout is given to perform the augmented SoC estimation integrated in a real-world testbench. Testing procedures specifically designed for both OCV Mapping and FNN-based approach are also explained and provided. A 2-hidden layer FNN is trained to acquire the nonlinear relationship between the training pulse and the ground-truth SoC. The experimental data is trained and the results are shown within 6-8mins computation time and an error boundary of 1.13% for charge and 0.80% for discharge, whereas OCV Mapping has approximately 3.35% SoC estimation error for charge and 1.86% for discharge even after 90 minutes relaxation