Human-Factors-in-Driving-Loop: Driver Identification and Verification via a Deep Learning Approach using Psychological Behavioral Data

Driver identification has been popular in the field of driving behavior analysis, which has a broad range of applications in anti-thief, driving style recognition, insurance strategy, and fleet management. However, most studies to date have only researched driver identification without a robust verification stage. This paper addresses driver identification and verification through a deep learning (DL) approach using psychological behavioral data, i.e., vehicle control operation data and eye movement data collected from a driving simulator and an eye tracker, respectively. We design an architecture that analyzes the segmentation windows of three-second data to capture unique driving characteristics and then differentiate drivers on that basis. The proposed model includes a fully convolutional network (FCN) and a squeeze-and-excitation (SE) block. Experimental results were obtained from 24 human participants driving in 12 different scenarios. The proposed driver identification system achieves an accuracy of 99.60% out of 15 drivers. To tackle driver verification, we combine the proposed architecture and a Siamese neural network, and then map all behavioral data into two embedding layers for similarity computation. The identification system achieves significant performance with average precision of 96.91%, recall of 95.80%, F1 score of 96.29%, and accuracy of 96.39%, respectively. Importantly, we scale out the verification system to imposter detection and achieve an average verification accuracy of 90.91%. These results imply the invariable characteristics from human factors rather than other traditional resources, which provides a superior solution for driving behavior authentication systems

GAN-CAN: A Novel Attack to Behavior-Based Driver Authentication Systems

openFor many years, car keys have been the sole mean of authentication in vehicles. Whether the access control process is physical or wireless, entrusting the ownership of a vehicle to a single token is prone to stealing attempts. Modern vehicles equipped with the Controller Area Network (CAN) bus technology collects a wealth of sensor data in real-time, covering aspects such as the vehicle, environment, and driver. This data can be processed and analyzed to gain valuable insights and solutions for human behavior analysis. For this reason, many researchers started developing behavior-based authentication systems. Many Machine Learning (ML) and Deep Learning models (DL) have been explored for behavior-based driver authentication, but the emphasis on security has not been a primary focus in the design of these systems. By collecting data in a moving vehicle, DL models can recognize patterns in the data and identify drivers based on their driving behavior. This can be used as an anti-theft system, as a thief would exhibit a different driving style compared to the vehicle owner. However, the assumption that an attacker cannot replicate the legitimate driver behavior falls under certain conditions. In this thesis, we propose GAN-CAN, the first attack capable of fooling state-of-the-art behavior-based driver authentication systems in a vehicle. Based on the adversary's knowledge, we propose different GAN-CAN implementations. Our attack leverages the lack of security in the CAN bus to inject suitably designed time-series data to mimic the legitimate driver. Our malicious time series data is generated through the integration of a modified reinforcement learning technique with Generative Adversarial Networks (GANs) with adapted training process. Furthermore we conduct a thorough investigation into the safety implications of the injected values throughout the attack. This meticulous study is conducted to guarantee that the introduced values do not in any way undermine the safety of the vehicle and the individuals inside it. Also, we formalize a real-world implementation of a driver authentication system considering possible vulnerabilities and exploits. We tested GAN-CAN in an improved version of the most efficient driver behavior-based authentication model in the literature. We prove that our attack can fool it with an attack success rate of up to 99%. We show how an attacker, without prior knowledge of the authentication system, can steal a car by deploying GAN-CAN in an off-the-shelf system in under 22 minutes. Moreover, by considering the safety importance of the injected values, we demonstrate that GAN-CAN can successfully deceive the authentication system without compromising the overall safety of the vehicle. This highlights the urgent need to address the security vulnerabilities present in behavior-based driver authentication systems. In the end, we suggest some possible countermeasures to the GAN-CAN attack.For many years, car keys have been the sole mean of authentication in vehicles. Whether the access control process is physical or wireless, entrusting the ownership of a vehicle to a single token is prone to stealing attempts. Modern vehicles equipped with the Controller Area Network (CAN) bus technology collects a wealth of sensor data in real-time, covering aspects such as the vehicle, environment, and driver. This data can be processed and analyzed to gain valuable insights and solutions for human behavior analysis. For this reason, many researchers started developing behavior-based authentication systems. Many Machine Learning (ML) and Deep Learning models (DL) have been explored for behavior-based driver authentication, but the emphasis on security has not been a primary focus in the design of these systems. By collecting data in a moving vehicle, DL models can recognize patterns in the data and identify drivers based on their driving behavior. This can be used as an anti-theft system, as a thief would exhibit a different driving style compared to the vehicle owner. However, the assumption that an attacker cannot replicate the legitimate driver behavior falls under certain conditions. In this thesis, we propose GAN-CAN, the first attack capable of fooling state-of-the-art behavior-based driver authentication systems in a vehicle. Based on the adversary's knowledge, we propose different GAN-CAN implementations. Our attack leverages the lack of security in the CAN bus to inject suitably designed time-series data to mimic the legitimate driver. Our malicious time series data is generated through the integration of a modified reinforcement learning technique with Generative Adversarial Networks (GANs) with adapted training process. Furthermore we conduct a thorough investigation into the safety implications of the injected values throughout the attack. This meticulous study is conducted to guarantee that the introduced values do not in any way undermine the safety of the vehicle and the individuals inside it. Also, we formalize a real-world implementation of a driver authentication system considering possible vulnerabilities and exploits. We tested GAN-CAN in an improved version of the most efficient driver behavior-based authentication model in the literature. We prove that our attack can fool it with an attack success rate of up to 99%. We show how an attacker, without prior knowledge of the authentication system, can steal a car by deploying GAN-CAN in an off-the-shelf system in under 22 minutes. Moreover, by considering the safety importance of the injected values, we demonstrate that GAN-CAN can successfully deceive the authentication system without compromising the overall safety of the vehicle. This highlights the urgent need to address the security vulnerabilities present in behavior-based driver authentication systems. In the end, we suggest some possible countermeasures to the GAN-CAN attack

Biometrijska identifikacija vozača

U bezbednosti saobraćaja često se javlja potreba za identifikacijom motornog vozila i vozača (identifikacija učionioca prekršaja, krivičnih dela protiv bezbednosti saobraćaja u situacijama odbeglih učesnika u saobraćajnim nezgodama i sl.). Radi identifikacije vozača, učesnika u saobraćaju, mogu se koristiti različite kriminalističko tehničke i biometrijske metode za identifikaciju. Sa napretkom nauke i tehnologije povećava se i mogućnost identifikacije lica, odnosno u ovom slučaju vozača motornog vozila. Tako, mogu se koristiti kamere instalirane na čvornim mestima koje imaju mogućnost identifikacije lica (facial recognition system) korišćenjem adekvatnih softvera, ali i posebni kompjuterski sistemi sa mogućnošću automatske identifikacije vozača. Pojedini kompjuterski senzori novijeg datuma ugrađeni u unutrašnjosti vozila mogu dati informaciju o stilu vožnje vozača, a koji takođe može ukazati na njegov identitet. Sem ovih savremenih metoda, mogu se koristiti i tradicionalne kriminalističko-tehničke metode za identifikaciju kao sto su vizuelno prepoznavanje i lični opis, fotografija, daktiloskopija i identifikacija na osnovu bioloških i drugih tragova. Pojedine tradicionalne metode se mogu koristiti u situacijama kada postoje očevici događaja, dok se neke koriste u situacijama kada postoje tragovi ostavljeni unutar vozila, na osnovu kojih je moguće utvrditi identitet vozača. Ovo je od naročitog značaja kada jedno vozilo koristi vise vozača i kada je potrebno izvršiti identifikaciju koji vozač je u konkretnoj situaciji koristio motorno vozilo. U mnogim kompanijama službena vozila su na korišćenju velikog broja vozača, te je precizna informacija koji vozač je u datom trenutku bio za volanom konkretnog vozila, naročito ako je ono učestvovalo u saobraćajnoj nezgodi, veoma značajna. U ovom radu će biti dat pregled metoda koje se mogu koristiti za identifikaciju vozača

Biometrijska identifikacija vozača

U bezbednosti saobraćaja često se javlja potreba za identifikacijom motornog vozila i vozača (identifikacija učionioca prekršaja, krivičnih dela protiv bezbednosti saobraćaja u situacijama odbeglih učesnika u saobraćajnim nezgodama i sl.). Radi identifikacije vozača, učesnika u saobraćaju, mogu se koristiti različite kriminalističko tehničke i biometrijske metode za identifikaciju. Sa napretkom nauke i tehnologije povećava se i mogućnost identifikacije lica, odnosno u ovom slučaju vozača motornog vozila. Tako, mogu se koristiti kamere instalirane na čvornim mestima koje imaju mogućnost identifikacije lica (facial recognition system) korišćenjem adekvatnih softvera, ali i posebni kompjuterski sistemi sa mogućnošću automatske identifikacije vozača. Pojedini kompjuterski senzori novijeg datuma ugrađeni u unutrašnjosti vozila mogu dati informaciju o stilu vožnje vozača, a koji takođe može ukazati na njegov identitet. Sem ovih savremenih metoda, mogu se koristiti i tradicionalne kriminalističko-tehničke metode za identifikaciju kao sto su vizuelno prepoznavanje i lični opis, fotografija, daktiloskopija i identifikacija na osnovu bioloških i drugih tragova. Pojedine tradicionalne metode se mogu koristiti u situacijama kada postoje očevici događaja, dok se neke koriste u situacijama kada postoje tragovi ostavljeni unutar vozila, na osnovu kojih je moguće utvrditi identitet vozača. Ovo je od naročitog značaja kada jedno vozilo koristi vise vozača i kada je potrebno izvršiti identifikaciju koji vozač je u konkretnoj situaciji koristio motorno vozilo. U mnogim kompanijama službena vozila su na korišćenju velikog broja vozača, te je precizna informacija koji vozač je u datom trenutku bio za volanom konkretnog vozila, naročito ako je ono učestvovalo u saobraćajnoj nezgodi, veoma značajna. U ovom radu će biti dat pregled metoda koje se mogu koristiti za identifikaciju vozača

Entropy in Image Analysis II

Image analysis is a fundamental task for any application where extracting information from images is required. The analysis requires highly sophisticated numerical and analytical methods, particularly for those applications in medicine, security, and other fields where the results of the processing consist of data of vital importance. This fact is evident from all the articles composing the Special Issue "Entropy in Image Analysis II", in which the authors used widely tested methods to verify their results. In the process of reading the present volume, the reader will appreciate the richness of their methods and applications, in particular for medical imaging and image security, and a remarkable cross-fertilization among the proposed research areas

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

Fundamentals

Volume 1 establishes the foundations of this new field. It goes through all the steps from data collection, their summary and clustering, to different aspects of resource-aware learning, i.e., hardware, memory, energy, and communication awareness. Machine learning methods are inspected with respect to resource requirements and how to enhance scalability on diverse computing architectures ranging from embedded systems to large computing clusters

Context-aware intelligent decisions: online assessment of heavy goods vehicle driving risk

There is a growing interest in assessing the impact of drivers' actions and behaviours on road safety due to the numerous road fatalities and costs attributed to them. For Heavy Goods Vehicle (HGV) drivers, assessing the road safety risks of their behaviours is a subject of interest for researchers, governments and transport companies, as nations rely on HGVs for the delivery of goods and services. However, HGV driving is a complex, dynamic, uncertain and multifaceted task, mostly influenced by individual traits and external contextual factors. Advanced computational and artificial intelligence (AI) methods have provided promising solutions to automatically characterise the manner by which drivers operate vehicle controls and assess their impact on road safety. However, several challenges and limitations are faced by the current intelligence-supported driving risk assessment approaches proposed by researchers, such as: (1) the lack of comprehensive driving risk datasets; (2) information about the impact of inevitable contextual factors on HGV drivers' responses is not considered, such as drivers' physical and mental states, weather conditions, traffic conditions, road geometry, road types, and work schedules; (3) ambiguity in the definition of driving behaviours is not considered; and (4) imprecision of AI models, and variability in experts' subjective views are not considered. To overcome the aforementioned challenges and limitations, this multidisciplinary research aims at exploring multiple sources of data including information about the impact of contextual factors captured from crucial stakeholders in the HGV sector to develop a reliable context-aware driving risk assessment framework. To achieve this aim, AI methods are explored to accurately detect drivers' driving styles, affective states and driving postures using telematics data, facial images, and driver posture images respectively. Subsequently, due to the lack of comprehensive driving risk datasets, fuzzy expert systems (FESs) are explored to fuse detected driving behaviours and perceived external factors using knowledge from domain experts. The key findings of this research are: (1) recurrent neural networks are effective in capturing the temporal dynamics and differences between the different types of driver distraction postures and affective states; (2) there is a trade-off between efficiency and privacy in processing facial images using AI approaches; (3) the fusion of driver behaviours and external factors using FESs produces realistic, reliable and fair driving risk assessments; and (4) a hierarchical representation of a decision-making process simplifies reasoning compared to flat representations