Explainable and Advisable Learning for Self-driving Vehicles

Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-to-interpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers, etc., can understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller's output, namely rationalizations. Our work has focused on the challenge of generating introspective explanations of deep models for self-driving vehicles. In Chapter 3, we begin by exploring the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). In the first stage, we use a visual attention model to train a convolution network end-to-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. In Chapter 4, we add an attention-based video-to-text model to produce textual explanations of model actions, e.g. "the car slows down because the road is wet". The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. These explainable systems represent an externalization of tacit knowledge. The network's opaque reasoning is simplified to a situation-specific dependence on a visible object in the image. This makes them brittle and potentially unsafe in situations that do not match training data. In Chapter 5, we propose to address this issue by augmenting training data with natural language advice from a human. Advice includes guidance about what to do and where to attend. We present the first step toward advice-giving, where we train an end-to-end vehicle controller that accepts advice. The controller adapts the way it attends to the scene (visual attention) and the control (steering and speed). Further, in Chapter 6, we propose a new approach that learns vehicle control with the help of long-term (global) human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (e.g. "I see a pedestrian crossing, so I stop"), and predict the controls, accordingly

A novel Big Data analytics and intelligent technique to predict driver's intent

Modern age offers a great potential for automatically predicting the driver's intent through the increasing miniaturization of computing technologies, rapid advancements in communication technologies and continuous connectivity of heterogeneous smart objects. Inside the cabin and engine of modern cars, dedicated computer systems need to possess the ability to exploit the wealth of information generated by heterogeneous data sources with different contextual and conceptual representations. Processing and utilizing this diverse and voluminous data, involves many challenges concerning the design of the computational technique used to perform this task. In this paper, we investigate the various data sources available in the car and the surrounding environment, which can be utilized as inputs in order to predict driver's intent and behavior. As part of investigating these potential data sources, we conducted experiments on e-calendars for a large number of employees, and have reviewed a number of available geo referencing systems. Through the results of a statistical analysis and by computing location recognition accuracy results, we explored in detail the potential utilization of calendar location data to detect the driver's intentions. In order to exploit the numerous diverse data inputs available in modern vehicles, we investigate the suitability of different Computational Intelligence (CI) techniques, and propose a novel fuzzy computational modelling methodology. Finally, we outline the impact of applying advanced CI and Big Data analytics techniques in modern vehicles on the driver and society in general, and discuss ethical and legal issues arising from the deployment of intelligent self-learning cars

Learning control policies of driverless vehicles from UAV video streams in complex urban environments

© 2019 by the authors. The way we drive, and the transport of today are going through radical changes. Intelligent mobility envisions to improve the e°ciency of traditional transportation through advanced digital technologies, such as robotics, artificial intelligence and Internet of Things. Central to the development of intelligent mobility technology is the emergence of connected autonomous vehicles (CAVs) where vehicles are capable of navigating environments autonomously. For this to be achieved, autonomous vehicles must be safe, trusted by passengers, and other drivers. However, it is practically impossible to train autonomous vehicles with all the possible tra°c conditions that they may encounter. The work in this paper presents an alternative solution of using infrastructure to aid CAVs to learn driving policies, specifically for complex junctions, which require local experience and knowledge to handle. The proposal is to learn safe driving policies through data-driven imitation learning of human-driven vehicles at a junction utilizing data captured from surveillance devices about vehicle movements at the junction. The proposed framework is demonstrated by processing video datasets captured from uncrewed aerial vehicles (UAVs) from three intersections around Europe which contain vehicle trajectories. An imitation learning algorithm based on long short-term memory (LSTM) neural network is proposed to learn and predict safe trajectories of vehicles. The proposed framework can be used for many purposes in intelligent mobility, such as augmenting the intelligent control algorithms in driverless vehicles, benchmarking driver behavior for insurance purposes, and for providing insights to city planning

A systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality.

학위논문(석사) -- 서울대학교대학원 : 행정대학원 글로벌행정전공, 2023. 2. 최태현.The society is anticipated to gain a lot from Autonomous Vehicles (AV), such as improved traffic flow and a decrease in accidents. They heavily rely on improvements in various Artificial Intelligence (AI) processes and strategies. Though some researchers in this field believe AV is the key to enhancing safety, others believe AV creates new challenges when it comes to ensuring the security of these new technology/systems and applications. The article conducts a systematic literature review on the relationship between autonomous vehicle technology and traffic-related mortality. According to inclusion and exclusion criteria, articles from EBSCO, ProQuest, IEEE Explorer, Web of Science were chosen, and they were then sorted. The findings reveal that the most of these publications have been published in advanced transport-related journals. Future improvements in the automobile industry and the development of intelligent transportation systems could help reduce the number of fatal traffic accidents. Technologies for autonomous cars provide effective ways to enhance the driving experience and reduce the number of traffic accidents. A multitude of driving-related problems, such as crashes, traffic, energy usage, and environmental pollution, will be helped by autonomous driving technology. More research is needed for the significant majority of the studies that were assessed. They need to be expanded so that they can be tested in real-world or computer-simulated scenarios, in better and more realistic scenarios, with better and more data, and in experimental designs where the results of the proposed strategy are compared to those of industry standards and competing strategies. Therefore, additional study with improved methods is needed. Another major area that requires additional research is the moral and ethical choices made by AVs. Government, policy makers, manufacturers, and designers all need to do many actions in order to deploy autonomous vehicles on the road effectively. The government should develop laws, rules, and an action plan in particular. It is important to create more effective programs that might encourage the adoption of emerging technology in transportation systems, such as driverless vehicles. In this regard, user perception becomes essential since it may inform designers about current issues and observations made by people. The perceptions of autonomous car users in developing countries like Azerbaijan haven't been thoroughly studied up to this point. The manufacturer has to fix the system flaw and needs a good data set for efficient operation. In the not-too-distant future, the widespread use of highly automated vehicles (AVs) may open up intriguing new possibilities for resolving persistent issues in current safety-related research. Further research is required to better understand and quantify the significant policy implications of Avs, taking into consideration factors like penetration rate, public adoption, technological advancements, traffic patterns, and business models. It only needs to take into account peer-reviewed, full-text journal papers for the investigation, but it's clear that a larger database and more documents would provide more results and a more thorough analysis.자율주행차(AV)를 통해 교통 흐름이 개선되고 사고가 줄어드는 등 사회가 얻는 것이 많을 것으로 예상된다. 그들은 다양한 인공지능(AI) 프로세스와 전략의 개선에 크게 의존한다. 이 분야의 일부 연구자들은 AV가 안전성을 향상시키는 열쇠라고 믿지만, 다른 연구자들은 AV가 이러한 새로운 기술/시스템 및 애플리케이션의 보안을 보장하는 것과 관련하여 새로운 문제를 야기한다고 믿는다. 이 논문은 자율주행차 기술과 교통 관련 사망률 사이의 관계에 대한 체계적인 문헌 검토를 수행한다. 포함 및 제외 기준에 따라 EBSCO, ProQuest, IEEE Explorer 및 Web of Science의 기사를 선택하고 분류했다.연구 결과는 이러한 출판물의 대부분이 고급 운송 관련 저널에 게재되었음을 보여준다. 미래의 자동차 산업의 개선과 지능형 교통 시스템의 개발은 치명적인 교통 사고의 수를 줄이는 데 도움이 될 수 있다. 자율주행 자동차 기술은 운전 경험을 향상시키고 교통 사고의 수를 줄일 수 있는 효과적인 방법을 제공한다. 충돌, 교통, 에너지 사용, 환경 오염과 같은 수많은 운전 관련 문제들은 자율 주행 기술에 의해 도움을 받을 것이다. 평가된 대부분의 연구에 대해 더 많은 연구가 필요하다. 실제 또는 컴퓨터 시뮬레이션 시나리오, 더 좋고 현실적인 시나리오, 더 좋고 더 많은 데이터, 그리고 제안된 전략 결과가 산업 표준 및 경쟁 전략의 결과와 비교되는 실험 설계에서 테스트될 수 있도록 확장되어야 한다. 따라서 개선된 방법에 대한 추가 연구가 필요하다. 추가 연구가 필요한 또 다른 주요 분야는 AV의 도덕적, 윤리적 선택이다. 정부, 정책 입안자, 제조업체 및 설계자는 모두 자율 주행 차량을 효과적으로 도로에 배치하기 위해 많은 조치를 취해야 한다. 정부는 특히 법, 규칙, 실행 계획을 개발해야 한다. 운전자 없는 차량과 같은 운송 시스템에서 새로운 기술의 채택을 장려할 수 있는 보다 효과적인 프로그램을 만드는 것이 중요하다. 이와 관련하여, 설계자에게 현재 이슈와 사람에 의한 관찰을 알려줄 수 있기 때문에 사용자 인식이 필수적이 된다.제조업체는 시스템 결함을 수정해야 하며 효율적인 작동을 위해 좋은 데이터 세트가 필요하다. 멀지 않은 미래에, 고도로 자동화된 차량(AV)의 광범위한 사용은 현재의 안전 관련 연구에서 지속적인 문제를 해결하기 위한 흥미로운 새로운 가능성을 열어줄 수 있다. 보급률, 공공 채택, 기술 발전, 교통 패턴 및 비즈니스 모델과 같은 요소를 고려하여 Avs의 중요한 정책 영향을 더 잘 이해하고 정량화하기 위한 추가 연구가 필요하다. 조사를 위해 동료 검토를 거친 전문 저널 논문만 고려하면 되지만, 데이터베이스가 커지고 문서가 많아지면 더 많은 결과와 더 철저한 분석이 제공될 것이 분명하다.Abstract 3 Table of Contents 6 List of Tables 7 List of Figures 7 List of Appendix 7 CHAPTER 1: INTRODUCTION 8 1.1. Background 8 1.2. Purpose of Research 13 CHAPTER 2: AUTONOMOUS VEHICLES 21 2.1. Intelligent Traffic Systems 21 2.2. System Architecture for Autonomous Vehicles 22 2.3. Key components in AV classification 27 CHAPTER 3: METHODOLOGY AND DATA COLLECTION PROCEDURE 35 CHAPTER 4: FINDINGS AND DISCUSSION 39 4.1. RQ1: Do autonomous vehicles reduce traffic-related deaths 40 4.2. RQ2: Are there any challenges to using autonomous vehicles 63 4.3. RQ3: As a developing country, how effective is the use of autonomous vehicles for reducing traffic mortality 72 CHAPTER 5: CONCLUSION 76 5.1. Summary 76 5.2. Implications and Recommendations 80 5.3. Limitation of the study 91 Bibliography 93 List of Tables Table 1: The 6 Levels of Autonomous Vehicles Table 2: Search strings Table 3: Inclusion and exclusion criteria List of Figures Figure 1: Traffic Death Comparison with Europe Figure 2: Research strategy and study selection process List of Appendix Appendix 1: List of selected articles석

Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions

[EN] Advances in information and signal processing technologies have a significant impact on autonomous driving (AD), improving driving safety while minimizing the efforts of human drivers with the help of advanced artificial intelligence (AI) techniques. Recently, deep learning (DL) approaches have solved several real-world problems of complex nature. However, their strengths in terms of control processes for AD have not been deeply investigated and highlighted yet. This survey highlights the power of DL architectures in terms of reliability and efficient real-time performance and overviews state-of-the-art strategies for safe AD, with their major achievements and limitations. Furthermore, it covers major embodiments of DL along the AD pipeline including measurement, analysis, and execution, with a focus on road, lane, vehicle, pedestrian, drowsiness detection, collision avoidance, and traffic sign detection through sensing and vision-based DL methods. In addition, we discuss on the performance of several reviewed methods by using different evaluation metrics, with critics on their pros and cons. Finally, this survey highlights the current issues of safe DL-based AD with a prospect of recommendations for future research, rounding up a reference material for newcomers and researchers willing to join this vibrant area of Intelligent Transportation Systems.This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) Grant funded by the Korea Government (MSIT) (2019-0-00136, Development of AI-Convergence Technologies for Smart City Industry Productivity Innovation); The work of Javier Del Ser was supported by the Basque Government through the EMAITEK and ELKARTEK Programs, as well as by the Department of Education of this institution (Consolidated Research Group MATHMODE, IT1294-19); VHCA received support from the Brazilian National Council for Research and Development (CNPq, Grant #304315/2017-6 and #430274/2018-1).Muhammad, K.; Ullah, A.; Lloret, J.; Del Ser, J.; De Albuquerque, VHC. (2021). Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions. IEEE Transactions on Intelligent Transportation Systems. 22(7):4316-4336. https://doi.org/10.1109/TITS.2020.30322274316433622