Increasing the Efficiency of Policy Learning for Autonomous Vehicles by Multi-Task Representation Learning

Driving in a dynamic, multi-agent, and complex urban environment is a difficult task requiring a complex decision-making policy. The learning of such a policy requires a state representation that can encode the entire environment. Mid-level representations that encode a vehicle's environment as images have become a popular choice. Still, they are quite high-dimensional, limiting their use in data-hungry approaches such as reinforcement learning. In this article, we propose to learn a low-dimensional and rich latent representation of the environment by leveraging the knowledge of relevant semantic factors. To do this, we train an encoder-decoder deep neural network to predict multiple application-relevant factors such as the trajectories of other agents and the ego car. Furthermore, we propose a hazard signal based on other vehicles' future trajectories and the planned route which is used in conjunction with the learned latent representation as input to a down-stream policy. We demonstrate that using the multi-head encoder-decoder neural network results in a more informative representation than a standard single-head model. In particular, the proposed representation learning and the hazard signal help reinforcement learning to learn faster, with increased performance and less data than baseline methods

Learning Based High-Level Decision Making for Abortable Overtaking in Autonomous Vehicles

Autonomous vehicles are a growing technology that aims to enhance safety, accessibility, efficiency, and convenience through autonomous maneuvers ranging from lane change to overtaking. Overtaking is one of the most challenging maneuvers for autonomous vehicles, and current techniques for autonomous overtaking are limited to simple situations. This paper studies how to increase safety in autonomous overtaking by allowing the maneuver to be aborted. We propose a decision-making process based on a deep Q-Network to determine if and when the overtaking maneuver needs to be aborted. The proposed algorithm is empirically evaluated in simulation with varying traffic situations, indicating that the proposed method improves safety during overtaking maneuvers. Furthermore, the approach is demonstrated in real-world experiments using the autonomous shuttle iseAuto.Comment: 11 pages, 16 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Increasing the Efficiency of Policy Learning for Autonomous Vehicles by Multi-Task Representation Learning

Tallenna OA-artikkeli, kun julkaistuDriving in a dynamic, multi-agent, and complex urban environment is a difficult task requiring a complex decision-making policy. The learning of such a policy requires a state representation that can encode the entire environment. Mid-level representations that encode a vehicle's environment as images have become a popular choice. Still, they are quite high-dimensional, limiting their use in data-hungry approaches such as reinforcement learning. In this article, we propose to learn a low-dimensional and rich latent representation of the environment by leveraging the knowledge of relevant semantic factors. To do this, we train an encoder-decoder deep neural network to predict multiple application-relevant factors such as the trajectories of other agents and the ego car. Furthermore, we propose a hazard signal based on other vehicles' future trajectories and the planned route which is used in conjunction with the learned latent representation as input to a down-stream policy. We demonstrate that using the multi-head encoder-decoder neural network results in a more informative representation than a standard single-head model. In particular, the proposed representation learning and the hazard signal help reinforcement learning to learn faster, with increased performance and less data than baseline methods.Peer reviewe

Vision Transformer for Learning Driving Policies in Complex and Dynamic Environments

Publisher Copyright: © 2022 IEEE.Driving in a complex and dynamic urban environment is a difficult task that requires a complex decision policy. In order to make informed decisions, one needs to gain an understanding of the long-range context and the importance of other vehicles. In this work, we propose to use Vision Transformer (ViT) to learn a driving policy in urban settings with birds-eye-view (BEV) input images. The ViT network learns the global context of the scene more effectively than with earlier proposed Convolutional Neural Networks (ConvNets). Furthermore, ViT's attention mechanism helps to learn an attention map for the scene which allows the ego car to determine which surrounding cars are important to its next decision. We demonstrate that a DQN agent with a ViT backbone outperforms baseline algorithms with ConvNet backbones pre-trained in various ways. In particular, the proposed method helps reinforcement learning algorithms to learn faster, with increased performance and less data than baselines.Peer reviewe

Evaluation of the Effect of Oral Vitamin B1 on Pain Due to Corneal Neuropathy after Cataract Surgery

Introduction: Cataracts are the leading cause of low vision and blindness in the world, and the only effective treatment for cataract vision impairment is surgery, which has common complications such as eye pain and burning, inflammation, and postoperative headache. The aim of this study was to evaluatethe effect of vitamin B1 on oral pain on corneal neuropathy after cataract surgery in Jiroft.Method and Materials: This randomized clinical trial study was performed on cataract patients who were candidates for surgery and referred to Imam Khomeini Hospital in Jiroft in 2020. Demographic information was collected through a designed checklist and the Numerical Pain Scale (NRS) was used to measure postoperative severity in the eye. The collected data were analyzed using SPSS-V21 software.Results: In this study, 130 patients with cataracts (intervention group: 65 and control group: 65) were studied. The intervention group consisted of 27 men (41.5 %) and 38 women (58.5 %) and the control group consisted of 25 men (38.5 %) and 40 women (61.5 %). On the third day and one week after surgery, there was no significant difference in the amount of eye pain and irritation caused by surgery in the intervention and controlgroups, and in the three months after surgery, the intervention group had mild eye pain and irritation.Conclusion: The results of this study showed that taking vitamin B1 orally affects eye irritation and pain caused by corneal neuropathy after cataract surgery and reduces eye irritation and pain intensity during 3months