8,564 research outputs found
Recommended from our members
Driver Eye Movements and the Application in Autonomous Driving
Despite the exciting progress in computer vision in the field of autonomous driving, understanding efficiently which cues or objects are the most crucial ones in a crowded traffic scene is still a big challenge. Human drivers can quickly identify the important visual cues or objects in their blurry periphery vision and then make eye movements to direct their more accurate foveal vision to the important regions. Therefore, driver eye movements may be what computer vision can borrow from human vision to make autonomous driving systems better at locating and understanding the important regions of crowded traffic scenes. Meanwhile, the large-scale datasets and advanced object recognition algorithms that emerged in the field of autonomous driving make it possible to study classical human vision science problems in natural driving situations. Here, we used driver eye movements to improve autonomous driving models and studied visual crowding—the bottleneck of human object recognition—in realistic driving situations through driver eye movements. First, we developed a new protocol that collects driver eye movements in an offline manner for large-scale driving video datasets. We built a deep neural network that predicts human driver gaze from dash camera videos for various driving scenarios. Our model outperformed the current state-of-the-art model. Furthermore, we incorporated the driver gaze prediction model into an autonomous driving model to make a new periphery-fovea multi-resolution driving model that predicts vehicle speed from dash camera videos. This model combines low-resolution input of the whole video frames and high-resolution input from predicted gaze locations to predict vehicle speed. We show that the added human gaze significantly improves the driving accuracy and that our periphery-fovea multi-resolution model outperforms a uni-resolution periphery-only model that has the same amount of floating-point operations. Finally, we studied visual crowding in driving situations. We show that crowding occurs in natural driving scenes and that the degree of crowding correlates with altered saccade localization in realistic driving-like situations. Together, these studies demonstrate the application of driver eye movements in making safer and more efficient autonomous driving models and show strong evidence of visual crowding in driving situations via the analysis of driver eye movements. These studies also present examples of combining human vision and computer vision to get mutual benefits from both fields
Egocentric Vision-based Future Vehicle Localization for Intelligent Driving Assistance Systems
Predicting the future location of vehicles is essential for safety-critical
applications such as advanced driver assistance systems (ADAS) and autonomous
driving. This paper introduces a novel approach to simultaneously predict both
the location and scale of target vehicles in the first-person (egocentric) view
of an ego-vehicle. We present a multi-stream recurrent neural network (RNN)
encoder-decoder model that separately captures both object location and scale
and pixel-level observations for future vehicle localization. We show that
incorporating dense optical flow improves prediction results significantly
since it captures information about motion as well as appearance change. We
also find that explicitly modeling future motion of the ego-vehicle improves
the prediction accuracy, which could be especially beneficial in intelligent
and automated vehicles that have motion planning capability. To evaluate the
performance of our approach, we present a new dataset of first-person videos
collected from a variety of scenarios at road intersections, which are
particularly challenging moments for prediction because vehicle trajectories
are diverse and dynamic.Comment: To appear on ICRA 201
- …