8,449 research outputs found
LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks
In this work, a deep learning approach has been developed to carry out road
detection by fusing LIDAR point clouds and camera images. An unstructured and
sparse point cloud is first projected onto the camera image plane and then
upsampled to obtain a set of dense 2D images encoding spatial information.
Several fully convolutional neural networks (FCNs) are then trained to carry
out road detection, either by using data from a single sensor, or by using
three fusion strategies: early, late, and the newly proposed cross fusion.
Whereas in the former two fusion approaches, the integration of multimodal
information is carried out at a predefined depth level, the cross fusion FCN is
designed to directly learn from data where to integrate information; this is
accomplished by using trainable cross connections between the LIDAR and the
camera processing branches.
To further highlight the benefits of using a multimodal system for road
detection, a data set consisting of visually challenging scenes was extracted
from driving sequences of the KITTI raw data set. It was then demonstrated
that, as expected, a purely camera-based FCN severely underperforms on this
data set. A multimodal system, on the other hand, is still able to provide high
accuracy. Finally, the proposed cross fusion FCN was evaluated on the KITTI
road benchmark where it achieved excellent performance, with a MaxF score of
96.03%, ranking it among the top-performing approaches
Interaction-aware Spatio-temporal Pyramid Attention Networks for Action Classification
Local features at neighboring spatial positions in feature maps have high
correlation since their receptive fields are often overlapped. Self-attention
usually uses the weighted sum (or other functions) with internal elements of
each local feature to obtain its weight score, which ignores interactions among
local features. To address this, we propose an effective interaction-aware
self-attention model inspired by PCA to learn attention maps. Furthermore,
since different layers in a deep network capture feature maps of different
scales, we use these feature maps to construct a spatial pyramid and then
utilize multi-scale information to obtain more accurate attention scores, which
are used to weight the local features in all spatial positions of feature maps
to calculate attention maps. Moreover, our spatial pyramid attention is
unrestricted to the number of its input feature maps so it is easily extended
to a spatio-temporal version. Finally, our model is embedded in general CNNs to
form end-to-end attention networks for action classification. Experimental
results show that our method achieves the state-of-the-art results on the
UCF101, HMDB51 and untrimmed Charades.Comment: Accepted by ECCV201
Bio-Inspired Computer Vision: Towards a Synergistic Approach of Artificial and Biological Vision
To appear in CVIUStudies in biological vision have always been a great source of inspiration for design of computer vision algorithms. In the past, several successful methods were designed with varying degrees of correspondence with biological vision studies, ranging from purely functional inspiration to methods that utilise models that were primarily developed for explaining biological observations. Even though it seems well recognised that computational models of biological vision can help in design of computer vision algorithms, it is a non-trivial exercise for a computer vision researcher to mine relevant information from biological vision literature as very few studies in biology are organised at a task level. In this paper we aim to bridge this gap by providing a computer vision task centric presentation of models primarily originating in biological vision studies. Not only do we revisit some of the main features of biological vision and discuss the foundations of existing computational studies modelling biological vision, but also we consider three classical computer vision tasks from a biological perspective: image sensing, segmentation and optical flow. Using this task-centric approach, we discuss well-known biological functional principles and compare them with approaches taken by computer vision. Based on this comparative analysis of computer and biological vision, we present some recent models in biological vision and highlight a few models that we think are promising for future investigations in computer vision. To this extent, this paper provides new insights and a starting point for investigators interested in the design of biology-based computer vision algorithms and pave a way for much needed interaction between the two communities leading to the development of synergistic models of artificial and biological vision
- …