76,311 research outputs found
Towards Visual Ego-motion Learning in Robots
Many model-based Visual Odometry (VO) algorithms have been proposed in the
past decade, often restricted to the type of camera optics, or the underlying
motion manifold observed. We envision robots to be able to learn and perform
these tasks, in a minimally supervised setting, as they gain more experience.
To this end, we propose a fully trainable solution to visual ego-motion
estimation for varied camera optics. We propose a visual ego-motion learning
architecture that maps observed optical flow vectors to an ego-motion density
estimate via a Mixture Density Network (MDN). By modeling the architecture as a
Conditional Variational Autoencoder (C-VAE), our model is able to provide
introspective reasoning and prediction for ego-motion induced scene-flow.
Additionally, our proposed model is especially amenable to bootstrapped
ego-motion learning in robots where the supervision in ego-motion estimation
for a particular camera sensor can be obtained from standard navigation-based
sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through
experiments, we show the utility of our proposed approach in enabling the
concept of self-supervised learning for visual ego-motion estimation in
autonomous robots.Comment: Conference paper; Submitted to IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS) 2017, Vancouver CA; 8 pages, 8 figures,
2 table
FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network
Pedestrian intention recognition is very important to develop robust and safe
autonomous driving (AD) and advanced driver assistance systems (ADAS)
functionalities for urban driving. In this work, we develop an end-to-end
pedestrian intention framework that performs well on day- and night- time
scenarios. Our framework relies on objection detection bounding boxes combined
with skeletal features of human pose. We study early, late, and combined (early
and late) fusion mechanisms to exploit the skeletal features and reduce false
positives as well to improve the intention prediction performance. The early
fusion mechanism results in AP of 0.89 and precision/recall of 0.79/0.89 for
pedestrian intention classification. Furthermore, we propose three new metrics
to properly evaluate the pedestrian intention systems. Under these new
evaluation metrics for the intention prediction, the proposed end-to-end
network offers accurate pedestrian intention up to half a second ahead of the
actual risky maneuver.Comment: 5 pages, 6 figures, 5 tables, IEEE Asilomar SS
Reactive direction control for a mobile robot: A locust-like control of escape direction emerges when a bilateral pair of model locust visual neurons are integrated
Locusts possess a bilateral pair of uniquely identifiable visual neurons that respond vigorously to
the image of an approaching object. These neurons are called the lobula giant movement
detectors (LGMDs). The locust LGMDs have been extensively studied and this has lead to the
development of an LGMD model for use as an artificial collision detector in robotic applications.
To date, robots have been equipped with only a single, central artificial LGMD sensor, and this
triggers a non-directional stop or rotation when a potentially colliding object is detected. Clearly,
for a robot to behave autonomously, it must react differently to stimuli approaching from
different directions. In this study, we implement a bilateral pair of LGMD models in Khepera
robots equipped with normal and panoramic cameras. We integrate the responses of these LGMD
models using methodologies inspired by research on escape direction control in cockroaches.
Using ‘randomised winner-take-all’ or ‘steering wheel’ algorithms for LGMD model integration,
the khepera robots could escape an approaching threat in real time and with a similar
distribution of escape directions as real locusts. We also found that by optimising these
algorithms, we could use them to integrate the left and right DCMD responses of real jumping
locusts offline and reproduce the actual escape directions that the locusts took in a particular
trial. Our results significantly advance the development of an artificial collision detection and
evasion system based on the locust LGMD by allowing it reactive control over robot behaviour.
The success of this approach may also indicate some important areas to be pursued in future
biological research
Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection
Effective fusion of complementary information captured by multi-modal sensors
(visible and infrared cameras) enables robust pedestrian detection under
various surveillance situations (e.g. daytime and nighttime). In this paper, we
present a novel box-level segmentation supervised learning framework for
accurate and real-time multispectral pedestrian detection by incorporating
features extracted in visible and infrared channels. Specifically, our method
takes pairs of aligned visible and infrared images with easily obtained
bounding box annotations as input and estimates accurate prediction maps to
highlight the existence of pedestrians. It offers two major advantages over the
existing anchor box based multispectral detection methods. Firstly, it
overcomes the hyperparameter setting problem occurred during the training phase
of anchor box based detectors and can obtain more accurate detection results,
especially for small and occluded pedestrian instances. Secondly, it is capable
of generating accurate detection results using small-size input images, leading
to improvement of computational efficiency for real-time autonomous driving
applications. Experimental results on KAIST multispectral dataset show that our
proposed method outperforms state-of-the-art approaches in terms of both
accuracy and speed
- …