32 research outputs found
Lightweight Monocular Depth Estimation Model by Joint End-to-End Filter pruning
Convolutional neural networks (CNNs) have emerged as the state-of-the-art in
multiple vision tasks including depth estimation. However, memory and computing
power requirements remain as challenges to be tackled in these models.
Monocular depth estimation has significant use in robotics and virtual reality
that requires deployment on low-end devices. Training a small model from
scratch results in a significant drop in accuracy and it does not benefit from
pre-trained large models. Motivated by the literature of model pruning, we
propose a lightweight monocular depth model obtained from a large trained
model. This is achieved by removing the least important features with a novel
joint end-to-end filter pruning. We propose to learn a binary mask for each
filter to decide whether to drop the filter or not. These masks are trained
jointly to exploit relations between filters at different layers as well as
redundancy within the same layer. We show that we can achieve around 5x
compression rate with small drop in accuracy on the KITTI driving dataset. We
also show that masking can improve accuracy over the baseline with fewer
parameters, even without enforcing compression loss
Geometry meets semantics for semi-supervised monocular depth estimation
Depth estimation from a single image represents a very exciting challenge in
computer vision. While other image-based depth sensing techniques leverage on
the geometry between different viewpoints (e.g., stereo or structure from
motion), the lack of these cues within a single image renders ill-posed the
monocular depth estimation task. For inference, state-of-the-art
encoder-decoder architectures for monocular depth estimation rely on effective
feature representations learned at training time. For unsupervised training of
these models, geometry has been effectively exploited by suitable images
warping losses computed from views acquired by a stereo rig or a moving camera.
In this paper, we make a further step forward showing that learning semantic
information from images enables to improve effectively monocular depth
estimation as well. In particular, by leveraging on semantically labeled images
together with unsupervised signals gained by geometry through an image warping
loss, we propose a deep learning approach aimed at joint semantic segmentation
and depth estimation. Our overall learning framework is semi-supervised, as we
deploy groundtruth data only in the semantic domain. At training time, our
network learns a common feature representation for both tasks and a novel
cross-task loss function is proposed. The experimental findings show how,
jointly tackling depth prediction and semantic segmentation, allows to improve
depth estimation accuracy. In particular, on the KITTI dataset our network
outperforms state-of-the-art methods for monocular depth estimation.Comment: 16 pages, Accepted to ACCV 201
Parallax Motion Effect Generation Through Instance Segmentation And Depth Estimation
Stereo vision is a growing topic in computer vision due to the innumerable
opportunities and applications this technology offers for the development of
modern solutions, such as virtual and augmented reality applications. To
enhance the user's experience in three-dimensional virtual environments, the
motion parallax estimation is a promising technique to achieve this objective.
In this paper, we propose an algorithm for generating parallax motion effects
from a single image, taking advantage of state-of-the-art instance segmentation
and depth estimation approaches. This work also presents a comparison against
such algorithms to investigate the trade-off between efficiency and quality of
the parallax motion effects, taking into consideration a multi-task learning
network capable of estimating instance segmentation and depth estimation at
once. Experimental results and visual quality assessment indicate that the
PyD-Net network (depth estimation) combined with Mask R-CNN or FBNet networks
(instance segmentation) can produce parallax motion effects with good visual
quality.Comment: 2020 IEEE International Conference on Image Processing (ICIP), Abu
Dhabi, United Arab Emirate