Forecasting motion and spatial positions of objects is of fundamental
importance, especially in safety-critical settings such as autonomous driving.
In this work, we address the issue by forecasting two different modalities that
carry complementary information, namely optical flow and depth. To this end we
propose FLODCAST a flow and depth forecasting model that leverages a multitask
recurrent architecture, trained to jointly forecast both modalities at once. We
stress the importance of training using flows and depth maps together,
demonstrating that both tasks improve when the model is informed of the other
modality. We train the proposed model to also perform predictions for several
timesteps in the future. This provides better supervision and leads to more
precise predictions, retaining the capability of the model to yield outputs
autoregressively for any future time horizon. We test our model on the
challenging Cityscapes dataset, obtaining state of the art results for both
flow and depth forecasting. Thanks to the high quality of the generated flows,
we also report benefits on the downstream task of segmentation forecasting,
injecting our predictions in a flow-based mask-warping framework.Comment: Submitted to Pattern Recognitio