In the field of generic object tracking numerous attempts have been made to
exploit deep features. Despite all expectations, deep trackers are yet to reach
an outstanding level of performance compared to methods solely based on
handcrafted features. In this paper, we investigate this key issue and propose
an approach to unlock the true potential of deep features for tracking. We
systematically study the characteristics of both deep and shallow features, and
their relation to tracking accuracy and robustness. We identify the limited
data and low spatial resolution as the main challenges, and propose strategies
to counter these issues when integrating deep features for tracking.
Furthermore, we propose a novel adaptive fusion approach that leverages the
complementary properties of deep and shallow features to improve both
robustness and accuracy. Extensive experiments are performed on four
challenging datasets. On VOT2017, our approach significantly outperforms the
top performing tracker from the challenge with a relative gain of 17% in EAO