Facial landmark detection by deep multi-task learning

Abstract

Abstract. Facial landmark detection has long been impeded by the problems of occlusion and pose variation. Instead of treating the de-tection task as a single and independent problem, we investigate the possibility of improving detection robustness through multi-task learn-ing. Specifically, we wish to optimize facial landmark detection together with heterogeneous but subtly correlated tasks, e.g.head pose estimation and facial attribute inference. This is non-trivial since different tasks have different learning difficulties and convergence rates. To address this prob-lem, we formulate a novel tasks-constrained deep model, with task-wise early stopping to facilitate learning convergence. Extensive evaluations show that the proposed task-constrained learning (i) outperforms exist-ing methods, especially in dealing with faces with severe occlusion and pose variation, and (ii) reduces model complexity drastically compared to the state-of-the-art method based on cascaded deep model [21].