Deep fisher discriminant analysis

Abstract

Fisher Discriminant Analysis’ linear nature and the usual eigen-analysis approach to its solution have limited the application of its underlying elegant idea. In this work we will take advantage of some recent partially equivalent formulations based on standard least squares regression to develop a simple Deep Neural Network (DNN) extension of Fisher’s analysis that greatly improves on its ability to cluster sample projections around their class means while keeping these apart. This is shown by the much better accuracies and g scores of class mean classifiers when applied to the features provided by simple DNN architectures than what can be achieved using Fisher’s linear onesWith partial support from Spain's grants TIN2013-42351- P, TIN2016-76406-P, TIN2015-70308-REDT and S2013/ICE-2845 CASI-CAMCM. Work supported also by project FACIL{Ayudas Fundaci on BBVA a Equipos de Investigación Científica 2016, the UAM{ADIC Chair for Data Science and Machine Learning and Instituto de Ingeniería del Conocimiento. The third author is also supported by the FPU{MEC grant AP-2012-5163. We gratefully acknowledge the use of the facilities of Centro de Computacón Científi ca (CCC) at UA