Skip to main content
Article thumbnail
Location of Repository

Optimally regularised kernel fisher discriminant analysis

By Kamel Saadi, Nicola L. C. Talbot and Gavin C. Cawley


Mika et al. [1] introduce a non-linear formulation of Fisher’s linear discriminant, based the now familiar “kernel trick”, demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets. In this paper, we extend an existing analytical expression for the leave-one-out cross-validation error [2] such that the leave-one-out error can be re-estimated following a change in the value of the regularisation parameter with a computational complexity of only O(ℓ2) operations, which is substantially less than the O(ℓ3) operations required for the basic training algorithm. This allows the regularisation parameter to be tuned at an essentially negligible computational cost. This is achieved by performing the discriminant analysis in canonical form. The proposed method is therefore a useful component of a model selection strategy for this class of kernel machines that alternates between updates of the kernel and regularisation parameters. Results obtained on real-world and synthetic benchmark datasets indicate that the proposed method is competitive with model selection based on k-fold cross-validation in terms of generalisation, whilst being considerably faster. Key words: model selection, cross-validation, least-squares support vector machin

Year: 2004
OAI identifier: oai:CiteSeerX.psu:
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • (external link)
  • (external link)
  • Suggested articles

    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.