Mika et al.  introduce a non-linear formulation of Fisher’s linear discriminant, based the now familiar “kernel trick”, demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets. In this paper, we extend an existing analytical expression for the leave-one-out cross-validation error  such that the leave-one-out error can be re-estimated following a change in the value of the regularisation parameter with a computational complexity of only O(ℓ2) operations, which is substantially less than the O(ℓ3) operations required for the basic training algorithm. This allows the regularisation parameter to be tuned at an essentially negligible computational cost. This is achieved by performing the discriminant analysis in canonical form. The proposed method is therefore a useful component of a model selection strategy for this class of kernel machines that alternates between updates of the kernel and regularisation parameters. Results obtained on real-world and synthetic benchmark datasets indicate that the proposed method is competitive with model selection based on k-fold cross-validation in terms of generalisation, whilst being considerably faster. Key words: model selection, cross-validation, least-squares support vector machin
To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.