Rigorous optimisation of multilinear discriminant analysis with Tucker and PARAFAC structures

Abstract Background We propose rigorously optimised supervised feature extraction methods for multilinear data based on Multilinear Discriminant Analysis (MDA) and demonstrate their usage on Electroencephalography (EEG) and simulated data. While existing MDA methods use heuristic optimisation procedures based on an ambiguous Tucker structure, we propose a rigorous approach via optimisation on the cross-product of Stiefel manifolds. We also introduce MDA methods with the PARAFAC structure. We compare the proposed approaches to existing MDA methods and unsupervised multilinear decompositions. Results We find that manifold optimisation substantially improves MDA objective functions relative to existing methods and on simulated data in general improve classification performance. However, we find similar classification performance when applied to the electroencephalography data. Furthermore, supervised approaches substantially outperform unsupervised mulitilinear methods whereas methods with the PARAFAC structure perform similarly to those with Tucker structures. Notably, despite applying the MDA procedures to raw Brain-Computer Interface data, their performances are on par with results employing ample pre-processing and they extract discriminatory patterns similar to the brain activity known to be elicited in the investigated EEG paradigms. Conclusion The proposed usage of manifold optimisation constitutes the first rigorous and monotonous optimisation approach for MDA methods and allows for MDA with the PARAFAC structure. Our results show that MDA methods applied to raw EEG data can extract discriminatory patterns when compared to traditional unsupervised multilinear feature extraction approaches, whereas the proposed PARAFAC structured MDA models provide meaningful patterns of activity

Beyond temperature scaling:Obtaining well-calibrated multiclass probabilities with Dirichlet calibration

Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single corrective multiplicative factor for inputs to the last softmax layer. On non-neural models the existing methods apply binary calibration in a pairwise or one-vs-rest fashion. We propose a natively multiclass calibration method applicable to classifiers from any model class, derived from Dirichlet distributions and generalising the beta calibration method from binary classification. It is easily implemented with neural nets since it is equivalent to log-transforming the uncalibrated probabilities, followed by one linear layer and softmax. Experiments demonstrate improved probabilistic predictions according to multiple measures (confidence-ECE, classwise-ECE, log-loss, Brier score) across a wide range of datasets and classifiers. Parameters of the learned Dirichlet calibration map provide insights to the biases in the uncalibrated model.Comment: Accepted for presentation at NeurIPS 201

Improved functional prediction of proteins by learning kernel combinations in multilabel settings

Background We develop a probabilistic model for combining kernel matrices to predict the function of proteins. It extends previous approaches in that it can handle multiple labels which naturally appear in the context of protein function. Results Explicit modeling of multilabels significantly improves the capability of learning protein function from multiple kernels. The performance and the interpretability of the inference model are further improved by simultaneously predicting the subcellular localization of proteins and by combining pairwise classifiers to consistent class membership estimates. Conclusion For the purpose of functional prediction of proteins, multilabels provide valuable information that should be included adequately in the training process of classifiers. Learning of functional categories gains from co-prediction of subcellular localization. Pairwise separation rules allow very detailed insights into the relevance of different measurements like sequence, structure, interaction data, or expression data. A preliminary version of the software can be downloaded from http://www.inf.ethz.ch/personal/vroth/KernelHMM/.ISSN:1471-210

Support vector regression to estimate the permeability enhancement of potential transdermal enhancers

This is the peer reviewed version of the following article: Shah, A., Sun, Y., Adams, R. G., Davey, N., Wilkinson, S. C. and Moss, G. P. (2016), Support vector regression to estimate the permeability enhancement of potential transdermal enhancers', Journal of Pharmacy and Pharmacology, Vol. 68 (2): 170–184, which has been published in final form at doi:10.1111/jphp.12508. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. © 2016 Royal Pharmaceutical Society.Objectives Searching for chemicals that will safely enhance transdermal drug delivery is a significant challenge. This study applies support vector regression (SVR) for the first time to estimating the optimal formulation design of transdermal hydrocortisone formulations. Methods The aim of this study was to apply SVR methods with two different kernels in order to estimate the enhancement ratio of chemical enhancers of permeability. Key findings A statistically significant regression SVR model was developed. It was found that SVR with a nonlinear kernel provided the best estimate of the enhancement ratio for a chemical enhancer. Conclusions Support vector regression is a viable method to develop predictive models of biological processes, demonstrating improvements over other methods. In addition, the results of this study suggest that a global approach to modelling a biological process may not necessarily be the best method and that a ‘mixed-methods’ approach may be best in optimising predictive models.Peer reviewedFinal Accepted Versio