54,064 research outputs found
Model fitting for small skin permeability data sets: hyperparameter optimisation in Gaussian Process Regression
This is the pre-peer reviewed version of the following article: Parivash Ashrafi, Yi Sun, Neil Davey, Roderick G. Adams, Simon C. Wilkinson, and Gary Patrick Moss, ‘Model fitting for small skin permeability data sets: hyperparameter optimisation in Gaussian Process Regression’, Journal of Pharmacy and Pharmacology, Vol. 70 (3): 361-373, March 2018, which has been published in final form at https://doi.org/10.1111/jphp.12863. Under embargo until 17 January 2019. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.Objectives The aim of this study was to investigate how to improve predictions from Gaussian Process models by optimising the model hyperparameters. Methods Optimisation methods, including Grid Search, Conjugate Gradient, Random Search, Evolutionary Algorithm and Hyper-prior, were evaluated and applied to previously published data. Data sets were also altered in a structured manner to reduce their size, which retained the range, or ‘chemical space’ of the key descriptors to assess the effect of the data range on model quality. Key findings The Hyper-prior Smoothbox kernel results in the best models for the majority of data sets, and they exhibited significantly better performance than benchmark quantitative structure–permeability relationship (QSPR) models. When the data sets were systematically reduced in size, the different optimisation methods generally retained their statistical quality, whereas benchmark QSPR models performed poorly. Conclusions The design of the data set, and possibly also the approach to validation of the model, is critical in the development of improved models. The size of the data set, if carefully controlled, was not generally a significant factor for these models and that models of excellent statistical quality could be produced from substantially smaller data sets.Peer reviewedFinal Accepted Versio
Inducing Gaussian Process Networks
Gaussian processes (GPs) are powerful but computationally expensive machine
learning models, requiring an estimate of the kernel covariance matrix for
every prediction. In large and complex domains, such as graphs, sets, or
images, the choice of suitable kernel can also be non-trivial to determine,
providing an additional obstacle to the learning task. Over the last decade,
these challenges have resulted in significant advances being made in terms of
scalability and expressivity, exemplified by, e.g., the use of inducing points
and neural network kernel approximations. In this paper, we propose inducing
Gaussian process networks (IGN), a simple framework for simultaneously learning
the feature space as well as the inducing points. The inducing points, in
particular, are learned directly in the feature space, enabling a seamless
representation of complex structured domains while also facilitating scalable
gradient-based learning methods. We consider both regression and (binary)
classification tasks and report on experimental results for real-world data
sets showing that IGNs provide significant advances over state-of-the-art
methods. We also demonstrate how IGNs can be used to effectively model complex
domains using neural network architectures
Forecasting of commercial sales with large scale Gaussian Processes
This paper argues that there has not been enough discussion in the field of
applications of Gaussian Process for the fast moving consumer goods industry.
Yet, this technique can be important as it e.g., can provide automatic feature
relevance determination and the posterior mean can unlock insights on the data.
Significant challenges are the large size and high dimensionality of commercial
data at a point of sale. The study reviews approaches in the Gaussian Processes
modeling for large data sets, evaluates their performance on commercial sales
and shows value of this type of models as a decision-making tool for
management.Comment: 1o pages, 5 figure
Spike-and-Slab Priors for Function Selection in Structured Additive Regression Models
Structured additive regression provides a general framework for complex
Gaussian and non-Gaussian regression models, with predictors comprising
arbitrary combinations of nonlinear functions and surfaces, spatial effects,
varying coefficients, random effects and further regression terms. The large
flexibility of structured additive regression makes function selection a
challenging and important task, aiming at (1) selecting the relevant
covariates, (2) choosing an appropriate and parsimonious representation of the
impact of covariates on the predictor and (3) determining the required
interactions. We propose a spike-and-slab prior structure for function
selection that allows to include or exclude single coefficients as well as
blocks of coefficients representing specific model terms. A novel
multiplicative parameter expansion is required to obtain good mixing and
convergence properties in a Markov chain Monte Carlo simulation approach and is
shown to induce desirable shrinkage properties. In simulation studies and with
(real) benchmark classification data, we investigate sensitivity to
hyperparameter settings and compare performance to competitors. The flexibility
and applicability of our approach are demonstrated in an additive piecewise
exponential model with time-varying effects for right-censored survival times
of intensive care patients with sepsis. Geoadditive and additive mixed logit
model applications are discussed in an extensive appendix
Penalized Likelihood and Bayesian Function Selection in Regression Models
Challenging research in various fields has driven a wide range of
methodological advances in variable selection for regression models with
high-dimensional predictors. In comparison, selection of nonlinear functions in
models with additive predictors has been considered only more recently. Several
competing suggestions have been developed at about the same time and often do
not refer to each other. This article provides a state-of-the-art review on
function selection, focusing on penalized likelihood and Bayesian concepts,
relating various approaches to each other in a unified framework. In an
empirical comparison, also including boosting, we evaluate several methods
through applications to simulated and real data, thereby providing some
guidance on their performance in practice
Decomposing feature-level variation with Covariate Gaussian Process Latent Variable Models
The interpretation of complex high-dimensional data typically requires the
use of dimensionality reduction techniques to extract explanatory
low-dimensional representations. However, in many real-world problems these
representations may not be sufficient to aid interpretation on their own, and
it would be desirable to interpret the model in terms of the original features
themselves. Our goal is to characterise how feature-level variation depends on
latent low-dimensional representations, external covariates, and non-linear
interactions between the two. In this paper, we propose to achieve this through
a structured kernel decomposition in a hybrid Gaussian Process model which we
call the Covariate Gaussian Process Latent Variable Model (c-GPLVM). We
demonstrate the utility of our model on simulated examples and applications in
disease progression modelling from high-dimensional gene expression data in the
presence of additional phenotypes. In each setting we show how the c-GPLVM can
extract low-dimensional structures from high-dimensional data sets whilst
allowing a breakdown of feature-level variability that is not present in other
commonly used dimensionality reduction approaches
- …