54,641 research outputs found
Generalized additive modelling with implicit variable selection by likelihood based boosting
The use of generalized additive models in statistical data analysis suffers from the restriction to few explanatory variables and the problems of selection of smoothing parameters. Generalized additive model boosting circumvents these problems by means of stagewise fitting of weak learners. A fitting procedure is derived which works for all simple exponential family distributions, including binomial, Poisson and normal response variables. The procedure combines the selection of variables and the determination of the appropriate amount of smoothing. As weak learners penalized regression splines and the newly introduced penalized stumps are considered. Estimates of standard deviations and stopping criteria which are notorious problems in iterative procedures are based on an approximate hat matrix. The method is shown to outperform common procedures for the fitting of generalized additive models. In particular in high dimensional settings it is the only method that works properly
An update on statistical boosting in biomedicine
Statistical boosting algorithms have triggered a lot of research during the
last decade. They combine a powerful machine-learning approach with classical
statistical modelling, offering various practical advantages like automated
variable selection and implicit regularization of effect estimates. They are
extremely flexible, as the underlying base-learners (regression functions
defining the type of effect for the explanatory variables) can be combined with
any kind of loss function (target function to be optimized, defining the type
of regression setting). In this review article, we highlight the most recent
methodological developments on statistical boosting regarding variable
selection, functional regression and advanced time-to-event modelling.
Additionally, we provide a short overview on relevant applications of
statistical boosting in biomedicine
Fast stable direct fitting and smoothness selection for Generalized Additive Models
Existing computationally efficient methods for penalized likelihood GAM
fitting employ iterative smoothness selection on working linear models (or
working mixed models). Such schemes fail to converge for a non-negligible
proportion of models, with failure being particularly frequent in the presence
of concurvity. If smoothness selection is performed by optimizing `whole model'
criteria these problems disappear, but until now attempts to do this have
employed finite difference based optimization schemes which are computationally
inefficient, and can suffer from false convergence. This paper develops the
first computationally efficient method for direct GAM smoothness selection. It
is highly stable, but by careful structuring achieves a computational
efficiency that leads, in simulations, to lower mean computation times than the
schemes based on working-model smoothness selection. The method also offers a
reliable way of fitting generalized additive mixed models
Global sensitivity analysis of computer models with functional inputs
Global sensitivity analysis is used to quantify the influence of uncertain
input parameters on the response variability of a numerical model. The common
quantitative methods are applicable to computer codes with scalar input
variables. This paper aims to illustrate different variance-based sensitivity
analysis techniques, based on the so-called Sobol indices, when some input
variables are functional, such as stochastic processes or random spatial
fields. In this work, we focus on large cpu time computer codes which need a
preliminary meta-modeling step before performing the sensitivity analysis. We
propose the use of the joint modeling approach, i.e., modeling simultaneously
the mean and the dispersion of the code outputs using two interlinked
Generalized Linear Models (GLM) or Generalized Additive Models (GAM). The
``mean'' model allows to estimate the sensitivity indices of each scalar input
variables, while the ``dispersion'' model allows to derive the total
sensitivity index of the functional input variables. The proposed approach is
compared to some classical SA methodologies on an analytical function. Lastly,
the proposed methodology is applied to a concrete industrial computer code that
simulates the nuclear fuel irradiation
Axiomatic Interpretability for Multiclass Additive Models
Generalized additive models (GAMs) are favored in many regression and binary
classification problems because they are able to fit complex, nonlinear
functions while still remaining interpretable. In the first part of this paper,
we generalize a state-of-the-art GAM learning algorithm based on boosted trees
to the multiclass setting, and show that this multiclass algorithm outperforms
existing GAM learning algorithms and sometimes matches the performance of full
complexity models such as gradient boosted trees.
In the second part, we turn our attention to the interpretability of GAMs in
the multiclass setting. Surprisingly, the natural interpretability of GAMs
breaks down when there are more than two classes. Naive interpretation of
multiclass GAMs can lead to false conclusions. Inspired by binary GAMs, we
identify two axioms that any additive model must satisfy in order to not be
visually misleading. We then develop a technique called Additive
Post-Processing for Interpretability (API), that provably transforms a
pre-trained additive model to satisfy the interpretability axioms without
sacrificing accuracy. The technique works not just on models trained with our
learning algorithm, but on any multiclass additive model, including multiclass
linear and logistic regression. We demonstrate the effectiveness of API on a
12-class infant mortality dataset.Comment: KDD 201
- …