18,945 research outputs found
Multi-Target Prediction: A Unifying View on Problems and Methods
Multi-target prediction (MTP) is concerned with the simultaneous prediction
of multiple target variables of diverse type. Due to its enormous application
potential, it has developed into an active and rapidly expanding research field
that combines several subfields of machine learning, including multivariate
regression, multi-label classification, multi-task learning, dyadic prediction,
zero-shot learning, network inference, and matrix completion. In this paper, we
present a unifying view on MTP problems and methods. First, we formally discuss
commonalities and differences between existing MTP problems. To this end, we
introduce a general framework that covers the above subfields as special cases.
As a second contribution, we provide a structured overview of MTP methods. This
is accomplished by identifying a number of key properties, which distinguish
such methods and determine their suitability for different types of problems.
Finally, we also discuss a few challenges for future research
Excess Optimism: How Biased is the Apparent Error of an Estimator Tuned by SURE?
Nearly all estimators in statistical prediction come with an associated
tuning parameter, in one way or another. Common practice, given data, is to
choose the tuning parameter value that minimizes a constructed estimate of the
prediction error of the estimator; we focus on Stein's unbiased risk estimator,
or SURE (Stein, 1981; Efron, 1986) which forms an unbiased estimate of the
prediction error by augmenting the observed training error with an estimate of
the degrees of freedom of the estimator. Parameter tuning via SURE minimization
has been advocated by many authors, in a wide variety of problem settings, and
in general, it is natural to ask: what is the prediction error of the
SURE-tuned estimator? An obvious strategy would be simply use the apparent
error estimate as reported by SURE, i.e., the value of the SURE criterion at
its minimum, to estimate the prediction error of the SURE-tuned estimator. But
this is no longer unbiased; in fact, we would expect that the minimum of the
SURE criterion is systematically biased downwards for the true prediction
error. In this paper, we formally describe and study this bias.Comment: 39 pages, 3 figure
- …