39 research outputs found
Prediction with Expert Advice under Discounted Loss
We study prediction with expert advice in the setting where the losses are
accumulated with some discounting---the impact of old losses may gradually
vanish. We generalize the Aggregating Algorithm and the Aggregating Algorithm
for Regression to this case, propose a suitable new variant of exponential
weights algorithm, and prove respective loss bounds.Comment: 26 pages; expanded (2 remarks -> theorems), some misprints correcte
Cascading Randomized Weighted Majority: A New Online Ensemble Learning Algorithm
With the increasing volume of data in the world, the best approach for
learning from this data is to exploit an online learning algorithm. Online
ensemble methods are online algorithms which take advantage of an ensemble of
classifiers to predict labels of data. Prediction with expert advice is a
well-studied problem in the online ensemble learning literature. The Weighted
Majority algorithm and the randomized weighted majority (RWM) are the most
well-known solutions to this problem, aiming to converge to the best expert.
Since among some expert, the best one does not necessarily have the minimum
error in all regions of data space, defining specific regions and converging to
the best expert in each of these regions will lead to a better result. In this
paper, we aim to resolve this defect of RWM algorithms by proposing a novel
online ensemble algorithm to the problem of prediction with expert advice. We
propose a cascading version of RWM to achieve not only better experimental
results but also a better error bound for sufficiently large datasets.Comment: 15 pages, 3 figure
A Second-order Bound with Excess Losses
We study online aggregation of the predictions of experts, and first show new
second-order regret bounds in the standard setting, which are obtained via a
version of the Prod algorithm (and also a version of the polynomially weighted
average algorithm) with multiple learning rates. These bounds are in terms of
excess losses, the differences between the instantaneous losses suffered by the
algorithm and the ones of a given expert. We then demonstrate the interest of
these bounds in the context of experts that report their confidences as a
number in the interval [0,1] using a generic reduction to the standard setting.
We conclude by two other applications in the standard setting, which improve
the known bounds in case of small excess losses and show a bounded regret
against i.i.d. sequences of losses
Four Facets of Forecast Felicity: Calibration, Predictiveness, Randomness and Regret
Machine learning is about forecasting. Forecasts, however, obtain their
usefulness only through their evaluation. Machine learning has traditionally
focused on types of losses and their corresponding regret. Currently, the
machine learning community regained interest in calibration. In this work, we
show the conceptual equivalence of calibration and regret in evaluating
forecasts. We frame the evaluation problem as a game between a forecaster, a
gambler and nature. Putting intuitive restrictions on gambler and forecaster,
calibration and regret naturally fall out of the framework. In addition, this
game links evaluation of forecasts to randomness of outcomes. Random outcomes
with respect to forecasts are equivalent to good forecasts with respect to
outcomes. We call those dual aspects, calibration and regret, predictiveness
and randomness, the four facets of forecast felicity