193,682 research outputs found
Probability smoothing
Contains fulltext :
227633.pdf (publisher's version ) (Open Access
On Probability Estimation by Exponential Smoothing
Probability estimation is essential for every statistical data compression
algorithm. In practice probability estimation should be adaptive, recent
observations should receive a higher weight than older observations. We present
a probability estimation method based on exponential smoothing that satisfies
this requirement and runs in constant time per letter. Our main contribution is
a theoretical analysis in case of a binary alphabet for various smoothing rate
sequences: We show that the redundancy w.r.t. a piecewise stationary model with
segments is for any bit sequence of length , an
improvement over redundancy of previous
approaches with similar time complexity
Pointwise Convergence in Probability of General Smoothing Splines
Establishing the convergence of splines can be cast as a variational problem
which is amenable to a -convergence approach. We consider the case in
which the regularization coefficient scales with the number of observations,
, as . Using standard theorems from the
-convergence literature, we prove that the general spline model is
consistent in that estimators converge in a sense slightly weaker than weak
convergence in probability for . Without further assumptions
we show this rate is sharp. This differs from rates for strong convergence
using Hilbert scales where one can often choose
Smoothing in Probability Estimation Trees
Classification learning is a type of supervised machine learning technique that uses a classification model (e.g. decision tree) to predict unknown class labels for previously unseen instances. In many applications it can be very useful to additionally obtain class probabilities for the different class labels. Decision trees that yield these probabilities are also called probability estimation trees (PETs). Smoothing is a technique used to improve the probability estimates. There are several existing smoothing methods, such as the Laplace correction, M-Estimate smoothing and M-Branch smoothing. Smoothing does not just apply to PETs. In the field of text compression, PPM in particular, smoothing methods play a important role. This thesis migrates smoothing methods from text compression to PETs. The newly migrated methods in PETs are compared with the best of the existing smoothing methods considered in this thesis under different experiment setups. Unpruned, pruned and bagged trees are considered in the experiments. The main finding is that the PPM-based methods yield the best probability estimate when used with bagged trees, but not when used with individual (pruned or unpruned) trees
- …