Fast learning rates for plug-in classifiers

It has been recently shown that, under the margin (or low noise) assumption, there exist classifiers attaining fast rates of convergence of the excess Bayes risk, that is, rates faster than $n^{-1/2}$. The work on this subject has suggested the following two conjectures: (i) the best achievable fast rate is of the order $n^{-1}$, and (ii) the plug-in classifiers generally converge more slowly than the classifiers based on empirical risk minimization. We show that both conjectures are not correct. In particular, we construct plug-in classifiers that can achieve not only fast, but also super-fast rates, that is, rates faster than $n^{-1}$. We establish minimax lower bounds showing that the obtained rates cannot be improved.Comment: Published at http://dx.doi.org/10.1214/009053606000001217 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Fast learning rates for plug-in classifiers under the margin condition

It has been recently shown that, under the margin (or low noise) assumption, there exist classifiers attaining fast rates of convergence of the excess Bayes risk, i.e., the rates faster than $n^{-1/2}$. The works on this subject suggested the following two conjectures: (i) the best achievable fast rate is of the order $n^{-1}$, and (ii) the plug-in classifiers generally converge slower than the classifiers based on empirical risk minimization. We show that both conjectures are not correct. In particular, we construct plug-in classifiers that can achieve not only the fast, but also the {\it super-fast} rates, i.e., the rates faster than $n^{-1}$. We establish minimax lower bounds showing that the obtained rates cannot be improved.Comment: 36 page

Relative complexity of random walks in random sceneries

Relative complexity measures the complexity of a probability preserving transformation relative to a factor being a sequence of random variables whose exponential growth rate is the relative entropy of the extension. We prove distributional limit theorems for the relative complexity of certain zero entropy extensions: RWRSs whose associated random walks satisfy the \alpha-stable CLT ($1<\alpha\le2$). The results give invariants for relative isomorphism of these.Comment: Published in at http://dx.doi.org/10.1214/11-AOP688 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

A low complexity algorithm for non-monotonically evolving fronts

A new algorithm is proposed to describe the propagation of fronts advected in the normal direction with prescribed speed function F. The assumptions on F are that it does not depend on the front itself, but can depend on space and time. Moreover, it can vanish and change sign. To solve this problem the Level-Set Method [Osher, Sethian; 1988] is widely used, and the Generalized Fast Marching Method [Carlini et al.; 2008] has recently been introduced. The novelty of our method is that its overall computational complexity is predicted to be comparable to that of the Fast Marching Method [Sethian; 1996], [Vladimirsky; 2006] in most instances. This latter algorithm is O(N^n log N^n) if the computational domain comprises N^n points. Our strategy is to use it in regions where the speed is bounded away from zero -- and switch to a different formalism when F is approximately 0. To this end, a collection of so-called sideways partial differential equations is introduced. Their solutions locally describe the evolving front and depend on both space and time. The well-posedness of those equations, as well as their geometric properties are addressed. We then propose a convergent and stable discretization of those PDEs. Those alternative representations are used to augment the standard Fast Marching Method. The resulting algorithm is presented together with a thorough discussion of its features. The accuracy of the scheme is tested when F depends on both space and time. Each example yields an O(1/N) global truncation error. We conclude with a discussion of the advantages and limitations of our method.Comment: 30 pages, 12 figures, 1 tabl

Ninomiya-Victoir scheme: strong convergence, antithetic version and application to multilevel estimators

In this paper, we are interested in the strong convergence properties of the Ninomiya-Victoir scheme which is known to exhibit weak convergence with order 2. We prove strong convergence with order $1/2$. This study is aimed at analysing the use of this scheme either at each level or only at the finest level of a multilevel Monte Carlo estimator: indeed, the variance of a multilevel Monte Carlo estimator is related to the strong error between the two schemes used on the coarse and fine grids at each level. Recently, Giles and Szpruch proposed a scheme permitting to construct a multilevel Monte Carlo estimator achieving the optimal complexity $O\left(\epsilon^{-2}\right)$ for the precision $\epsilon$. In the same spirit, we propose a modified Ninomiya-Victoir scheme, which may be strongly coupled with order $1$ to the Giles-Szpruch scheme at the finest level of a multilevel Monte Carlo estimator. Numerical experiments show that this choice improves the efficiency, since the order $2$ of weak convergence of the Ninomiya-Victoir scheme permits to reduce the number of discretization levels