On the asymptotic behavior of some Algorithms

A simple approach is presented to study the asymptotic behavior of some algorithms with an underlying tree structure. It is shown that some asymptotic oscillating behaviors can be precisely analyzed without resorting to complex analysis techniques as it is usually done in this context. A new explicit representation of periodic functions involved is obtained at the same time.Comment: November 200

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

In this work, we consider an inhomogeneous (discrete time) Markov chain and are interested in its long time behavior. We provide sufficient conditions to ensure that some of its asymptotic properties can be related to the ones of a homogeneous (continuous time) Markov process. Renowned examples such as a bandit algorithms, weighted random walks or decreasing step Euler schemes are included in our framework. Our results are related to functional limit theorems, but the approach differs from the standard "Tightness/Identification" argument; our method is unified and based on the notion of pseudotrajectories on the space of probability measures

Graph Parameters, Universal Obstructions, and WQO

We introduce the notion of universal obstruction of a graph parameter, with respect to some quasi-ordering relation. Universal obstructions may serve as compact characterizations of the asymptotic behavior of graph parameters. We provide order-theoretic conditions which imply that such a characterization is finite and, when this is the case, we present some algorithmic implications on the existence of fixed-parameter algorithms

Regret bound for Narendra-Shapiro bandit algorithms

Narendra-Shapiro (NS) algorithms are bandit-type algorithms that were introduced in the 1960s in view of applications to Psychology or clinical trials. The long time behavior of such algorithms has been studied in depth but it seems that few results exist in a non-asymptotic setting, which can be of primary interest for applications. In this paper, we focus on the study of the regret of NS-algorithms and address the following question: are the Narendra-Shapiro (NS) bandit algorithms competitive from this non-asymptotic point of view? In our main result, we show that some competitive bounds can be obtained in their penalized version (introduced in [14]). More precisely, up to a slight modification, the regret of the penalized two-armed bandit algorithm is uniformly bounded by C \sqrt{n} (where C is a positive constant made explicit in the paper). We also generalize existing convergence and rate of convergence results to the multi-armed case of the over-penalized bandit algorithm, including the convergence toward the invariant measure of a Piecewise Deterministic Markov Process (PDMP) after a suitable renormalization. Finally, ergodic properties of this PDMP are given in the multi-armed case

Hessian and approximated Hessian matrices in maximum likelihood estimation: a Monte Carlo study

Full information maximum likelihood estimation of econometric models, linear and nonlinear in variables, is performed by means of two gradient algorithms, using either the Hessian matrix or a computationally simpler approximation. In the first part of the paper, the behavior of the two methods in getting the optimum is investigated with Monte Carlo experimentation on some models of small and medium size. In the second part of the paper, the behavior of the two matrices in producing estimates of the asymptotic covariance matrix of coefficients is analyzed and, again. experimented with Monte Carlo on the same models. Some systematic differences are evidenced.Hessian matrix, full information maximum likelihood, Newton like methods, gradient methods, covariance matrix estimators