Asymptotic Bias of Stochastic Gradient Search

The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and Lojasiewicz inequality), tight bounds on the asymptotic bias of the iterates generated by such an algorithm are derived. The obtained results hold under mild conditions and cover a broad class of high-dimensional nonlinear algorithms. Using these results, the asymptotic properties of the policy-gradient (reinforcement) learning and adaptive population Monte Carlo sampling are studied. Relying on the same results, the asymptotic behavior of the recursive maximum split-likelihood estimation in hidden Markov models is analyzed, too.Comment: arXiv admin note: text overlap with arXiv:0907.102

Analyticity of Entropy Rates of Continuous-State Hidden Markov Models

The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates is shown for analytically parameterized models. The obtained results hold under relatively mild conditions and cover several classes of hidden Markov models met in practice. These results are relevant for several (theoretically and practically) important problems arising in statistical inference, system identification and information theory

Bias of Particle Approximations to Optimal Filter Derivative

In many applications, a state-space model depends on a parameter which needs to be inferred from a data set. Quite often, it is necessary to perform the parameter inference online. In the maximum likelihood approach, this can be done using stochastic gradient search and the optimal filter derivative. However, the optimal filter and its derivative are not analytically tractable for a non-linear state-space model and need to be approximated numerically. In [Poyiadjis, Doucet and Singh, Biometrika 2011], a particle approximation to the optimal filter derivative has been proposed, while the corresponding $L_{p}$ error bonds and the central limit theorem have been provided in [Del Moral, Doucet and Singh, SIAM Journal on Control and Optimization 2015]. Here, the bias of this particle approximation is analyzed. We derive (relatively) tight bonds on the bias in terms of the number of particles. Under (strong) mixing conditions, the bounds are uniform in time and inversely proportional to the number of particles. The obtained results apply to a (relatively) broad class of state-space models met in practice

Про фулерени з перших вуст (від редакції)

Brian Doucet uses ten buildings in and around Detroit to tell the story of the rise, decline and prospects of the Motor City

Reweighting for Nonequilibrium Markov Processes Using Sequential Importance Sampling Methods

We present a generic reweighting method for nonequilibrium Markov processes. With nonequilibrium Monte Carlo simulations at a single temperature, one calculates the time evolution of physical quantities at different temperatures, which greatly saves the computational time. Using the dynamical finite-size scaling analysis for the nonequilibrium relaxation, one can study the dynamical properties of phase transitions together with the equilibrium ones. We demonstrate the procedure for the Ising model with the Metropolis algorithm, but the present formalism is general and can be applied to a variety of systems as well as with different Monte Carlo update schemes.Comment: accepted for publication in Phys. Rev. E (Rapid Communications

Stability of Optimal Filter Higher-Order Derivatives

In many scenarios, a state-space model depends on a parameter which needs to be inferred from data. Using stochastic gradient search and the optimal filter (first-order) derivative, the parameter can be estimated online. To analyze the asymptotic behavior of online methods for parameter estimation in non-linear state-space models, it is necessary to establish results on the existence and stability of the optimal filter higher-order derivatives. The existence and stability properties of these derivatives are studied here. We show that the optimal filter higher-order derivatives exist and forget initial conditions exponentially fast. We also show that the optimal filter higher-order derivatives are geometrically ergodic. The obtained results hold under (relatively) mild conditions and apply to state-space models met in practice

Migration and the Social Order in Erie County, New York: 1855

Mass transiency remains the most striking and consistent finding to emerge from quantitative studies of Victorian North America. In almost every place where historians have looked at least half, often two thirds, of the adults present at one end of a decade had left ten years later, and rates based on shorter periods reveal a stream of people constantly flowing through nineteenth-century cities. Although 363,000 people lived in Boston in 1880 and 448,000 in 1890, during the decade about one and one-half million people actually had dwelled within the city. When Victorians sought a symbol of progress, they often chose the steam engine; had they wanted a metaphor for their cities, they could have found none more apt than the railroad station. In this paper we confront the question of transiency. Using the New York State Census of 1855 for the entire city of Buffalo and a 10 percent sample of household heads in rural Erie County, we attempt a method of estimating persistence (the proportion of the population remaining in a given place) that is different from that used by most historians. Given the richness of the census, we are able to inquire with great detail into the factors that determined length of residence in a nineteenth-century city and its surrounding countryside