Harris recurrence of Metropolis-within-Gibbs and trans-dimensional Markov chains

A $\phi$-irreducible and aperiodic Markov chain with stationary probability distribution will converge to its stationary distribution from almost all starting points. The property of Harris recurrence allows us to replace ``almost all'' by ``all,'' which is potentially important when running Markov chain Monte Carlo algorithms. Full-dimensional Metropolis--Hastings algorithms are known to be Harris recurrent. In this paper, we consider conditions under which Metropolis-within-Gibbs and trans-dimensional Markov chains are or are not Harris recurrent. We present a simple but natural two-dimensional counter-example showing how Harris recurrence can fail, and also a variety of positive results which guarantee Harris recurrence. We also present some open problems. We close with a discussion of the practical implications for MCMC algorithms.Comment: Published at http://dx.doi.org/10.1214/105051606000000510 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Positive Harris recurrence and exponential ergodicity of the basic affine jump-diffusion

In this paper we find the transition densities of the basic affine jump-diffusion (BAJD), which is introduced by Duffie and Garleanu [D. Duffie and N. Garleanu, Risk and valuation of collateralized debt obligations, Financial Analysts Journal 57(1) (2001), pp. 41--59] as an extension of the CIR model with jumps. We prove the positive Harris recurrence and exponential ergodicity of the BAJD. Furthermore we prove that the unique invariant probability measure $\pi$ of the BAJD is absolutely continuous with respect to the Lebesgue measure and we also derive a closed form formula for the density function of $\pi$.Comment: 21 pape

Affine Jump-Diffusions: Stochastic Stability and Limit Theorems

Affine jump-diffusions constitute a large class of continuous-time stochastic models that are particularly popular in finance and economics due to their analytical tractability. Methods for parameter estimation for such processes require ergodicity in order establish consistency and asymptotic normality of the associated estimators. In this paper, we develop stochastic stability conditions for affine jump-diffusions, thereby providing the needed large-sample theoretical support for estimating such processes. We establish ergodicity for such models by imposing a `strong mean reversion' condition and a mild condition on the distribution of the jumps, i.e. the finiteness of a logarithmic moment. Exponential ergodicity holds if the jumps have a finite moment of a positive order. In addition, we prove strong laws of large numbers and functional central limit theorems for additive functionals for this class of models

On some ergodicity properties for time inhomogeneous Markov processes with TT-periodic semigroup

We consider a time inhomogeneous strong Markov process $(\xi_t)_{t\ge 0}$ taking values in a Polish state space whose semigroup has a $T$-periodic structure. We give simple conditions which imply ergodicity of the grid chain $(\xi_{kT})_{k\in \mathbb{N}_0}$ In case of $d$-dimensional possibly degenerate diffusions, the conditions are stated in terms of drift and diffusion coefficient of the process

Ergodicity and limit theorems for degenerate diffusions with time periodic drift. Applications to a stochastic Hodgkin-Huxley model

We formulate simple criteria for positive Harris recurrence of strongly degenerate stochastic differential equations with smooth coefficients when the drift depends on time and space and is periodic in the time argument. There is no time dependence in the diffusion coefficient. Our criteria rely on control systems and the support theorem, existence of an attainable inner point of full weak Hoermander dimension and of some Lyapunov function. Positive Harris recurrence enables us to prove limit theorems for such diffusions. As an application, we consider a stochastic Hodgkin-Huxley model for a spiking neuron including its dendritic input. The latter carries some deterministic periodic signal coded in its drift coefficient and is the only source of noise for the whole system. This amounts to a 5d SDE driven by 1d Brownian motion for which we can prove positive Harris recurrence using our criteria. This approach provides us with laws of large numbers which allow to describe the spiking activity of the neuron in the long run

Ergodicity for a stochastic Hodgkin-Huxley model driven by Ornstein-Uhlenbeck type input

We consider a model describing a neuron and the input it receives from its dendritic tree when this input is a random perturbation of a periodic deterministic signal, driven by an Ornstein-Uhlenbeck process. The neuron itself is modeled by a variant of the classical Hodgkin-Huxley model. Using the existence of an accessible point where the weak Hoermander condition holds and the fact that the coefficients of the system are analytic, we show that the system is non-degenerate. The existence of a Lyapunov function allows to deduce the existence of (at most a finite number of) extremal invariant measures for the process. As a consequence, the complexity of the system is drastically reduced in comparison with the deterministic system.Comment: arXiv admin note: text overlap with arXiv:1207.019

Long-time behavior of stable-like processes

In this paper, we consider a long-time behavior of stable-like processes. A stable-like process is a Feller process given by the symbol $p(x,\xi)=-i\beta(x)\xi+\gamma(x)|\xi|^{\alpha(x)},$ where $\alpha(x)\in(0,2)$, $\beta(x)\in\R$ and $\gamma(x)\in(0,\infty)$. More precisely, we give sufficient conditions for recurrence, transience and ergodicity of stable-like processes in terms of the stability function $\alpha(x)$, the drift function $\beta(x)$ and the scaling function $\gamma(x)$. Further, as a special case of these results we give a new proof for the recurrence and transience property of one-dimensional symmetric stable L\'{e}vy processes with the index of stability $\alpha\neq1.$Comment: To appear in: Stochastic Processes and their Application

MaxWeight Scheduling: Asymptotic Behavior of Unscaled Queue-Differentials in Heavy Traffic

The model is a "generalized switch", serving multiple traffic flows in discrete time. The switch uses MaxWeight algorithm to make a service decision (scheduling choice) at each time step, which determines the probability distribution of the amount of service that will be provided. We are primarily motivated by the following question: in the heavy traffic regime, when the switch load approaches critical level, will the service processes provided to each flow remain "smooth" (i.e., without large gaps in service)? Addressing this question reduces to the analysis of the asymptotic behavior of the unscaled queue-differential process in heavy traffic. We prove that the stationary regime of this process converges to that of a positive recurrent Markov chain, whose structure we explicitly describe. This in turn implies asymptotic "smoothness" of the service processes.Comment: 9 page

Validity of heavy-traffic steady-state approximations in many-server queues with abandonment

We consider GI/Ph/n+M parallel-server systems with a renewal arrival process, a phase-type service time distribution, n homogenous servers, and an exponential patience time distribution with positive rate. We show that in the Halfin-Whitt regime, the sequence of stationary distributions corresponding to the normalized state processes is tight. As a consequence, we establish an interchange of heavy traffic and steady state limits for GI/Ph/n+M queues

Linear Convergence on Positively Homogeneous Functions of a Comparison Based Step-Size Adaptive Randomized Search: the (1+1) ES with Generalized One-fifth Success Rule

In the context of unconstraint numerical optimization, this paper investigates the global linear convergence of a simple probabilistic derivative-free optimization algorithm (DFO). The algorithm samples a candidate solution from a standard multivariate normal distribution scaled by a step-size and centered in the current solution. This solution is accepted if it has a better objective function value than the current one. Crucial to the algorithm is the adaptation of the step-size that is done in order to maintain a certain probability of success. The algorithm, already proposed in the 60's, is a generalization of the well-known Rechenberg's $(1+1)$ Evolution Strategy (ES) with one-fifth success rule which was also proposed by Devroye under the name compound random search or by Schumer and Steiglitz under the name step-size adaptive random search. In addition to be derivative-free, the algorithm is function-value-free: it exploits the objective function only through comparisons. It belongs to the class of comparison-based step-size adaptive randomized search (CB-SARS). For the convergence analysis, we follow the methodology developed in a companion paper for investigating linear convergence of CB-SARS: by exploiting invariance properties of the algorithm, we turn the study of global linear convergence on scaling-invariant functions into the study of the stability of an underlying normalized Markov chain (MC). We hence prove global linear convergence by studying the stability (irreducibility, recurrence, positivity, geometric ergodicity) of the normalized MC associated to the $(1+1)$-ES. More precisely, we prove that starting from any initial solution and any step-size, linear convergence with probability one and in expectation occurs. Our proof holds on unimodal functions that are the composite of strictly increasing functions by positively homogeneous functions with degree $\alpha$ (assumed also to be continuously differentiable). This function class includes composite of norm functions but also non-quasi convex functions. Because of the composition by a strictly increasing function, it includes non continuous functions. We find that a sufficient condition for global linear convergence is the step-size increase on linear functions, a condition typically satisfied for standard parameter choices. While introduced more than 40 years ago, we provide here the first proof of global linear convergence for the $(1+1)$-ES with generalized one-fifth success rule and the first proof of linear convergence for a CB-SARS on such a class of functions that includes non-quasi convex and non-continuous functions. Our proof also holds on functions where linear convergence of some CB-SARS was previously proven, namely convex-quadratic functions (including the well-know sphere function)