Some new results on sample path optimality in ergodic control of diffusions

We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s. sample path optimal stationary Markov control requires finite second moments of the first hitting times $\tau$ of bounded domains over all admissible controls. We show that this can be considerably weakened: ${\mathrm E}[\tau^2]$ may be replaced with ${\mathrm E}[\tau\ln^+(\tau)]$, thus reducing the required rate of convergence of averages from polynomial to logarithmic. A Foster-Lyapunov condition which guarantees this is also exhibited. Moreover, we study a large class of models that are neither uniformly stable, nor have a near-monotone running cost, and we exhibit sufficient conditions for the existence of a sample path optimal stationary Markov control.Comment: 10 page

A variational formula for risk-sensitive control of diffusions in Rd\mathbb{R}^d

We address the variational problem for the generalized principal eigenvalue on $\mathbb{R}^d$ of linear and semilinear elliptic operators associated with nondegenerate diffusions controlled through the drift. We establish the Collatz-Wielandt formula for potentials that vanish at infinity under minimal hypotheses, and also for general potentials under blanket geometric ergodicity assumptions. We also present associated results having the flavor of a refined maximum principle.Comment: 19 page

Infinite Horizon Average Optimality of the N-network Queueing Model in the Halfin-Whitt Regime

We study the infinite horizon optimal control problem for N-network queueing systems, which consist of two customer classes and two server pools, under average (ergodic) criteria in the Halfin-Whitt regime. We consider three control objectives: 1) minimizing the queueing (and idleness) cost, 2) minimizing the queueing cost while imposing a constraint on idleness at each server pool, and 3) minimizing the queueing cost while requiring fairness on idleness. The running costs can be any nonnegative convex functions having at most polynomial growth. For all three problems we establish asymptotic optimality, namely, the convergence of the value functions of the diffusion-scaled state process to the corresponding values of the controlled diffusion limit. We also present a simple state-dependent priority scheduling policy under which the diffusion-scaled state process is geometrically ergodic in the Halfin-Whitt regime, and some results on convergence of mean empirical measures which facilitate the proofs.Comment: 35 page

Infinite horizon asymptotic average optimality for large-scale parallel server networks

We study infinite-horizon asymptotic average optimality for parallel server network with multiple classes of jobs and multiple server pools in the Halfin-Whitt regime. Three control formulations are considered: 1) minimizing the queueing and idleness cost, 2) minimizing the queueing cost under a constraints on idleness at each server pool, and 3) fairly allocating the idle servers among different server pools. For the third problem, we consider a class of bounded-queue, bounded-state (BQBS) stable networks, in which any moment of the state is bounded by that of the queue only (for both the limiting diffusion and diffusion-scaled state processes). We show that the optimal values for the diffusion-scaled state processes converge to the corresponding values of the ergodic control problems for the limiting diffusion. We present a family of state-dependent Markov balanced saturation policies (BSPs) that stabilize the controlled diffusion-scaled state processes. It is shown that under these policies, the diffusion-scaled state process is exponentially ergodic, provided that at least one class of jobs has a positive abandonment rate. We also establish useful moment bounds, and study the ergodic properties of the diffusion-scaled state processes, which play a crucial role in proving the asymptotic optimality.Comment: 35 pages. arXiv admin note: text overlap with arXiv:1602.0327

Ergodic Diffusion Control of Multiclass Multi-Pool Networks in the Halfin-Whitt Regime

We consider Markovian multiclass multi-pool networks with heterogeneous server pools, each consisting of many statistically identical parallel servers, where the bipartite graph of customer classes and server pools forms a tree. Customers form their own queue and are served in the first-come first-served discipline, and can abandon while waiting in queue. Service rates are both class and pool dependent. The objective is to study the limiting diffusion control problems under the long run average (ergodic) cost criteria in the Halfin--Whitt regime. Two formulations of ergodic diffusion control problems are considered: (i) both queueing and idleness costs are minimized, and (ii) only the queueing cost is minimized while a constraint is imposed upon the idleness of all server pools. We develop a recursive leaf elimination algorithm that enables us to obtain an explicit representation of the drift for the controlled diffusions. Consequently, we show that for the limiting controlled diffusions, there always exists a stationary Markov control under which the diffusion process is geometrically ergodic. The framework developed in our earlier work is extended to address a broad class of ergodic diffusion control problems with constraints. We show that that the unconstrained and constrained problems are well posed, and we characterize the optimal stationary Markov controls via HJB equations.Comment: 32 page

A counterexample to a nonlinear version of the Krein-Rutman theorem by R. Mahadevan

In this short note we present a simple counterexample to a nonlinear version of the Krein-Rutman theorem reported in [Nonlinear Anal. 11 (2007), 3084-3090]. Correct versions of this theorem, and related results for superadditive maps are also presented.Comment: To appear in Nonlinear Analysis, 6 page

On a Class of Stochastic Differential Equations With Jumps and Its Properties

We study stochastic differential equations with jumps with no diffusion part. We provide some basic stochastic characterizations of solutions of the corresponding non-local partial differential equations and prove the Harnack inequality for a class of these operators. We also establish key connections between the recurrence properties of these jump processes and the non-local partial differential operator. One of the key results is the regularity of solutions of the Dirichlet problem for a class of operators with locally weakly H\"older continuous kernels.Comment: 40 page

On the Capacity of Multiple Access Channels with State Information and Feedback

In this paper, the multiple access channel (MAC) with channel state is analyzed in a scenario where a) the channel state is known non-causally to the transmitters and b) there is perfect causal feedback from the receiver to the transmitters. An achievable region and an outer bound are found for a discrete memoryless MAC that extend existing results, bringing together ideas from the two separate domains of MAC with state and MAC with feedback. Although this achievable region does not match the outer bound in general, special cases where they meet are identified. In the case of a Gaussian MAC, a specialized achievable region is found by using a combination of dirty paper coding and a generalization of the Schalkwijk-Kailath, Ozarow and Merhav-Weissman schemes, and this region is found to be capacity achieving. Specifically, it is shown that additive Gaussian interference that is known non-causally to the transmitter causes no loss in capacity for the Gaussian MAC with feedback.Comment: Prelimenary result appears in ISIT 200

Strict monotonicity of principal eigenvalues of elliptic operators in Rd\mathbb{R}^d and risk-sensitive control

This paper studies the eigenvalue problem on $\mathbb{R}^d$ for a class of second order, elliptic operators of the form $\mathscr{L} = a^{ij}\partial_{x_i}\partial_{x_j} + b^{i}\partial_{x_i} + f$, associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function $f$ fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton-Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.Comment: 45 page

Ergodicity of L\'evy-driven SDEs arising from multiclass many-server queues

We study the ergodic properties of a class of multidimensional piecewise Ornstein-Uhlenbeck processes with jumps, which contains the limit of the queueing processes arising in multiclass many-server queues with heavy-tailed arrivals and/or asymptotically negligible service interruptions in the Halfin-Whitt regime as special cases. In these queueing models, the It\^o equations have a piecewise linear drift, and are driven by either (1) a Brownian motion and a pure-jump L\'evy process, or (2) an anisotropic L\'evy process with independent one-dimensional symmetric $\alpha$-stable components, or (3) an anisotropic L\'evy process as in (2) and a pure-jump L\'evy process. We also study the class of models driven by a subordinate Brownian motion, which contains an isotropic (or rotationally invariant) $\alpha$-stable L\'evy process as a special case. We identify conditions on the parameters in the drift, the L\'evy measure and/or covariance function which result in subexponential and/or exponential ergodicity. We show that these assumptions are sharp, and we identify some key necessary conditions for the process to be ergodic. In addition, we show that for the queueing models described above with no abandonment, the rate of convergence is polynomial, and we provide a sharp quantitative characterization of the rate via matching upper and lower bounds.Comment: 42 page