Existence of optimal controls for singular control problems with state constraints

We establish the existence of an optimal control for a general class of singular control problems with state constraints. The proof uses weak convergence arguments and a time rescaling technique. The existence of optimal controls for Brownian control problems \citehar, associated with a broad family of stochastic networks, follows as a consequence.Comment: Published at http://dx.doi.org/10.1214/105051606000000556 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Singular control with state constraints on unbounded domain

We study a class of stochastic control problems where a cost of the form \begin{equation}\mathbb{E}\int_{[0,\infty)}e^{-\beta s}[\ell(X_s) ds+h(Y^{\circ}_s) d|Y|_s]\end{equation} is to be minimized over control processes $Y$ whose increments take values in a cone $\mathbb{Y}$ of $\mathbb{R}^p$, keeping the state process $X=x+B+GY$ in a cone $\mathbb{X}$ of $\mathbb{R}^k$, $k\le p$. Here, $x\in\mathbb{X}$, $B$ is a Brownian motion with drift $b$ and covariance $\Sigma$, $G$ is a fixed matrix, and $Y^{\circ}$ is the Radon--Nikodym derivative $dY/d|Y|$. Let $\mathcal{L}=-(1/2)trace(\Sigma D^2)-b\cdot D$ where $D$ denotes the gradient. Solutions to the corresponding dynamic programming PDE, \begin{equation}[(\mathcal{L}+\beta)f-\ell]\vee\sup_{y\in\mathbb{Y}:|Gy|=1 }[-Gy\cdot Df-h(y)]=0,\end{equation} on $\mathbb{X}^o$ are considered with a polynomial growth condition and are required to be supersolution up to the boundary (corresponding to a ``state constraint'' boundary condition on $\partial\mathbb{X}$). Under suitable conditions on the problem data, including continuity and nonnegativity of $\ell$ and $h$, and polynomial growth of $\ell$, our main result is the unique viscosity-sense solvability of the PDE by the control problem's value function in appropriate classes of functions. In some cases where uniqueness generally fails to hold in the class of functions that grow at most polynomially (e.g., when $h=0$), our methods provide uniqueness within the class of functions that, in addition, have compact level sets. The results are new even in the following special cases: (1) The one-dimensional case $k=p=1$, $\mathbb{X}=\mathbb{Y}=\mathbb{R}_+$; (2) The first-order case $\Sigma=0$; (3) The case where $\ell$ and $h$ are linear. The proofs combine probabilistic arguments and viscosity solution methods. Our framework covers a wide range of diffusion control problems that arise from queueing networks in heavy traffic.Comment: Published at http://dx.doi.org/10.1214/009117906000000359 in the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

Some Fluctuation Results for Weakly Interacting Multi-type Particle System

A collection of $N$-diffusing interacting particles where each particle belongs to one of $K$ different populations is considered. Evolution equation for a particle from population $k$ depends on the $K$ empirical measures of particle states corresponding to the various populations and the form of this dependence may change from one population to another. In addition, the drift coefficients in the particle evolution equations may depend on a factor that is common to all particles and which is described through the solution of a stochastic differential equation coupled, through the empirical measures, with the $N$-particle dynamics. We are interested in the asymptotic behavior as $N\to \infty$. Although the full system is not exchangeable, particles in the same population have an exchangeable distribution. Using this structure, one can prove using standard techniques a law of large numbers result and a propagation of chaos property. In the current work we study fluctuations about the law of large number limit. For the case where the common factor is absent the limit is given in terms of a Gaussian field whereas in the presence of a common factor it is characterized through a mixture of Gaussian distributions. We also obtain, as a corollary, new fluctuation results for disjoint sub-families of single type particle systems, i.e. when $K=1$. Finally, we establish limit theorems for multi-type statistics of such weakly interacting particles, given in terms of multiple Wiener integrals.Comment: 47 page

Large deviations for multidimensional state-dependent shot noise processes

Shot noise processes are used in applied probability to model a variety of physical systems in, for example, teletraffic theory, insurance and risk theory and in the engineering sciences. In this work we prove a large deviation principle for the sample-paths of a general class of multidimensional state-dependent Poisson shot noise processes. The result covers previously known large deviation results for one dimensional state-independent shot noise processes with light tails. We use the weak convergence approach to large deviations, which reduces the proof to establishing the appropriate convergence of certain controlled versions of the original processes together with relevant results on existence and uniqueness

Near critical catalyst reactant branching processes with controlled immigration

Near critical catalyst-reactant branching processes with controlled immigration are studied. The reactant population evolves according to a branching process whose branching rate is proportional to the total mass of the catalyst. The bulk catalyst evolution is that of a classical continuous time branching process; in addition there is a specific form of immigration. Immigration takes place exactly when the catalyst population falls below a certain threshold, in which case the population is instantaneously replenished to the threshold. Such models are motivated by problems in chemical kinetics where one wants to keep the level of a catalyst above a certain threshold in order to maintain a desired level of reaction activity. A diffusion limit theorem for the scaled processes is presented, in which the catalyst limit is described through a reflected diffusion, while the reactant limit is a diffusion with coefficients that are functions of both the reactant and the catalyst. Stochastic averaging principles under fast catalyst dynamics are established. In the case where the catalyst evolves "much faster" than the reactant, a scaling limit, in which the reactant is described through a one dimensional SDE with coefficients depending on the invariant distribution of the reflected diffusion, is obtained. Proofs rely on constrained martingale problem characterizations, Lyapunov function constructions, moment estimates that are uniform in time and the scaling parameter and occupation measure techniques.Comment: Published in at http://dx.doi.org/10.1214/12-AAP894 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic

In this work we study the problem of asymptotically optimal control of a well-known multi-class queuing network, referred to as the ``crisscross network,'' in heavy traffic. We consider exponential inter-arrival and service times, linear holding cost and an infinite horizon discounted cost criterion. In a suitable parameter regime, this problem has been studied in detail by Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133-2171] using viscosity solution methods. In this work, using the pathwise solution of the Brownian control problem, we present an elementary and transparent treatment of the problem (with the identical parameter regime as in [SIAM J. Control Optim. 34 (1996) 2133-2171]) using large deviation ideas introduced in [Ann. Appl. Probab. 10 (2000) 75-103, Ann. Appl. Probab. 11 (2001) 608-649]. We obtain an asymptotically optimal scheduling policy which is of threshold type. The proof is of independent interest since it is one of the few results which gives the asymptotic optimality of a control policy for a network with a more than one-dimensional workload process.Comment: Published at http://dx.doi.org/10.1214/105051605000000250 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org