389 research outputs found
Discrete-time controlled markov processes with average cost criterion: a survey
This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation
Whittle Index Policy for Crawling Ephemeral Content
We consider a task of scheduling a crawler to retrieve content from several
sites with ephemeral content. A user typically loses interest in ephemeral
content, like news or posts at social network groups, after several days or
hours. Thus, development of timely crawling policy for such ephemeral
information sources is very important. We first formulate this problem as an
optimal control problem with average reward. The reward can be measured in the
number of clicks or relevant search requests. The problem in its initial
formulation suffers from the curse of dimensionality and quickly becomes
intractable even with moderate number of information sources. Fortunately, this
problem admits a Whittle index, which leads to problem decomposition and to a
very simple and efficient crawling policy. We derive the Whittle index and
provide its theoretical justification
Average optimality for continuous-time Markov decision processes under weak continuity conditions
This article considers the average optimality for a continuous-time Markov
decision process with Borel state and action spaces and an arbitrarily
unbounded nonnegative cost rate. The existence of a deterministic stationary
optimal policy is proved under a different and general set of conditions as
compared to the previous literature; the controlled process can be explosive,
the transition rates can be arbitrarily unbounded and are weakly continuous,
the multifunction defining the admissible action spaces can be neither
compact-valued nor upper semi-continuous, and the cost rate is not necessarily
inf-compact
Optimal sequential vector quantization of Markov sources
Includes bibliographical references (p. 30-31).Supported by U.S. Army grant. PAAL03-92-G-0115 Supported by a Homi Bhabha Fellowship and the Center for Intelligent Control Systems.V.S. Borkar, Sanjoy K. Mitter, Sekhar Tatikonda
- …