1,265 research outputs found
Bornes quantitatives pour la convergence en temps long de processus de Markov
National audienceSi l'on sait assez bien décrire qualitativement le comportement de nombreux processus de Markov (existence et unicité d'une mesure invariante, convergence exponentielle à l'équilibre, etc), il est en général beaucoup plus difficile d'obtenir des bornes explicites pour la vitesse de convergence à l'équilibre
Allocation conjointe de puissance et rendement d'un utilisateur cognitif exploitant les retransmissions d'un utilisateur primaire : le cas du canal en Z
National audienceDans cet article, nous considérons le problème de l’allocation conjointe de puissance et de rendement pour un utilisateur secondaire exploitant le protocole de retransmission d’un utilisateur primaire. Nous proposons un algorithme, basé sur les Processus de Markov Décisionnels (MDP), permettant de calculer une allocation optimale pour le problème de la maximisation du débit de l’utilisateur secondaire tout en garantissant un débit minimal pour l’utilisateur primaire
Hitting Times in Markov Chains with Restart and their Application to Network Centrality
Motivated by applications in telecommunications, computer scienceand physics,
we consider a discrete-time Markov process withrestart. At each step the
process eitherwith a positive probability restarts from a given distribution,
orwith the complementary probability continues according to a Markovtransition
kernel. The main contribution of the present work is thatwe obtain an explicit
expression for the expectation of the hittingtime (to a given target set) of
the process with restart.The formula is convenient when considering the problem
of optimizationof the expected hitting time with respect to the restart
probability.We illustrate our results with two examplesin uncountable and
countable state spaces andwith an application to network centrality
Absolutely Continuous Compensators
We give sufficient conditions on the underlying filtration such that all
totally inaccessible stopping times have compensators which are absolutely
continuous. If a semimartingale, strong Markov process X has a representation
as a solution of a stochastic differential equation driven by a Wiener process,
Lebesgue measure, and a Poisson random measure, then all compensators of
totally inaccessible stopping times are absolutely continuous with respect to
the minimal filtration generated by X. However Cinlar and Jacod have shown that
all semimartingale strong Markov processes, up to a change of time and space,
have such a representation
Arr\^et optimal pour les processus de Markov forts et les fonctions affines
In this Note we study optimal stopping problems for strong Markov processes
and affine functions. We give a justification of the Snell envelope form using
standard results of optimal stopping. We also justify the convexity of the
value function, and without a priori restriction to a particular class of
stopping times, we deduce that the smallest optimal stopping time is
necessarily a hitting time
- …