120 research outputs found
Player aggregation in the traveling inspector model
We consider a model of dynamic inspection/surveillance of
a number of facilities in different geographical locations. The inspector in
this process travels from one facility to another and performs an
inspection at each facility he visits. His aim is to devise an inspection/
travel schedule which minimizes the losses to society (or to his employer)
resulting both from undetected violations of the regulations and from the
costs of the policing operation. This model is formulated as a non-cooperative,
single-controller, stochastic game. The existence of stationary Nash
equilibria is established as a consequence of aggregating all the inspectees
into a single “aggregated inspectee”. It is shown that such player
aggregation causes no loss of generality under very mild assumptions. A
notion of an “optimal Nash equilibrium” for the inspector is introduced
and proven to be well-defined in this context. The issue of the inspector’s
power to “enforce” such an equilibrium is also discussed
Hamiltonian cycles and subsets of discounted occupational measures
We study a certain polytope arising from embedding the Hamiltonian cycle
problem in a discounted Markov decision process. The Hamiltonian cycle problem
can be reduced to finding particular extreme points of a certain polytope
associated with the input graph. This polytope is a subset of the space of
discounted occupational measures. We characterize the feasible bases of the
polytope for a general input graph , and determine the expected numbers of
different types of feasible bases when the underlying graph is random. We
utilize these results to demonstrate that augmenting certain additional
constraints to reduce the polyhedral domain can eliminate a large number of
feasible bases that do not correspond to Hamiltonian cycles. Finally, we
develop a random walk algorithm on the feasible bases of the reduced polytope
and present some numerical results. We conclude with a conjecture on the
feasible bases of the reduced polytope.Comment: revised based on referees comment
On production and abatement time scales in sustainable development. Can we loosen the sustainability screw ?
In this paper we carry out a preliminary exploration of a time scales' conjecture, which postulates that "reasonable" notions of sustainability must include a suitable synchronisation of time scales of both the processes of human development and those of the natural environment. We perform our analysis within a coarse, ?ve variable, model of man-nature interactions expressed as a system of differential equations where production and human capital are coupled with both renewable and non-renewable natural resource. We demonstrate a phenomenon that we name the "sustainability screw" that describes a spiral like trajectory of the three key variables: non-renewable and renewable resources as well as the production capital. Under many plausible scenarios, this spiral tends unacceptably fast to an undesirable equilibrium. However, we also show that by adjusting the ratio of "intensity of production effort" and "intensity of abatement effort", parameters of the relative time scales of production and natural recovery processes can be altered in a manner that produces, arguably, more sustainable trajectories.sustainable optimization systems, viability, multiple time scale
Weighted Markov Decision Processes with perturbation
In this paper we consider the weighted reward MDP’s
with perturbation. We give the proof of existence of a
delta-optimal simple ultimately deterministic policy under
the assumption of “scalar value”. We also prove
that there exists a delta-i-optimal simple ultimately deterministic
policy in the perturbed weighted MDP, for
all e E [0, e*) even without the assumption of “scalar
value”
Perturbation theory for semi-Markov control problems
In earlier work, the authors considered the perturbation of systems undergoing Markov processes in which the times between two consecutive decision time points were equidistant. They now consider perturbations of processes for which the times between transition are random variables. These are called semi-Markov processes
The embedding of the traveling salesman problem in a Markov Decision Process
In this paper we derive a new LP-relaxation of the Traveling
Salesman Problem (TSP, for short). This formulation
comes from first embedding the TSP in a Markov Decision
Process (MDP: for short), and from perturbing this MDP
appropriately
Perturbation and stability theory for Markov control problems
A unified approach to the asymptotic analysis of a Markov decision process disturbed by an ε-additive perturbation is proposed. Irrespective of whether the perturbation is regular or singular, the underlying control problem that needs to be understood is the limit Markov control problem. The properties of this problem are the subject of this study
- …