3,437 research outputs found
An MDP decomposition approach for traffic control at isolated signalized intersections
This article presents a novel approach for the dynamic control of a signalized intersection. At the intersection, there is a number of arrival flows of cars, each having a single queue (lane). The set of all flows is partitioned into disjoint combinations of nonconflicting flows that will receive green together. The dynamic control of the traffic lights is based on the numbers of cars waiting in the queues. The problem concerning when to switch (and which combination to serve next) is modeled as a Markovian decision process in discrete time. For large intersections (i.e., intersections with a large number of flows), the number of states becomes tremendously large, prohibiting straightforward optimization using value iteration or policy iteration. Starting from an optimal (or nearly optimal) fixed-cycle strategy, a one-step policy improvement is proposed that is easy to compute and is shown to give a close to optimal strategy for the dynamic proble
Transient handover blocking probabilities in road covering cellular mobile networks
This paper investigates handover and fresh call blocking probabilities for subscribers moving along a road in a traffic jam passing through consecutive cells of a wireless network. It is observed and theoretically motivated that the handover blocking probabilities show a sharp peak in the initial part of a traffic jam roughly at the moment when the traffic jam starts covering a new cell. The theoretical motivation relates handover blocking probabilities to blocking probabilities in the M/D/C/C queue with time-varying arrival rates. We provide a numerically efficient recursion for these blocking probabilities. \u
Successive approximations for the average Markov reward game : the communicating case
This paper considers the two-person zero-sum Markov game with finite state and action spaces at the criterion of average reward per unit time. For two types of Markov games, the communicating game and the simply connected game, it is shown that the method of successive approximations provides good bounds on the value of the game and nearly-optimal stationary strategies for the two players
The method of successive approximations for the discounted Markov game
This paper presents a number of successive approximation algorithms for the repeated two-person zero-sum game called Markov game using the criterion of total expected discounted rewards. As Wessels [12] did for Markov decision processes stopping times are introduced in order to simplify the proofs. It is shown that each algorithm provides upper and lower bounds for the value of the game and nearly optimal stationary strategies for both players
Spin-Dependent Electron Transmission Model for Chiral Molecules in Mesoscopic Devices
Various device-based experiments have indicated that electron transfer in
certain chiral molecules may be spin-dependent, a phenomenon known as the
Chiral Induced Spin Selectivity (CISS) effect. However, due to the complexity
of these devices and a lack of theoretical understanding, it is not always
clear to what extent the chiral character of the molecules actually contributes
to the magnetic-field-dependent signals in these experiments. To address this
issue, we report here an electron transmission model that evaluates the role of
the CISS effect in two-terminal and multi-terminal linear-regime electron
transport experiments. Our model reveals that for the CISS effect, the
chirality-dependent spin transmission is accompanied by a spin-flip electron
reflection process. Furthermore, we show that more than two terminals are
required in order to probe the CISS effect in the linear regime. In addition,
we propose two types of multi-terminal nonlocal transport measurements that can
distinguish the CISS effect from other magnetic-field-dependent signals. Our
model provides an effective tool to review and design CISS-related transport
experiments, and to enlighten the mechanism of the CISS effect itself
A cyclic production scheme for multi-item production systems with backlog : part 1
This paper is part 1 of two companion papers dealing with a multi-item production system in which the production is controlled by a fixed cycle scheme. The cycle consists of a production period with a fixed number of production times that can be used for production or idling, followed by a vacation. The duration of the vacation is independent of the production period. Demand arrives according to a (compound) Poisson process and is satisfied from stock or backlogged. The embedded process is modeled in discrete time and analyzed using generating functions. The optimal base stock level is derived from a newsvendor type relation. The model is extended to one with time slot dependent base stock levels. The results are used to construct a presumably optimal fixed cycle policy. In part 2 this fixed cycle policy is used to construct a dynamic production policy
On Markov games
In the paper it is demonstrated, how a dynamic programming approach may be useful for the analysis of Markov games. Markov games with finitely many stages are dealt with extensively. The existence of optimal Markov strategies is proven for finite stage Markov games using a shortcut of a proof by Derman for the analogous result for Markov decision processes. For Markov games with a countably infinite number of stages some results are summarized. Here again the results and the methods of prove have much in common with results and proofs for Markov decision processes. Actually the theory of Markov games is a generalisation. The paper contains short introductions into the theories of matrix games and tree games
Unified description of bulk and interface-enhanced spin pumping
The dynamics of non-equilibrium spin accumulation generated in metals or
semiconductors by rf magnetic field pumping is treated within a diffusive
picture. The dc spin accumulation produced in a uniform system by a rotating
applied magnetic field or by a precessing magnetization of a weak ferromagnet
is in general given by a (small) fraction of hbar omega, where omega is the
rotation or precession frequency. With the addition of a neighboring,
field-free region and allowing for the diffusion of spins, the spin
accumulation is dramatically enhanced at the interface, saturating at the
universal value hbar omega in the limit of long spin relaxation time. This
effect can be maximized when the system dimensions are of the order of sqrt(2pi
D omega), where D is the diffusion constant. We compare our results to the
interface spin pumping theory of A. Brataas et al. [Phys. Rev. B 66, 060404(R)
(2002)]
- ā¦