An MDP decomposition approach for traffic control at isolated signalized intersections

This article presents a novel approach for the dynamic control of a signalized intersection. At the intersection, there is a number of arrival flows of cars, each having a single queue (lane). The set of all flows is partitioned into disjoint combinations of nonconflicting flows that will receive green together. The dynamic control of the traffic lights is based on the numbers of cars waiting in the queues. The problem concerning when to switch (and which combination to serve next) is modeled as a Markovian decision process in discrete time. For large intersections (i.e., intersections with a large number of flows), the number of states becomes tremendously large, prohibiting straightforward optimization using value iteration or policy iteration. Starting from an optimal (or nearly optimal) fixed-cycle strategy, a one-step policy improvement is proposed that is easy to compute and is shown to give a close to optimal strategy for the dynamic proble

Transient handover blocking probabilities in road covering cellular mobile networks

This paper investigates handover and fresh call blocking probabilities for subscribers moving along a road in a traffic jam passing through consecutive cells of a wireless network. It is observed and theoretically motivated that the handover blocking probabilities show a sharp peak in the initial part of a traffic jam roughly at the moment when the traffic jam starts covering a new cell. The theoretical motivation relates handover blocking probabilities to blocking probabilities in the M/D/C/C queue with time-varying arrival rates. We provide a numerically efficient recursion for these blocking probabilities. \u

Markov games : an annotated bibliography

No abstract

The method of successive approximations for the discounted Markov game

This paper presents a number of successive approximation algorithms for the repeated two-person zero-sum game called Markov game using the criterion of total expected discounted rewards. As Wessels [12] did for Markov decision processes stopping times are introduced in order to simplify the proofs. It is shown that each algorithm provides upper and lower bounds for the value of the game and nearly optimal stationary strategies for both players

Successive approximations for the average Markov reward game : the communicating case

This paper considers the two-person zero-sum Markov game with finite state and action spaces at the criterion of average reward per unit time. For two types of Markov games, the communicating game and the simply connected game, it is shown that the method of successive approximations provides good bounds on the value of the game and nearly-optimal stationary strategies for the two players

Spin-Dependent Electron Transmission Model for Chiral Molecules in Mesoscopic Devices

Various device-based experiments have indicated that electron transfer in certain chiral molecules may be spin-dependent, a phenomenon known as the Chiral Induced Spin Selectivity (CISS) effect. However, due to the complexity of these devices and a lack of theoretical understanding, it is not always clear to what extent the chiral character of the molecules actually contributes to the magnetic-field-dependent signals in these experiments. To address this issue, we report here an electron transmission model that evaluates the role of the CISS effect in two-terminal and multi-terminal linear-regime electron transport experiments. Our model reveals that for the CISS effect, the chirality-dependent spin transmission is accompanied by a spin-flip electron reflection process. Furthermore, we show that more than two terminals are required in order to probe the CISS effect in the linear regime. In addition, we propose two types of multi-terminal nonlocal transport measurements that can distinguish the CISS effect from other magnetic-field-dependent signals. Our model provides an effective tool to review and design CISS-related transport experiments, and to enlighten the mechanism of the CISS effect itself

A cyclic production scheme for multi-item production systems with backlog : part 1

This paper is part 1 of two companion papers dealing with a multi-item production system in which the production is controlled by a fixed cycle scheme. The cycle consists of a production period with a fixed number of production times that can be used for production or idling, followed by a vacation. The duration of the vacation is independent of the production period. Demand arrives according to a (compound) Poisson process and is satisfied from stock or backlogged. The embedded process is modeled in discrete time and analyzed using generating functions. The optimal base stock level is derived from a newsvendor type relation. The model is extended to one with time slot dependent base stock levels. The results are used to construct a presumably optimal fixed cycle policy. In part 2 this fixed cycle policy is used to construct a dynamic production policy

Unified description of bulk and interface-enhanced spin pumping

The dynamics of non-equilibrium spin accumulation generated in metals or semiconductors by rf magnetic field pumping is treated within a diffusive picture. The dc spin accumulation produced in a uniform system by a rotating applied magnetic field or by a precessing magnetization of a weak ferromagnet is in general given by a (small) fraction of hbar omega, where omega is the rotation or precession frequency. With the addition of a neighboring, field-free region and allowing for the diffusion of spins, the spin accumulation is dramatically enhanced at the interface, saturating at the universal value hbar omega in the limit of long spin relaxation time. This effect can be maximized when the system dimensions are of the order of sqrt(2pi D omega), where D is the diffusion constant. We compare our results to the interface spin pumping theory of A. Brataas et al. [Phys. Rev. B 66, 060404(R) (2002)]

Circuit-Model Analysis for Spintronic Devices with Chiral Molecules as Spin Injectors

Recent research discovered that charge transfer processes in chiral molecules can be spin selective and named the effect chiral-induced spin selectivity (CISS). Follow-up work studied hybrid spintronic devices with conventional electronic materials and chiral (bio)molecules. However, a theoretical foundation for the CISS effect is still in development and the spintronic signals were not evaluated quantitatively. We present a circuit-model approach that can provide quantitative evaluations. Our analysis assumes the scheme of a recent experiment that used photosystem~I (PSI) as spin injectors, for which we find that the experimentally observed signals are, under any reasonable assumptions on relevant PSI time scales, too high to be fully due to the CISS effect. We also show that the CISS effect can in principle be detected using the same type of solid-state device, and by replacing silver with graphene, the signals due to spin generation can be enlarged four orders of magnitude. Our approach thus provides a generic framework for analyzing this type of experiments and advancing the understanding of the CISS effect