Finite-action approximation of Markov decision processes

Due to copyright restrictions, the access to the full text of this article is only available via subscription.In this chapter, we study the finite-action approximation of optimal control policies for discrete-time Markov decision processes (MDPs) with Borel state and action spaces, under discounted and average cost criteria. One main motivation for considering this problem stems from the optimal information transmission problem in networked control systems. In many applications of networked control, perfect transmission of the control actions to an actuator is infeasible when there is a communication channel of finite capacity between a controller and an actuator. Hence, the actions of the controller must be discretized (quantized) to facilitate reliable transmission. Although the problem of optimal information transmission from a plant/sensor to a controller has been studied extensively (see, e.g., [148] and references therein), much less is known about the problem of transmitting actions from a controller to an actuator. Such transmission schemes usually require a simple encoding/decoding rule since the actuator does not have the computational capability of the controller to use complex algorithms. For this reason, time-invariant scalar quantization is a practically useful encoding method for controller-actuator communication

Goodness-of-fit Tests Based on the Kernel Density Estimator

Given an i.i.d. sample drawn from a density "f" on the real line, the problem of testing whether "f" is in a given class of densities is considered. Testing procedures constructed on the basis of minimizing the "L" 1-distance between a kernel density estimate and any density in the hypothesized class are investigated. General non-asymptotic bounds are derived for the power of the test. It is shown that the concentration of the data-dependent smoothing factor and the 'size' of the hypothesized class of densities play a key role in the performance of the test. Consistency and non-asymptotic performance bounds are established in several special cases, including testing simple hypotheses, translation/scale classes and symmetry. Simulations are also carried out to compare the behaviour of the method with the Kolmogorov-Smirnov test and an "L" 2 density-based approach due to Fan ["Econ. Theory" 10 (1994) 316]. Copyright 2005 Board of the Foundation of the Scandinavian Journal of Statistics..