A review on analysis and synthesis of nonlinear stochastic systems with randomly occurring incomplete information

Copyright q 2012 Hongli Dong et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.In the context of systems and control, incomplete information refers to a dynamical system in which knowledge about the system states is limited due to the difficulties in modeling complexity in a quantitative way. The well-known types of incomplete information include parameter uncertainties and norm-bounded nonlinearities. Recently, in response to the development of network technologies, the phenomenon of randomly occurring incomplete information has become more and more prevalent. Such a phenomenon typically appears in a networked environment. Examples include, but are not limited to, randomly occurring uncertainties, randomly occurring nonlinearities, randomly occurring saturation, randomly missing measurements and randomly occurring quantization. Randomly occurring incomplete information, if not properly handled, would seriously deteriorate the performance of a control system. In this paper, we aim to survey some recent advances on the analysis and synthesis problems for nonlinear stochastic systems with randomly occurring incomplete information. The developments of the filtering, control and fault detection problems are systematically reviewed. Latest results on analysis and synthesis of nonlinear stochastic systems are discussed in great detail. In addition, various distributed filtering technologies over sensor networks are highlighted. Finally, some concluding remarks are given and some possible future research directions are pointed out. © 2012 Hongli Dong et al.This work was supported in part by the National Natural Science Foundation of China under Grants 61273156, 61134009, 61273201, 61021002, and 61004067, the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant GR/S27658/01, the Royal Society of the UK, the National Science Foundation of the USA under Grant No. HRD-1137732, and the Alexander von Humboldt Foundation of German

Automated Experiment Design for Data-Efficient Verification of Parametric Markov Decision Processes

We present a new method for statistical verification of quantitative properties over a partially unknown system with actions, utilising a parameterised model (in this work, a parametric Markov decision process) and data collected from experiments performed on the underlying system. We obtain the confidence that the underlying system satisfies a given property, and show that the method uses data efficiently and thus is robust to the amount of data available. These characteristics are achieved by firstly exploiting parameter synthesis to establish a feasible set of parameters for which the underlying system will satisfy the property; secondly, by actively synthesising experiments to increase amount of information in the collected data that is relevant to the property; and finally propagating this information over the model parameters, obtaining a confidence that reflects our belief whether or not the system parameters lie in the feasible set, thereby solving the verification problem.Comment: QEST 2017, 18 pages, 7 figure

Deception in Optimal Control

In this paper, we consider an adversarial scenario where one agent seeks to achieve an objective and its adversary seeks to learn the agent's intentions and prevent the agent from achieving its objective. The agent has an incentive to try to deceive the adversary about its intentions, while at the same time working to achieve its objective. The primary contribution of this paper is to introduce a mathematically rigorous framework for the notion of deception within the context of optimal control. The central notion introduced in the paper is that of a belief-induced reward: a reward dependent not only on the agent's state and action, but also adversary's beliefs. Design of an optimal deceptive strategy then becomes a question of optimal control design on the product of the agent's state space and the adversary's belief space. The proposed framework allows for deception to be defined in an arbitrary control system endowed with a reward function, as well as with additional specifications limiting the agent's control policy. In addition to defining deception, we discuss design of optimally deceptive strategies under uncertainties in agent's knowledge about the adversary's learning process. In the latter part of the paper, we focus on a setting where the agent's behavior is governed by a Markov decision process, and show that the design of optimally deceptive strategies under lack of knowledge about the adversary naturally reduces to previously discussed problems in control design on partially observable or uncertain Markov decision processes. Finally, we present two examples of deceptive strategies: a "cops and robbers" scenario and an example where an agent may use camouflage while moving. We show that optimally deceptive strategies in such examples follow the intuitive idea of how to deceive an adversary in the above settings

Resource Management with Stochastic Recharge and Environmental Threats

Exploitation diminishes the capacity of renewable resources to with-stand environmental stress, increasing their vulnerability to extreme conditions that may trigger abrupt changes. The onset of such events depends on the coincidence of extreme environmental conditions (environmental threat) and the resource state (determining its resilience). When the former is uncertain and the latter evolves stochastically, the uncertainty regarding the event occurrence is the result of the combined effect of these two uncertain components. We analyzed optimal resource management in such a setting. Existence of an optimal stationary policy is established and long run properties are characterized. A numerical illustration based on actual data is presented.Stochastic stock dynamics, event uncertainty, Markov decision process, optimal stationary policy, Environmental Economics and Policy,

On Markov Chains with Uncertain Data

In this paper, a general method is described to determine uncertainty intervals for performance measures of Markov chains given an uncertainty region for the parameters of the Markov chains. We investigate the effects of uncertainties in the transition probabilities on the limiting distributions, on the state probabilities after n steps, on mean sojourn times in transient states, and on absorption probabilities for absorbing states. We show that the uncertainty effects can be calculated by solving linear programming problems in the case of interval uncertainty for the transition probabilities, and by second order cone optimization in the case of ellipsoidal uncertainty. Many examples are given, especially Markovian queueing examples, to illustrate the theory.Markov chain;Interval uncertainty;Ellipsoidal uncertainty;Linear Programming;Second Order Cone Optimization

Trading Safety Versus Performance: Rapid Deployment of Robotic Swarms with Robust Performance Constraints

In this paper we consider a stochastic deployment problem, where a robotic swarm is tasked with the objective of positioning at least one robot at each of a set of pre-assigned targets while meeting a temporal deadline. Travel times and failure rates are stochastic but related, inasmuch as failure rates increase with speed. To maximize chances of success while meeting the deadline, a control strategy has therefore to balance safety and performance. Our approach is to cast the problem within the theory of constrained Markov Decision Processes, whereby we seek to compute policies that maximize the probability of successful deployment while ensuring that the expected duration of the task is bounded by a given deadline. To account for uncertainties in the problem parameters, we consider a robust formulation and we propose efficient solution algorithms, which are of independent interest. Numerical experiments confirming our theoretical results are presented and discussed