The Separation Principle in Stochastic Control, Redux

Over the last 50 years a steady stream of accounts have been written on the separation principle of stochastic control. Even in the context of the linear-quadratic regulator in continuous time with Gaussian white noise, subtle difficulties arise, unexpected by many, that are often overlooked. In this paper we propose a new framework for establishing the separation principle. This approach takes the viewpoint that stochastic systems are well-defined maps between sample paths rather than stochastic processes per se and allows us to extend the separation principle to systems driven by martingales with possible jumps. While the approach is more in line with "real-life" engineering thinking where signals travel around the feedback loop, it is unconventional from a probabilistic point of view in that control laws for which the feedback equations are satisfied almost surely, and not deterministically for every sample path, are excluded.Comment: 23 pages, 6 figures, 2nd revision: added references, correction

Optimal Policy with Partial Information in a Forward-Looking Model: Certainty-Equivalence Redux

This paper proves a certainty equivalence result for optimal policy under commitment with symmetric partial information about the state of the economy in a model with forward-looking variables. This result is used in our previous paper, Indicator Variables for Optimal Policy,' which synthesizes what is known about the case of symmetric partial information, and derives useful general formulas for computation of the optimal policy response coefficients and efficient estimates of the state of the economy in the context of a fairly general forward-looking rational-expectations model. In particular, our proof takes into account that, under commitment, the policymaker can affect the future evolution of the observable variables, and thereby potentially affect the future information available.

Steering state statistics with output feedback

Consider a linear stochastic system whose initial state is a random vector with a specified Gaussian distribution. Such a distribution may represent a collection of particles abiding by the specified system dynamics. In recent publications, we have shown that, provided the system is controllable, it is always possible to steer the state covariance to any specified terminal Gaussian distribution using state feedback. The purpose of the present work is to show that, in the case where only partial state observation is available, a necessary and sufficient condition for being able to steer the system to a specified terminal Gaussian distribution for the state vector is that the terminal state covariance be greater (in the positive-definite sense) than the error covariance of a corresponding Kalman filter.Comment: 10 pages, 2 figure

Optimal control of the state statistics for a linear stochastic system

We consider a variant of the classical linear quadratic Gaussian regulator (LQG) in which penalties on the endpoint state are replaced by the specification of the terminal state distribution. The resulting theory considerably differs from LQG as well as from formulations that bound the probability of violating state constraints. We develop results for optimal state-feedback control in the two cases where i) steering of the state distribution is to take place over a finite window of time with minimum energy, and ii) the goal is to maintain the state at a stationary distribution over an infinite horizon with minimum power. For both problems the distribution of noise and state are Gaussian. In the first case, we show that provided the system is controllable, the state can be steered to any terminal Gaussian distribution over any specified finite time-interval. In the second case, we characterize explicitly the covariance of admissible stationary state distributions that can be maintained with constant state-feedback control. The conditions for optimality are expressed in terms of a system of dynamically coupled Riccati equations in the finite horizon case and in terms of algebraic conditions for the stationary case. In the case where the noise and control share identical input channels, the Riccati equations for finite-horizon steering become homogeneous and can be solved in closed form. The present paper is largely based on our recent work in arxiv.org/abs/1408.2222, arxiv.org/abs/1410.3447 and presents an overview of certain key results.Comment: 7 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:1410.344

The modularity of action and perception revisited using control theory and active inference

The assumption that action and perception can be investigated independently is entrenched in theories, models and experimental approaches across the brain and mind sciences. In cognitive science, this has been a central point of contention between computationalist and 4Es (enactive, embodied, extended and embedded) theories of cognition, with the former embracing the “classical sandwich”, modular, architecture of the mind and the latter actively denying this separation can be made. In this work we suggest that the modular independence of action and perception strongly resonates with the separation principle of control theory and furthermore that this principle provides formal criteria within which to evaluate the implications of the modularity of action and perception. We will also see that real-time feedback with the environment, often considered necessary for the definition of 4Es ideas, is not however a sufficient condition to avoid the “classical sandwich”. Finally, we argue that an emerging framework in the cognitive and brain sciences, active inference, extends ideas derived from control theory to the study of biological systems while disposing of the separation principle, describing non-modular models of behaviour strongly aligned with 4Es theories of cognition

COINSTAC: A Privacy Enabled Model and Prototype for Leveraging and Processing Decentralized Brain Imaging Data

The field of neuroimaging has embraced the need for sharing and collaboration. Data sharing mandates from public funding agencies and major journal publishers have spurred the development of data repositories and neuroinformatics consortia. However, efficient and effective data sharing still faces several hurdles. For example, open data sharing is on the rise but is not suitable for sensitive data that are not easily shared, such as genetics. Current approaches can be cumbersome (such as negotiating multiple data sharing agreements). There are also significant data transfer, organization and computational challenges. Centralized repositories only partially address the issues. We propose a dynamic, decentralized platform for large scale analyses called the Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC). The COINSTAC solution can include data missing from central repositories, allows pooling of both open and ``closed'' repositories by developing privacy-preserving versions of widely-used algorithms, and incorporates the tools within an easy-to-use platform enabling distributed computation. We present an initial prototype system which we demonstrate on two multi-site data sets, without aggregating the data. In addition, by iterating across sites, the COINSTAC model enables meta-analytic solutions to converge to ``pooled-data'' solutions (i.e. as if the entire data were in hand). More advanced approaches such as feature generation, matrix factorization models, and preprocessing can be incorporated into such a model. In sum, COINSTAC enables access to the many currently unavailable data sets, a user friendly privacy enabled interface for decentralized analysis, and a powerful solution that complements existing data sharing solutions

Strong experimental guarantees in ultrafast quantum random number generation

We describe a methodology and standard of proof for experimental claims of quantum random number generation (QRNG), analogous to well-established methods from precision measurement. For appropriately constructed physical implementations, lower bounds on the quantum contribution to the average min-entropy can be derived from measurements on the QRNG output. Given these bounds, randomness extractors allow generation of nearly perfect "{\epsilon}-random" bit streams. An analysis of experimental uncertainties then gives experimentally derived confidence levels on the {\epsilon} randomness of these sequences. We demonstrate the methodology by application to phase-diffusion QRNG, driven by spontaneous emission as a trusted randomness source. All other factors, including classical phase noise, amplitude fluctuations, digitization errors and correlations due to finite detection bandwidth, are treated with paranoid caution, i.e., assuming the worst possible behaviors consistent with observations. A data-constrained numerical optimization of the distribution of untrusted parameters is used to lower bound the average min-entropy. Under this paranoid analysis, the QRNG remains efficient, generating at least 2.3 quantum random bits per symbol with 8-bit digitization and at least 0.83 quantum random bits per symbol with binary digitization, at a confidence level of 0.99993. The result demonstrates ultrafast QRNG with strong experimental guarantees.Comment: 11 pages, 9 figure