On the relation of nonanticipative rate distortion function and filtering theory

In this paper the relation between nonanticipative rate distortion function (RDF) and Bayesian filtering theory is investigated using the topology of weak convergence of probability measures on Polish spaces. The relation is established via an optimization on the space of conditional distributions of the so-called directed information subject to fidelity constraints. Existence of the optimal reproduction distribution of the nonanticipative RDF is shown, while the optimal nonanticipative reproduction conditional distribution for stationary processes is derived in closed form. The realization procedure of nonanticipative RDF which is equivalent to joint-source channel matching for symbol-by-symbol transmission is described, while an example is introduced to illustrate the concepts.Comment: 6 pages, 4 figures, final version submitted for publication at 12th Biannual European Control Conference (ECC), 201

Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss-Markov Processes

In this paper, we develop {finite-time horizon} causal filters using the nonanticipative rate distortion theory. We apply the {developed} theory to {design optimal filters for} time-varying multidimensional Gauss-Markov processes, subject to a mean square error fidelity constraint. We show that such filters are equivalent to the design of an optimal \texttt{\{encoder, channel, decoder\}}, which ensures that the error satisfies {a} fidelity constraint. Moreover, we derive a universal lower bound on the mean square error of any estimator of time-varying multidimensional Gauss-Markov processes in terms of conditional mutual information. Unlike classical Kalman filters, the filter developed is characterized by a reverse-waterfilling algorithm, which ensures {that} the fidelity constraint is satisfied. The theoretical results are demonstrated via illustrative examples.Comment: 35 pages, 6 figures, submitted for publication in SIAM Journal on Control and Optimization (SICON

An integrative perspective to LQ and ℓ∞ control for delayed and quantized systems

Deterministic and stochastic approaches to handle uncertainties may incur very different complexities in computation time and memory usage, in addition to different uncertainty models. For linear systems with delay and rate constrained communications between the observer and the controller, previous work shows that a deterministic approach, the ℓ ∞ control has low complexity but can only handle bounded disturbances. In this article, we take a stochastic approach and propose a linear-quadratic (LQ) controller that can handle arbitrarily large disturbance but has large complexity in time and space. The differences in robustness and complexity of the ℓ ∞ and LQ controllers motivate the design of a hybrid controller that interpolates between the two: The ℓ ∞ controller is applied when the disturbance is not too large (normal mode) and the LQ controller is resorted to otherwise (acute mode). We characterize the switching behavior between the normal and acute modes. Using our theoretical bounds which are supplemented by numerical experiments, we show that the hybrid controller can achieve a sweet spot in the robustness-complexity tradeoff, i.e., reject occasional large disturbance while operating with low complexity most of the time

Kodierung von Gaußmaßen

Es sei $gamma$ ein Gaußmaß auf der Borelschen $sigma$-Algebra $mathcal B$ des separablen Banachraums $B$. Für $X:Omega o B$ gelte $P_X=gamma$. Wir untersuchen den mittleren Fehler, der bei Kodierung von $gamma$ respektive $X$ mit $Ninmathbb N$ Punkten entsteht, und bestimmen untere und obere Abschätzungen für die Asymptotik ($N oinfty$) dieses Fehlers. Hierbei betrachten wir zu $r>0$ Gütekriterien wie folgt: Deterministische Kodierung $delta_2(N,r) := inf_{y_1,ldots,y_Nin B}Emin_{k=1,ldots,N}X-y_k^r.$ Zufällige Kodierung $delta_3(N,r) := inf_ u Emin_{k=1,ldots,N}X-Y_k^r.$ Die $(Y_k)$ seien hierbei i.i.d., unabhängig von $X$, und nach $u$ verteilt. Das Infimum wird über alle Wahrscheinlichkeitsmaße $u$ gebildet. Für das Gütekriterium $delta_4(cdot,r)$ wird ausgehend von der Definition von $delta_3(cdot,r)$ $u$ nicht optimal, sondern $u=gamma$ gewählt. Das Gütekriterium $delta_1(cdot,r)$ ergibt sich aus der Quellkodierungstheorie nach Shannon. Es gilt $delta_1(cdot,r) le delta_2(cdot,r) le delta_3(cdot,r) le delta_4(cdot,r).$ Wir stellen folgenden Zusammenhang zwischen der Asymptotik von $delta_4(cdot,r)$ und den logarithmischen kleinen Abweichungen von $gamma$ her: Es gebe $kappa,a>0$ und $binR$ mit psi(varepsilon) := -log P{X1.Let $gamma$ be a Gaussian measure on the Borel $sigma$-algebra $mathcal B$ of the separable Banach space $B$. Let $X:Omega o B$ with $P_X=gamma$. We investigate the average error in coding $gamma$ resp. $X$ with $Ninmathbb N$ points and obtain lower and upper bounds for the error asymptotics ($N oinfty$). We consider, given $r>0$, fidelity criterions as follows: Deterministic Coding $delta_2(N,r) := inf_{y_1,ldots,y_Nin B}Emin_{k=1,ldots,N}X-y_k^r.$ Random Coding $delta_3(N,r) := inf_ u Emin_{k=1,ldots,N}X-Y_k^r.$ The $(Y_k)$ above are i.i.d., independent of $X$, and distributed according to $u$. The infimum is taken with respect to all probability measures $u$. For the fidelity criterion $delta_4(cdot,r)$, starting from the definition of $delta_3(cdot,r)$, $u$ is not chosen optimal, but as $u=gamma$. The fidelity criterion $delta_1(cdot,r)$ is given according to the source coding theory of Shannon. The fidelity criterions are connected through $delta_1(cdot,r) le delta_2(cdot,r) le delta_3(cdot,r) le delta_4(cdot,r).$ We obtain the following connection between the asymptotics of $delta_4(cdot,r)$ and the den logarithmic small deviations of $gamma$: Let $kappa,a>0$ and $binR$ with psi(varepsilon) := -log P{X1