Metrizable uniform spaces

Three themes of general topology: quotient spaces; absolute retracts; and inverse limits - are reapproached here in the setting of metrizable uniform spaces, with an eye to applications in geometric and algebraic topology. The results include: 1) If f: A -> Y is a uniformly continuous map, where X and Y are metric spaces and A is a closed subset of X, we show that the adjunction space X\cup_f Y with the quotient uniformity (hence also with the topology thereof) is metrizable, by an explicit metric. This yields natural constructions of cone, join and mapping cylinder in the category of metrizable uniform spaces, which we show to coincide with those based on subspace (of a normed linear space); on product (with a cone); and on the isotropy of the l_2 metric. 2) We revisit Isbell's theory of uniform ANRs, as refined by Garg and Nhu in the metrizable case. The iterated loop spaces \Omega^n P of a pointed compact polyhedron P are shown to be uniform ANRs. Four characterizations of uniform ANRs among metrizable uniform spaces X are given: (i) the completion of X is a uniform ANR, and the remainder is uniformly a Z-set in the completion; (ii) X is uniformly locally contractible and satisfies the Hahn approximation property; (iii) X is uniformly \epsilon-homotopy dominated by a uniform ANR for each \epsilon>0; (iv) X is an inverse limit of uniform ANRs with "nearly splitting" bonding maps.Comment: 93 pages. v5: a little bit of new stuff added. Proposition 8.7, entire section 10, Lemma 14.11, Proposition 18.4. Possibly something els

Entropic gradient flow structure of quantum Markov semigroups

Gegenstand der vorliegenden Arbeit ist die Konstruktion einer nichtkommutativen Transportmetrik, die es erlaubt, spursymmetrische vollständig Markovsche Halbgruppen als Gradientenfluss eines Entropiefunktionals aufzufassen. Eine vollständig Markovsche Halbgruppe ist eine Halbgruppe von unitalen, vollständig positiven Operatoren auf einer von Neumann algebra mit gewissen Stetigkeitseigenschaften. Ein Gradientenfluss eines Funktionals auf einem metrischen Raum ist eine Kurve, die zu jedem Zeitpunkt in die Richtung des steilsten Abstieges fließt. Es ist in einer Reihe von Fällen bekannt, dass man die Gradientenflüsse der Boltzmann-Entropie oder ihres nichtkommutativen Analogons, der von Neumann-Entropie, bezüglich geeigneter Transportmetriken als Lösungen von linearen Evolutionsgleichungen charakterisieren kann, zum Beispiel der Wärmeleitungsgleichung oder der Lindblad Master Equation. In dieser Arbeit wird gezeigt, dass das gemeinsame zugrundeliegende Prinzip in all diesen Fällen die Markoveigenschaft der linearen Evolutionsgleichung ist. Dazu wird für eine gegebene spursymmetrische vollständig Markovsche Halbgruppe eine Transportmetrik auf dem Raum der Dichteoperatoren konstruiert, die die Metriken in den oben genannten Fällen verallgemeinert. Es wird bewiesen, dass unter geeigneten Voraussetzungen die gegebene Halbgruppe der eindeutige Gradientenfluss der von Neumann-Entropie ist. Als Konsequenzen werden Semikonvexität der Entropie entlang von Geodäten und Funktionalungleichungenfür die Halbgruppe diskutiert

Understanding representation learning for deep reinforcement learning

Representation learning is essential to practical success of reinforcement learning. Through a state representation, an agent can describe its environment to efficiently explore the state space, generalize to new states and perform credit assignment from delayed feedback. These representations may be state abstractions, hand-engineered or fixed features or implied by a neural network. In this thesis, we investigate several desirable theoretical properties of state representations and, using this categorization, design novel principled RL algorithms aiming at learning these state representations at scale through deep learning. First, we consider state abstractions induced by behavioral metrics and their generalization properties. We show that supporting the continuity of the value function is central to generalization in reinforcement learning. Together with this formalization, we provide an empirical evaluation comparing various metrics and demonstrating the importance of the choice of a neighborhood in RL algorithms. Then, we draw on statistical learning theory to characterize what it means for arbitrary state features to generalize in RL. We introduce a new notion called effective dimension of a representation that drives the generalization to unseen states and demonstrate its usefulness for value-based deep reinforcement learning in Atari games. The third contribution of this dissertation is a scalable algorithm to learn a state representation from a very large number of auxiliary tasks through deep learning. It is a stochastic gradient descent method to learn the principal components of a target matrix by means of a neural network from a handful of entries. Finally, the last part presents our findings on how the state representation in reinforcement learning influences the quality of an agent’s predictions but is also shaped by these predictions. We provide a formal mathematical model for studying this phenomenon and show how these theoretical results can be leveraged to improve the quality of the learning process

Measures and all that --- A Tutorial

This tutorial gives an overview of some of the basic techniques of measure theory. It includes a study of Borel sets and their generators for Polish and for analytic spaces, the weak topology on the space of all finite positive measures including its metrics, as well as measurable selections. Integration is covered, and product measures are introduced, both for finite and for arbitrary factors, with an application to projective systems. Finally, the duals of the Lp-spaces are discussed, together with the Radon-Nikodym Theorem and the Riesz Representation Theorem. Case studies include applications to stochastic Kripke models, to bisimulations, and to quotients for transition kernels

State-similarity metrics for continuous Markov decision processes

In recent years, various metrics have been developed for measuring the similarity of states in probabilistic transition systems (Desharnais et al., 1999; van Breugel & Worrell, 2001a). In the context of Markov decision processes, we have devised metrics providing a robust quantitative analogue of bisimulation. Most importantly, the metric distances can be used to bound the differences in the optimal value function that is integral to reinforcement learning (Ferns et al. 2004; 2005). More recently, we have discovered an efficient algorithm to calculate distances in the case of finite systems (Ferns et al., 2006). In this thesis, we seek to properly extend state-similarity metrics to Markov decision processes with continuous state spaces both in theory and in practice. In particular, we provide the first distance-estimation scheme for metrics based on bisimulation for continuous probabilistic transition systems. Our work, based on statistical sampling and infinite dimensional linear programming, is a crucial first step in real-world planning; many practical problems are continuous in nature, e.g. robot navigation, and often a parametric model or crude finite approximation does not suffice. State-similarity metrics allow us to reason about the quality of replacing one model with another. In practice, they can be used directly to aggregate states