22 research outputs found
Handling polymorphic algebraic effects
Algebraic effects and handlers are a powerful abstraction mechanism to
represent and implement control effects. In this work, we study their extension
with parametric polymorphism that allows abstracting not only expressions but
also effects and handlers. Although polymorphism makes it possible to reuse and
reason about effect implementations more effectively, it has long been known
that a naive combination of polymorphic effects and let-polymorphism breaks
type safety. Although type safety can often be gained by restricting let-bound
expressions---e.g., by adopting value restriction or weak polymorphism---we
propose a complementary approach that restricts handlers instead of let-bound
expressions. Our key observation is that, informally speaking, a handler is
safe if resumptions from the handler do not interfere with each other. To
formalize our idea, we define a call-by-value lambda calculus that supports
let-polymorphism and polymorphic algebraic effects and handlers, design a type
system that rejects interfering handlers, and prove type safety of our
calculus.Comment: Added the errata for the ESOP'19 paper (page 28
Denotational validation of higher-order Bayesian inference
We present a modular semantic account of Bayesian inference algorithms for
probabilistic programming languages, as used in data science and machine
learning. Sophisticated inference algorithms are often explained in terms of
composition of smaller parts. However, neither their theoretical justification
nor their implementation reflects this modularity. We show how to conceptualise
and analyse such inference algorithms as manipulating intermediate
representations of probabilistic programs using higher-order functions and
inductive types, and their denotational semantics. Semantic accounts of
continuous distributions use measurable spaces. However, our use of
higher-order functions presents a substantial technical difficulty: it is
impossible to define a measurable space structure over the collection of
measurable functions between arbitrary measurable spaces that is compatible
with standard operations on those functions, such as function application. We
overcome this difficulty using quasi-Borel spaces, a recently proposed
mathematical structure that supports both function spaces and continuous
distributions. We define a class of semantic structures for representing
probabilistic programs, and semantic validity criteria for transformations of
these representations in terms of distribution preservation. We develop a
collection of building blocks for composing representations. We use these
building blocks to validate common inference algorithms such as Sequential
Monte Carlo and Markov Chain Monte Carlo. To emphasize the connection between
the semantic manipulation and its traditional measure theoretic origins, we use
Kock's synthetic measure theory. We demonstrate its usefulness by proving a
quasi-Borel counterpart to the Metropolis-Hastings-Green theorem
Functional Programming for Modular Bayesian Inference
We present an architectural design of a library for Bayesian modelling and inference in modern functional programming languages. The novel aspect of our approach are modular implementations of existing state-of- the-art inference algorithms. Our design relies on three inherently functional features: higher-order functions, inductive data-types, and support for either type-classes or an expressive module system. We provide a perfor- mant Haskell implementation of this architecture, demonstrating that high-level and modular probabilistic programming can be added as a library in sufficiently expressive languages. We review the core abstractions in this architecture: inference representations, inference transformations, and inference representation transformers. We then implement concrete instances of these abstractions, counterparts to particle filters and Metropolis-Hastings samplers, which form the basic building blocks of our library. By composing these building blocks we obtain state-of-the-art inference algorithms: Resample-Move Sequential Monte Carlo, Particle Marginal Metropolis-Hastings, and Sequential Monte Carlo Squared. We evaluate our implementation against existing probabilistic programming systems and find it is already com- petitively performant, although we conjecture that existing functional programming optimisation techniques could reduce the overhead associated with the abstractions we use. We show that our modular design enables deterministic testing of inherently stochastic Monte Carlo algorithms. Finally, we demonstrate using OCaml that an expressive module system can also implement our design
A monad for full ground reference cells
We present a denotational account of dynamic allocation of potentially cyclic memory cells using a monad on a functor category. We identify the collection of heaps as an object in a different functor category equipped with a monad for adding hiding/encapsulation capabilities to the heaps. We derive a monad for full ground references supporting effect masking by applying a state monad transformer to the encapsulation monad. To evaluate the monad, we present a denotational semantics for a call-by-value calculus with full ground references, and validate associated code transformations
Fully abstract models for effectful λ-calculi via category-theoretic logical relations
We present a construction which, under suitable assumptions, takes a model of Moggi’s computational λ-calculus with sum types, effect operations and primitives, and yields a model that is adequate and fully abstract. The construction, which uses the theory of fibrations, categorical glueing, ⊤⊤-lifting, and ⊤⊤-closure, takes inspiration from O’Hearn & Riecke’s fully abstract model for PCF. Our construction can be applied in the category of sets and functions, as well as the category of diffeological spaces and smooth maps and the category of quasi-Borel spaces, which have been studied as semantics for differentiable and probabilistic programming
A monad for full ground reference cells
We present a denotational account of dynamic allocation of potentially cyclic memory cells using a monad on a functor category. We identify the collection of heaps as an object in a different functor category equipped with a monad for adding hiding/encapsulation capabilities to the heaps. We derive a monad for full ground references supporting effect masking by applying a state monad transformer to the encapsulation monad. To evaluate the monad, we present a denotational semantics for a call-by-value calculus with full ground references, and validate associated code transformations
A monad for full ground reference cells
We present a denotational account of dynamic allocation of potentially cyclic memory cells using a monad on a functor category. We identify the collection of heaps as an object in a different functor category equipped with a monad for adding hiding/encapsulation capabilities to the heaps. We derive a monad for full ground references supporting effect masking by applying a state monad transformer to the encapsulation monad. To evaluate the monad, we present a denotational semantics for a call-by-value calculus with full ground references, and validate associated code transformations
Bayesian Inversion by ω-complete cone duality
The process of inverting Markov kernels relates to the important subject of Bayesian modelling and learning. In fact, Bayesian update is exactly kernel inversion. In this paper, we investigate how and when Markov kernels (aka stochastic relations, or probabilistic mappings, or simply kernels) can be inverted. We address the question both directly on the category of measurable spaces, and indirectly by interpreting kernels as Markov operators: For the direct option, we introduce a typed version of the category of Markov kernels and use the so-called 'disintegration of measures'. Here, one has to specialise to measurable spaces borne from a simple class of topological spaces -e.g. Polish spaces (other choices are possible). Our method and result greatly simplify a recent development in Ref. [4]. For the operator option, we use a cone version of the category of Markov operators (kernels seen as predicate transformers). That is to say, our linear operators are not just continuous, but are required to satisfy the stronger condition of being ω-chain-continuous. 1 Prior work shows that one obtains an adjunction in the form of a pair of contravariant and inverse functors between the categories of L 1 - and L ∞ -cones [3]. Inversion, seen through the operator prism, is just adjunction.2 No topological assumption is needed. We show that both categories (Markov kernels and ω-chain-continuous Markov operators) are related by a family of contravariant functors T p for 1 ≤ p ≤∞. The T p 's are Kleisli extensions of (duals of) conditional expectation functors introduced in Ref. [3]. With this bridge in place, we can prove that both notions of inversion agree when both defined: if f is a kernel, and f †its direct inverse, then T ∞ (f) †= T ∞ (f †)
A monad for full ground reference cells
We present a denotational account of dynamic allocation of potentially cyclic memory cells using a monad on a functor category. We identify the collection of heaps as an object in a different functor category equipped with a monad for adding hiding/encapsulation capabilities to the heaps. We derive a monad for full ground references supporting effect masking by applying a state monad transformer to the encapsulation monad. To evaluate the monad, we present a denotational semantics for a call-by-value calculus with full ground references, and validate associated code transformations