818 research outputs found
Liesel: A Probabilistic Programming Framework for Developing Semi-Parametric Regression Models and Custom Bayesian Inference Algorithms
Liesel is a probabilistic programming framework focusing on but not limited
to semi-parametric regression. It comprises a graph-based model building
library, a Markov chain Monte Carlo (MCMC) library with support for modular
inference algorithms combining multiple kernels (both implemented in Python),
and an R interface (RLiesel) for the configuration of semi-parametric
regression models. Each component can be used independently of the others, e.g.
the MCMC library also works with third-party model implementations. Our goal
with Liesel is to facilitate a new research workflow in computational
statistics: In a first step, the researcher develops a model graph with
pre-implemented and well-tested building blocks as a base model, e.g. using
RLiesel. Then, the graph can be manipulated to incorporate new research ideas,
before the MCMC library can be used to run and analyze a default or
user-defined MCMC procedure. The researcher has the option to combine powerful
MCMC algorithms such as the No U-Turn Sampler (NUTS) with self-written kernels.
Various tools for chain post-processing and diagnostics are also provided.
Considering all its components, Liesel enables efficient and reliable
statistical research on complex models and estimation algorithms. It depends on
JAX as a numerical computing library. This way, it can benefit from the latest
machine learning technology such as automatic differentiation, just-in-time
(JIT) compilation, and the use of high-performance computing devices such as
tensor processing units (TPUs)
Cumulative Distribution Functions As The Foundation For Probabilistic Models
This thesis discusses applications of probabilistic and connectionist models for
constructing and training cumulative distribution functions (CDFs). First, it is shown
how existing tools from the copula literature can be combined to build probabilistic
models. It is found that this simple construction leads to numerical and scalability
issues that make training and inference challenging.
Next, several innovative ideas, combining neural networks, automatic differentiation
and copula functions, introduce how to assemble black-box probabilistic
models. The basic building block is a cumulative distribution function that is straightforward
to construct, composed of arithmetic operations and nonlinear functions.
There is no need to assume any specific parametric probability density function
(PDF), making the model flexible and normalisation unnecessary. The only requirement
is to design a computational graph that parameterises monotonically
non-decreasing functions with a constrained range. Training can be then performed
using standard tools from any neural network software library.
Finally, factorial hidden Markov models (FHMMs) for sequential data are
presented. It is shown how to leverage cumulative distribution functions in the
form of the Gaussian copula and amortised stochastic variational method to encode
hidden Markov chains coherently. This approach enables efficient learning and
inference to model long sequences of high-dimensional data with long-range dependencies.
Tackling such complex problems was impossible with the established
FHMM approximate inference algorithm.
It is empirically verified on several problems that some of the estimators introduced
in this work can perform comparably or better than the currently popular
models. Especially for tasks requiring tail-area or marginal probabilities that can be
read directly from a cumulative distribution function
Introduction to Linear Algebra: Models, Methods, and Theory
This book develops linear algebra around matrices. Vector spaces in the abstract are not considered, only vector spaces associated with matrices. This book puts problem solving and an intuitive treatment of theory first, with a proof-oriented approach intended to come in a second course, the same way that calculus is taught. The book\u27s organization is straightforward: Chapter 1 has introductory linear models; Chapter 2 has the basics of matrix algebra; Chapter 3 develops different ways to solve a system of equations; Chapter 4 has applications, and Chapter 5 has vector-space theory associated with matrices and related topics such as pseudoinverses and orthogonalization. Many linear algebra textbooks start immediately with Gaussian elimination, before any matrix algebra. Here we first pose problems in Chapter 1, then develop a mathematical language for representing and recasting the problems in Chapter 2, and then look at ways to solve the problems in Chapter 3-four different solution methods are presented with an analysis of strengths and weaknesses of each.https://commons.library.stonybrook.edu/ams-books/1000/thumbnail.jp
- …