71 research outputs found
Modelling conditional probabilities with Riemann-Theta Boltzmann Machines
The probability density function for the visible sector of a Riemann-Theta
Boltzmann machine can be taken conditional on a subset of the visible units. We
derive that the corresponding conditional density function is given by a
reparameterization of the Riemann-Theta Boltzmann machine modelling the
original probability density function. Therefore the conditional densities can
be directly inferred from the Riemann-Theta Boltzmann machine.Comment: 7 pages, 3 figures, in proceedings of the 19th International Workshop
on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2019
Riemann-Theta Boltzmann Machine
A general Boltzmann machine with continuous visible and discrete integer
valued hidden states is introduced. Under mild assumptions about the connection
matrices, the probability density function of the visible units can be solved
for analytically, yielding a novel parametric density function involving a
ratio of Riemann-Theta functions. The conditional expectation of a hidden state
for given visible states can also be calculated analytically, yielding a
derivative of the logarithmic Riemann-Theta function. The conditional
expectation can be used as activation function in a feedforward neural network,
thereby increasing the modelling capacity of the network. Both the Boltzmann
machine and the derived feedforward neural network can be successfully trained
via standard gradient- and non-gradient-based optimization techniques.Comment: 29 pages, 11 figures, final version published in Neurocomputin
Reason, causation and compatibility with the phenomena
'Reason, Causation and Compatibility with the Phenomena' strives to give answers to the philosophical problem of the interplay between realism, explanation and experience. This book is a compilation of essays that recollect significant conceptions of rival terms such as determinism and freedom, reason and appearance, power and knowledge. This title discusses the progress made in epistemology and natural philosophy, especially the steps that led from the ancient theory of atomism to the modern quantum theory, and from mathematization to analytic philosophy. Moreover, it provides possible gateways from modern deadlocks of theory either through approaches to consciousness or through historical critique of intellectual authorities.
This work will be of interest to those either researching or studying in colleges and universities, especially in the departments of philosophy, history of science, philosophy of science, philosophy of physics and quantum mechanics, history of ideas and culture. Greek and Latin Literature students and instructors may also find this book to be both a fascinating and valuable point of reference
Algorithms and architectures for MCMC acceleration in FPGAs
Markov Chain Monte Carlo (MCMC) is a family of stochastic algorithms which are used to draw random samples from arbitrary probability distributions. This task is necessary to solve a variety of problems in Bayesian modelling, e.g. prediction and model comparison, making MCMC a fundamental tool in modern statistics. Nevertheless, due to the increasing complexity of Bayesian models, the explosion in the amount of data they need to handle and the computational intensity of many MCMC algorithms, performing MCMC-based inference is often impractical in real applications. This thesis tackles this computational problem by proposing Field Programmable Gate Array (FPGA) architectures for accelerating MCMC and by designing novel MCMC algorithms and optimization methodologies which are tailored for FPGA implementation. The contributions of this work include: 1) An FPGA architecture for the Population-based MCMC algorithm, along with two modified versions of the algorithm which use custom arithmetic precision in large parts of the implementation without introducing error in the output. Mapping the two modified versions to an FPGA allows for more parallel modules to be instantiated in the same chip area. 2) An FPGA architecture for the Particle MCMC algorithm, along with a novel algorithm which combines Particle MCMC and Population-based MCMC to tackle multi-modal distributions. A proposed FPGA architecture for the new algorithm achieves higher datapath utilization than the Particle MCMC architecture. 3) A generic method to optimize the arithmetic precision of any MCMC algorithm that is implemented on FPGAs. The method selects the minimum precision among a given set of precisions, while guaranteeing a user-defined bound on the output error. By applying the above techniques to large-scale Bayesian problems, it is shown that significant speedups (one or two orders of magnitude) are possible compared to state-of-the-art MCMC algorithms implemented on CPUs and GPUs, opening the way for handling complex statistical analyses in the era of ubiquitous, ever-increasing data.Open Acces
Applications of Monte Carlo Methods in Biology, Medicine and Other Fields of Science
This volume is an eclectic mix of applications of Monte Carlo methods in many fields of research should not be surprising, because of the ubiquitous use of these methods in many fields of human endeavor. In an attempt to focus attention on a manageable set of applications, the main thrust of this book is to emphasize applications of Monte Carlo simulation methods in biology and medicine
Recommended from our members
Exploring Probability Measures with Markov Processes
In many domains where mathematical modelling is applied, a deterministic description of the system at hand is insufficient, and so it is useful to model systems as being in some way stochastic. This is often achieved by modeling the state of the system as being drawn from a probability measure, which is usually given algebraically, i.e. as a formula. While this representation can be useful for deriving certain characteristics of the system, it is by now well-appreciated that many questions about stochastic systems are best-answered by looking at samples from the associated probability measure. In this thesis, we seek to develop and analyse efficient techniques for generating samples from a given probability measure, with a focus on algorithms which simulate a Markov process with the desired invariant measure.
The first work presented in this thesis considers the use of Piecewise-Deterministic Markov Processes (PDMPs) for generating samples. In contrast to usual approaches, PDMPs are i) defined as continuous-time processes, and ii) are typically non-reversible with respect to their invariant measure. These distinctions pose computational and theoretical challenges for the design, analysis, and implementation of PDMP-based samplers. The key contribution of this work is to develop a transparent characterisation of how one can construct a PDMP (within the class of trajectorially-reversible processes) which admits the desired invariant measure, and to offer actionable recommendations on how these processes should be designed in practice.
The second work presented in this thesis considers the task of sampling from a probability measure on a discrete space. While work in recent years has made it possible to apply sampling algorithms to probability measures with differentiable densities on continuous spaces in a reasonably generic way, samplers on discrete spaces are still largely derived on a case-by-case basis. The contention of this work is that this is not necessary, and that one can in fact define quite generally-applicable algorithms which can sample efficiently from discrete probability measures. The contributions are then to propose a small collection of algorithms for this task, and verify their efficiency empirically. Building
on the previous chapter’s work, our samplers are again defined in continuous time and non-reversible, each of which offer noticeable benefits in efficiency.
The third work presented in this thesis concerns a theoretical study of a particular class of Markov Chain-based sampling algorithms which make use of parallel computing resources. The Markov Chains which are produced by this algorithm are mathematically equivalent to a standard Metropolis-Hastings chain, but their real-time convergence properties are affected nontrivially by the application of parallelism. The contribution of this work is to analyse the convergence behaviour of these chains, and to use the ‘optimal scaling’ framework (as developed by Roberts, Rosenthal, and others) to make recommendations concerning the tuning of such algorithms in practice.
The introductory chapters provide a general overview on the task of generating samples from a probability measure, with particular focus on methods involving Markov processes. There is also an interlude on the relative benefits of i) continuous-time and ii) non-reversible Markov processes for sampling, which are intended to provide additional context for the reading of the first two works.PhD Studentship paid for by Cantab Capital Institute for the Mathematics of Informatio
- …