654 research outputs found
LIPIcs, Volume 251, ITCS 2023, Complete Volume
LIPIcs, Volume 251, ITCS 2023, Complete Volum
A Practitioner's Guide to Bayesian Inference in Pharmacometrics using Pumas
This paper provides a comprehensive tutorial for Bayesian practitioners in
pharmacometrics using Pumas workflows. We start by giving a brief motivation of
Bayesian inference for pharmacometrics highlighting limitations in existing
software that Pumas addresses. We then follow by a description of all the steps
of a standard Bayesian workflow for pharmacometrics using code snippets and
examples. This includes: model definition, prior selection, sampling from the
posterior, prior and posterior simulations and predictions, counter-factual
simulations and predictions, convergence diagnostics, visual predictive checks,
and finally model comparison with cross-validation. Finally, the background and
intuition behind many advanced concepts in Bayesian statistics are explained in
simple language. This includes many important ideas and precautions that users
need to keep in mind when performing Bayesian analysis. Many of the algorithms,
codes, and ideas presented in this paper are highly applicable to clinical
research and statistical learning at large but we chose to focus our
discussions on pharmacometrics in this paper to have a narrower scope in mind
and given the nature of Pumas as a software primarily for pharmacometricians
Pairwise versus mutual independence: visualisation, actuarial applications and central limit theorems
Accurately capturing the dependence between risks, if it exists, is an increasingly relevant topic of actuarial research. In recent years, several authors have started to relax the traditional 'independence assumption', in a variety of actuarial settings. While it is known that 'mutual independence' between random variables is not equivalent to their 'pairwise independence', this thesis aims to provide a better understanding of the materiality of this difference. The distinction between mutual and pairwise independence matters because, in practice, dependence is often assessed via pairs only, e.g., through correlation matrices, rank-based measures of association, scatterplot matrices, heat-maps, etc. Using such pairwise methods, it is possible to miss some forms of dependence. In this thesis, we explore how material the difference between pairwise and mutual independence is, and from several angles.
We provide relevant background and motivation for this thesis in Chapter 1, then conduct a literature review in Chapter 2.
In Chapter 3, we focus on visualising the difference between pairwise and mutual independence. To do so, we propose a series of theoretical examples (some of them new) where random variables are pairwise independent but (mutually) dependent, in short, PIBD. We then develop new visualisation tools and use them to illustrate what PIBD variables can look like. We showcase that the dependence involved is possibly very strong. We also use our visualisation tools to identify subtle forms of dependence, which would otherwise be hard to detect.
In Chapter 4, we review common dependence models (such has elliptical distributions and Archimedean copulas) used in actuarial science and show that they do not allow for the possibility of PIBD data. We also investigate concrete consequences of the 'nonequivalence' between pairwise and mutual independence. We establish that many results which hold for mutually independent variables do not hold under sole pairwise independent. Those include results about finite sums of random variables, extreme value theory and bootstrap methods. This part thus illustrates what can potentially 'go wrong' if one assumes mutual independence where only pairwise independence holds.
Lastly, in Chapters 5 and 6, we investigate the question of what happens for PIBD variables 'in the limit', i.e., when the sample size goes to infi nity. We want to see if the 'problems' caused by dependence vanish for sufficiently large samples. This is a broad question, and we concentrate on the important classical Central Limit Theorem (CLT), for which we fi nd that the answer is largely negative. In particular, we construct new sequences of PIBD variables (with arbitrary margins) for which a CLT does not hold. We derive explicitly the asymptotic distribution of the standardised mean of our sequences, which allows us to illustrate the extent of the 'failure' of a CLT for PIBD variables. We also propose a general methodology to construct dependent K-tuplewise independent (K an arbitrary integer) sequences of random variables with arbitrary margins. In the case K = 3, we use this methodology to derive explicit examples of triplewise independent sequences for which no CLT hold. Those results illustrate that mutual independence is a crucial assumption within CLTs, and that having larger samples is not always a viable solution to the problem of non-independent data
Reconstruction and Synthesis of Human-Scene Interaction
In this thesis, we argue that the 3D scene is vital for understanding, reconstructing, and synthesizing human motion. We present several approaches which take the scene into consideration in reconstructing and synthesizing Human-Scene Interaction (HSI). We first observe that state-of-the-art pose estimation methods ignore the 3D scene and hence reconstruct poses that are inconsistent with the scene. We address this by proposing a pose estimation method that takes the 3D scene explicitly into account. We call our method PROX for Proximal Relationships with Object eXclusion. We leverage the data generated using PROX and build a method to automatically place 3D scans of people with clothing in scenes. The core novelty of our method is encoding the proximal relationships between the human and the scene in a novel HSI model, called POSA for Pose with prOximitieS and contActs. POSA is limited to static HSI, however. We propose a real-time method for synthesizing dynamic HSI, which we call SAMP for Scene-Aware Motion Prediction. SAMP enables virtual humans to navigate cluttered indoor scenes and naturally interact with objects. Data-driven kinematic models, like SAMP, can produce high-quality motion when applied in environments similar to those shown in the dataset. However, when applied to new scenarios, kinematic models can struggle to generate realistic behaviors that respect scene constraints. In contrast, we present InterPhys which uses adversarial imitation learning and reinforcement learning to train physically-simulated characters that perform scene interaction tasks in a physical and life-like manner
Probabilistic Inference for Model Based Control
Robotic systems are essential for enhancing productivity, automation, and performing hazardous tasks. Addressing the unpredictability of physical systems, this thesis advances robotic planning and control under uncertainty, introducing learning-based methods for managing uncertain parameters and adapting to changing environments in real-time.
Our first contribution is a framework using Bayesian statistics for likelihood-free inference of model parameters. This allows employing complex simulators for designing efficient, robust controllers. The method, integrating the unscented transform with a variant of information theoretical model predictive control, shows better performance in trajectory evaluation compared to Monte Carlo sampling, easing the computational load in various control and robotics tasks.
Next, we reframe robotic planning and control as a Bayesian inference problem, focusing on the posterior distribution of actions and model parameters. An implicit variational inference algorithm, performing Stein Variational Gradient Descent, estimates distributions over model parameters and control inputs in real-time. This Bayesian approach effectively handles complex multi-modal posterior distributions, vital for dynamic and realistic robot navigation.
Finally, we tackle diversity in high-dimensional spaces. Our approach mitigates underestimation of uncertainty in posterior distributions, which leads to locally optimal solutions. Using the theory of rough paths, we develop an algorithm for parallel trajectory optimisation, enhancing solution diversity and avoiding mode collapse. This method extends our variational inference approach for trajectory estimation, employing diversity-enhancing kernels and leveraging path signature representation of trajectories. Empirical tests, ranging from 2-D navigation to robotic manipulators in cluttered environments, affirm our method's efficiency, outperforming existing alternatives
Assessing prioritization measures for a private land conservation program in the U.S. Prairie Pothole Region
Private land conservation has become an important tool for protecting biodiversity and habitat, but methods for prioritizing and scheduling conservation on private land are still being developed. While return on investment methods have been suggested as a potential path forward, the different processes linking private landscapes to the socioeconomic systems in which they are embedded create unique challenges for scheduling conservation with this approach. We investigated a range of scheduling approaches within a return on investment framework for breeding waterfowl and broods in the Prairie Pothole Region of North Dakota, South Dakota, and Montana. Current conservation targeting for waterfowl in the region focuses mostly on the distribution and abundance of breeding waterfowl. We tested whether MaxGain approaches for waterfowl conservation differed from MinLoss approaches in terms of return on investment and which approach performed best in avoiding loss of waterfowl and broods separately. We also examined variation in results based upon the temporal scale of the abundance layers used for input and compared the region's current scheduling approach with results from our simulations. Our results suggested that MinLoss was the most efficient scheduling approach for both breeding waterfowl and broods and that using just breeding waterfowl to target areas for conservation programs might cause organizations to overlook important areas for broods, particularly over shorter timespans. The higher efficiency of MinLoss approaches in our simulations also indicated that incorporating probability of wetland drainage into decision-making improved the overall return on investment. We recommend that future conservation scheduling for easements in the region and for private land conservation in general include some form of return on investment or cost-effective analysis to make conservation more transparent
Recommended from our members
Rigorous Experimentation For Reinforcement Learning
Scientific fields make advancements by leveraging the knowledge created by others to push the boundary of understanding. The primary tool in many fields for generating knowledge is empirical experimentation. Although common, generating accurate knowledge from empirical experiments is often challenging due to inherent randomness in execution and confounding variables that can obscure the correct interpretation of the results. As such, researchers must hold themselves and others to a high degree of rigor when designing experiments. Unfortunately, most reinforcement learning (RL) experiments lack this rigor, making the knowledge generated from experiments dubious. This dissertation proposes methods to address central issues in RL experimentation.
Evaluating the performance of an RL algorithm is the most common type of experiment in RL literature. Most performance evaluations are often incapable of answering a specific research question and produce misleading results. Thus, the first issue we address is how to create a performance evaluation procedure that holds up to scientific standards.
Despite the prevalence of performance evaluation, these types of experiments produce limited knowledge, e.g., they can only show how well an algorithm worked and not why, and they require significant amounts of time and computational resources. As an alternative, this dissertation proposes that scientific testing, the process of conducting carefully controlled experiments designed to further the knowledge and understanding of how an algorithm works, should be the primary form of experimentation.
Lastly, this dissertation provides a case study using policy gradient methods, showing how scientific testing can replace performance evaluation as the primary form of experimentation. As a result, this dissertation can motivate others in the field to adopt more rigorous experimental practices
A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery
Semantic segmentation (classification) of Earth Observation imagery is a
crucial task in remote sensing. This paper presents a comprehensive review of
technical factors to consider when designing neural networks for this purpose.
The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural
Networks (RNNs), Generative Adversarial Networks (GANs), and transformer
models, discussing prominent design patterns for these ANN families and their
implications for semantic segmentation. Common pre-processing techniques for
ensuring optimal data preparation are also covered. These include methods for
image normalization and chipping, as well as strategies for addressing data
imbalance in training samples, and techniques for overcoming limited data,
including augmentation techniques, transfer learning, and domain adaptation. By
encompassing both the technical aspects of neural network design and the
data-related considerations, this review provides researchers and practitioners
with a comprehensive and up-to-date understanding of the factors involved in
designing effective neural networks for semantic segmentation of Earth
Observation imagery.Comment: 145 pages with 32 figure
Some Supervision Required: Incorporating Oracle Policies in Reinforcement Learning via Epistemic Uncertainty Metrics
An inherent problem of reinforcement learning is performing exploration of an
environment through random actions, of which a large portion can be
unproductive. Instead, exploration can be improved by initializing the learning
policy with an existing (previously learned or hard-coded) oracle policy,
offline data, or demonstrations. In the case of using an oracle policy, it can
be unclear how best to incorporate the oracle policy's experience into the
learning policy in a way that maximizes learning sample efficiency. In this
paper, we propose a method termed Critic Confidence Guided Exploration (CCGE)
for incorporating such an oracle policy into standard actor-critic
reinforcement learning algorithms. More specifically, CCGE takes in the oracle
policy's actions as suggestions and incorporates this information into the
learning scheme when uncertainty is high, while ignoring it when the
uncertainty is low. CCGE is agnostic to methods of estimating uncertainty, and
we show that it is equally effective with two different techniques.
Empirically, we evaluate the effect of CCGE on various benchmark reinforcement
learning tasks, and show that this idea can lead to improved sample efficiency
and final performance. Furthermore, when evaluated on sparse reward
environments, CCGE is able to perform competitively against adjacent algorithms
that also leverage an oracle policy. Our experiments show that it is possible
to utilize uncertainty as a heuristic to guide exploration using an oracle in
reinforcement learning. We expect that this will inspire more research in this
direction, where various heuristics are used to determine the direction of
guidance provided to learning.Comment: Under review at TML
Reasoning about quantities and concepts: studies in social learning
We live and learn in a ‘society of mind’. This means that we form beliefs not
just based on our own observations and prior expectations but also based on the
communications from other people, such as our social network peers. Across seven
experiments, I study how people combine their own private observations with other
people’s communications to form and update beliefs about the environment. I will
follow the tradition of rational analysis and benchmark human learning against optimal Bayesian inference at Marr’s computational level. To accommodate human
resource constraints and cognitive biases, I will further contrast human learning
with a variety of process level accounts. In Chapters 2–4, I examine how people
reason about simple environmental quantities. I will focus on the effect of dependent information sources on the success of group and individual learning across a
series of single-player and multi-player judgement tasks. Overall, the results from
Chapters 2–4 highlight the nuances of real social network dynamics and provide
insights into the conditions under which we can expect collective success versus
failures such as the formation of inaccurate worldviews. In Chapter 5, I develop a
more complex social learning task which goes beyond estimation of environmental
quantities and focuses on inductive inference with symbolic concepts. Here, I investigate how people search compositional theory spaces to form and adapt their
beliefs, and how symbolic belief adaptation interfaces with individual and social
learning in a challenging active learning task. Results from Chapter 5 suggest that
people might explore compositional theory spaces using local incremental search;
and that it is difficult for people to use another person’s learning data to improve
upon their hypothesis
- …