Parallel Metropolis chains with cooperative adaptation

Monte Carlo methods, such as Markov chain Monte Carlo (MCMC) algorithms, have become very popular in signal processing over the last years. In this work, we introduce a novel MCMC scheme where parallel MCMC chains interact, adapting cooperatively the parameters of their proposal functions. Furthermore, the novel algorithm distributes the computational effort adaptively, rewarding the chains which are providing better performance and, possibly even stopping other ones. These extinct chains can be reactivated if the algorithm considers necessary. Numerical simulations shows the benefits of the novel scheme

Orthogonal parallel MCMC methods for sampling and optimization

Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, several schemes employing multiple parallel MCMC chains have been recently introduced. In this work, we describe a novel parallel interacting MCMC scheme, called {\it orthogonal MCMC} (O-MCMC), where a set of "vertical" parallel MCMC chains share information using some "horizontal" MCMC techniques working on the entire population of current states. More specifically, the vertical chains are led by random-walk proposals, whereas the horizontal MCMC techniques employ independent proposals, thus allowing an efficient combination of global exploration and local approximation. The interaction is contained in these horizontal iterations. Within the analysis of different implementations of O-MCMC, novel schemes in order to reduce the overall computational cost of parallel multiple try Metropolis (MTM) chains are also presented. Furthermore, a modified version of O-MCMC for optimization is provided by considering parallel simulated annealing (SA) algorithms. Numerical results show the advantages of the proposed sampling scheme in terms of efficiency in the estimation, as well as robustness in terms of independence with respect to initial values and the choice of the parameters

Layered adaptive importance sampling

Monte Carlo methods represent the de facto standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performance of any such method is strictly related to the specification of the proposal distribution, such that unfortunate choices easily wreak havoc on the resulting estimators. In this work, we introduce a layered (i.e., hierarchical) procedure to generate samples employed within a Monte Carlo scheme. This approach ensures that an appropriate equivalent proposal density is always obtained automatically (thus eliminating the risk of a catastrophic performance), although at the expense of a moderate increase in the complexity. Furthermore, we provide a general unified importance sampling (IS) framework, where multiple proposal densities are employed and several IS schemes are introduced by applying the so-called deterministic mixture approach. Finally, given these schemes, we also propose a novel class of adaptive importance samplers using a population of proposals, where the adaptation is driven by independent parallel or interacting Markov chain Monte Carlo (MCMC) chains. The resulting algorithms efficiently combine the benefits of both IS and MCMC methods.Peer reviewe

Efficient Bayesian inference via Monte Carlo and machine learning algorithms

Mención Internacional en el título de doctorIn many fields of science and engineering, we are faced with an inverse problem where we aim to recover an unobserved parameter or variable of interest from a set of observed variables. Bayesian inference is a probabilistic approach for inferring this unknown parameter that has become extremely popular, finding application in myriad problems in fields such as machine learning, signal processing, remote sensing and astronomy. In Bayesian inference, all the information about the parameter is summarized by the posterior distribution. Unfortunately, the study of the posterior distribution requires the computation of complicated integrals, that are analytically intractable and need to be approximated. Monte Carlo is a huge family of sampling algorithms for performing optimization and numerical integration that has become the main horsepower for carrying out Bayesian inference. The main idea of Monte Carlo is that we can approximate the posterior distribution by a set of samples, obtained by an iterative process that involves sampling from a known distribution. Markov chain Monte Carlo (MCMC) and importance sampling (IS) are two important groups of Monte Carlo algorithms. This thesis focuses on developing and analyzing Monte Carlo algorithms (either MCMC, IS or combination of both) under different challenging scenarios presented below. In summary, in this thesis we address several important points, enumerated (a)–(f), that currently represent a challenge in Bayesian inference via Monte Carlo. A first challenge that we address is the problematic exploration of the parameter space by off-the-shelf MCMC algorithms when there is (a) multimodality, or with (b) highly concentrated posteriors. Another challenge that we address is the (c) proposal construction in IS. Furtheremore, in recent applications we need to deal with (d) expensive posteriors, and/or we need to handle (e) noisy posteriors. Finally, the Bayesian framework also offers a way of comparing competing hypothesis (models) in a principled way by means of marginal likelihoods. Hence, a task that arises as of fundamental importance is (f) marginal likelihood computation. Chapters 2 and 3 deal with (a), (b), and (c). In Chapter 2, we propose a novel population MCMC algorithm called Parallel Metropolis-Hastings Coupler (PMHC). PMHC is very suitable for multimodal scenarios since it works with a population of states, instead of a single one, hence allowing for sharing information. PMHC combines independent exploration by the use of parallel Metropolis-Hastings algorithms, with cooperative exploration by the use of a population MCMC technique called Normal Kernel Coupler. In Chapter 3, population MCMC are combined with IS within the layered adaptive IS (LAIS) framework. The combination of MCMC and IS serves two purposes. First, an automatic proposal construction. Second, it aims at increasing the robustness, since the MCMC samples are not used directly to form the sample approximation of the posterior. The use of minibatches of data is proposed to deal with highly concentrated posteriors. Other extensions for reducing the costs with respect to the vanilla LAIS framework, based on recycling and clustering, are discussed and analyzed. Chapters 4, 5 and 6 deal with (c), (d) and (e). The use of nonparametric approximations of the posterior plays an important role in the design of efficient Monte Carlo algorithms. Nonparametric approximations of the posterior can be obtained using machine learning algorithms for nonparametric regression, such as Gaussian Processes and Nearest Neighbors. Then, they can serve as cheap surrogate models, or for building efficient proposal distributions. In Chapter 4, in the context of expensive posteriors, we propose adaptive quadratures of posterior expectations and the marginal likelihood using a sequential algorithm that builds and refines a nonparametric approximation of the posterior. In Chapter 5, we propose Regression-based Adaptive Deep Importance Sampling (RADIS), an adaptive IS algorithm that uses a nonparametric approximation of the posterior as the proposal distribution. We illustrate the proposed algorithms in applications of astronomy and remote sensing. Chapter 4 and 5 consider noiseless posterior evaluations for building the nonparametric approximations. More generally, in Chapter 6 we give an overview and classification of MCMC and IS schemes using surrogates built with noisy evaluations. The motivation here is the study of posteriors that are both costly and noisy. The classification reveals a connection between algorithms that use the posterior approximation as a cheap surrogate, and algorithms that use it for building an efficient proposal. We illustrate specific instances of the classified schemes in an application of reinforcement learning. Finally, in Chapter 7 we study noisy IS, namely, IS when the posterior evaluations are noisy, and derive optimal proposal distributions for the different estimators in this setting. Chapter 8 deals with (f). In Chapter 8, we provide with an exhaustive review of methods for marginal likelihood computation, with special focus on the ones based on Monte Carlo. We derive many connections among the methods and compare them in several simulations setups. Finally, in Chapter 9 we summarize the contributions of this thesis and discuss some potential avenues of future research.Programa de Doctorado en Ingeniería Matemática por la Universidad Carlos III de MadridPresidente: Valero Laparra Pérez-Muelas.- Secretario: Michael Peter Wiper.- Vocal: Omer Deniz Akyildi

Phylogenetic studies in the genus Jamesbrittenia tribe Manuleae, family Scrophulariaceae

Word processed copy.Includes bibliographical references.Jamesbrittenia is a genus of 84 largely perennial sub-shrubs and herbs with a wide distribution in southern Africa (except J. dissecta in Sudan, Egypt and India). Plastid (rps16 and psbA-trnH) and nuclear (GScp) sequences were obtained for 42 species, mostly from the arid winter-rainfall west and southern regions of southern Africa. Applying both parsimony and Bayesian inference to combined molecular and morphological data sets, a phylogenetic hypothesis which is robust at the deeper nodes, was produced. This supports the monophyly of Jamesbrittenia, and also indicates the early divergence of J. ramosissima and three major clades

Response Of Forest Birds To Spotted Wing Drosophila (Drosophila Suzukii Matsumura), A Novel Invasive Fruit Pest, At Allegheny National Forest

Non-native invasive insect pests can have dramatic impacts on native ecosystems, feeding on plant foliage, wood, or sap. Little is known, however, about how fruit-targeting NNIIPs may affect native ecosystems. Spotted wing Drosophila (Drosophila suzukii Matsumura, SWD) is a recently introduced invasive vinegar fly that parasitizes the fruits of many plant species in the United States. While its activity in agricultural systems is well-documented, little is known about its activity in forest ecosystems, despite growing evidence of its presence and parasitism of fruits there. Parasitism could reduce fruit attractiveness for vertebrate fruit consumers, including migratory birds. As such, this may reduce soft mast food availability for birds in early successional habitat during the post-breeding season and fall migration, interfering with energy accumulation when demands are greatest. This thesis includes 4 chapters focused on understanding factors influencing SWD in invaded forest habitat and whether SWD activity associates with avian abundances, species richness, body condition, and fruit consumption at Allegheny National Forest in northwestern Pennsylvania. In Chapter 1, I provide context and justification for this research. I first describe how NNIIPs are known to influence forest ecosystems. I then review information on how SWD affects fruit crops in agricultural systems and provide a framework detailing how fruit parasitism by SWD may alter forest ecosystems. I lay out the research on fruit parasitism and fruit consumption by wildlife, especially fruit-consuming birds. I relate this to cascading effects on the following: 1) seed dispersal, 2) forest community structure and composition, and 3) food resources for wildlife. Finally, I highlight important research needs to uncover how SWD may affect forest ecosystems. In Chapter 2, I investigate how SWD distribution and abundance are influenced by forest composition, with respect to dominance by black cherry (Prunus serotina), a highly parasitized host, and local fruit resources in a forested ecosystem. From July to October 2019 and 2020 I trapped SWD in 21 regenerating timber harvests and surrounding forest canopies, representing 2 different cover types, and I collected Allegheny blackberry (Rubus allegheniensis) fruit samples in those regenerating timber harvests to observe for emergence of SWD. Relative abundance of SWD in regenerating timber harvests and surrounding forest canopy correlated to host resources at the local (i.e., sampling point) level but not the adjacent cover type. Parasitism of R. allegheniensis fruits positively correlated with local R allegheniensis fruits and basal area of P. serotina in adjacent mature forest. The results of this study provide the first evidence for the importance of local resources over those in an adjacent cover type or surrounding forest for predicting SWD trap captures, and the importance of local fruit resources and host plant density in adjacent cover type for SWD fruit parasitism in invaded forest ecosystems. I suggest these relationships can help predict the timing and abundance of SWD in both natural and semi-natural systems where wild fruiting plants are abundant, providing valuable information for monitoring and management planning. In Chapter 3, I identify factors, primarily those related to SWD abundance and fruit parasitism, influencing frugivorous and non-frugivorous bird relative abundance and species richness in regenerating timber harvests invaded by SWD during the post-breeding season and fall migration. From July to September 2019 and 2020 I sampled bird communities using mist net captures in 21 regenerating timber harvests, and I trapped SWD and sampled R. allegheniensis fruits for SWD parasitism at each mist net. One species, eastern towhee (Pipilo erythrophthalmus), correlated negatively with SWD trap captures. Two species, hooded warbler (Setophaga citrina) and ovenbird (Seiurus aurocapilla), correlated negatively and positively to parasitism of fruits, respectively. Post-breeding and migratory bird relative abundances were otherwise unrelated to SWD variables in regenerating harvests. Frugivorous birds either did not alter fruit consumption in response to SWD, or birds’ abilities to alter diet and foraging behavior masked relationships to SWD. Eastern towhee likely perceived fruits covered with flies as less desirable for consumption or may have been deterred by SWD swarms. Ovenbirds and hooded warblers altered their fruit consumption based on their responses to parasitism in modeling results. Alternatively, SWD activity may have affected arthropod communities, shifting composition and affecting some birds via trophic cascading effects. I highlight the importance of investigating arthropod community responses and the actual consumption of fruit resources in habitats invaded by SWD as critical next steps to understanding the ecological effects of SWD. In Chapter 4, I evaluate how factors, primarily those related to SWD abundance and fruit parasitism, relate to body condition of several bird species, and probability of fruit consumption by frugivorous birds in regenerating timber harvests invaded by SWD during the post-breeding season and fall migration. From July to September 2019 and 2020 I gathered data on body condition indices—subcutaneous fat, feather molt progression, and scaled mass index (SMI)— and collected fecal samples for evidence of fruit consumption for individuals of several bird species captured using mist nets in 21 regenerating timber harvests. I also trapped SWD and sampled R. allegheniensis fruits for SWD parasitism at each mist net. Multiple condition indices for several species correlated with relative abundance of SWD and parasitism of fruits, though the directions of relationships varied and were species-specific. Probability of R. allegheniensis fruit consumption by frugivorous birds correlated positively with parasitism of fruits at both the guild and species level. Frugivorous birds were more likely to consume parasitized fruits, and one species, gray catbird, experienced changes to nutritional intake and subsequent condition as a result. Fruit parasitism and greater abundances of adult SWD may have altered arthropod communities, altering composition and changing an additional food resource for both non-frugivorous and frugivorous birds. Most species improved condition indices as the season progressed, likely due to dietary flexibility. I list several critical avenues for research to better understand the relationships observed in this study: 1) how bird condition changes in relation to foraging microhabitat selection and SWD, 2) selection or rejection of SWD-parasitized fruits by frugivores related to nutritional metrics, and 3) arthropod community responses to SWD

Uncomplicated urinary tract infection in primary care; evaluation of point of care tests and patient management

Antibiotic resistance is an increasing global public health problem. Resistance is increasing sharply in gram-negative organisms, including Escherichia coli (E. coli), the main causative organism for community-acquired urinary tract infection (UTI). Antimicrobial stewardship strategies in primary care to help contain antibiotic resistance include supporting general practitioners (GPs) in deciding whether to prescribe an antibiotic for UTI and selecting the most appropriate antibiotic. In this thesis, I aim to describe the management of uncomplicated UTI in primary care and evaluate potential point of care tests (POCT) to assist the diagnosis and/or appropriate prescribing of antibiotics for uncomplicated UTI. The program of work includes: 1. Laboratory evaluation of a culture-based test that allows the quantification, identification and susceptibility profile of infecting bacteria from urine (FlexicultTM). 2. Evaluation of a novel chromatic sensing technique to identify bacterially infected urine compared to visual assessment of urine turbidity and urinalysis dipsticks. 3. Systematic review and analysis of data (descriptive and multi-level modelling) from an international primary care based observational study to describe UTI management. I identified unwarranted variation in clinical management of UTI between countries and between general practices within countries. Empirical antibiotic prescribing for UTI in Europe is high and treatment is generally prescribed for longer than guidelines recommend. FlexicultTM identifying bacterial UTI. The use of FlexicultTM in practice may support GPs in screening out negative samples reducing the proportion of patients that are prescribed antibiotics empirically. Chromatic sensing and visually assessing turbidity were equally useful at identifying negative urine samples and both improved the analytic performance of urinalysis dipsticks. The chromatic sensing system requires development prior to further evaluation

Bayesian models of category acquisition and meaning development

The ability to organize concepts (e.g., dog, chair) into efficient mental representations, i.e., categories (e.g., animal, furniture) is a fundamental mechanism which allows humans to perceive, organize, and adapt to their world. Much research has been dedicated to the questions of how categories emerge and how they are represented. Experimental evidence suggests that (i) concepts and categories are represented through sets of features (e.g., dogs bark, chairs are made of wood) which are structured into different types (e.g, behavior, material); (ii) categories and their featural representations are learnt jointly and incrementally; and (iii) categories are dynamic and their representations adapt to changing environments. This thesis investigates the mechanisms underlying the incremental and dynamic formation of categories and their featural representations through cognitively motivated Bayesian computational models. Models of category acquisition have been extensively studied in cognitive science and primarily tested on perceptual abstractions or artificial stimuli. In this thesis, we focus on categories acquired from natural language stimuli, using nouns as a stand-in for their reference concepts, and their linguistic contexts as a representation of the concepts’ features. The use of text corpora allows us to (i) develop large-scale unsupervised models thus simulating human learning, and (ii) model child category acquisition, leveraging the linguistic input available to children in the form of transcribed child-directed language. In the first part of this thesis we investigate the incremental process of category acquisition. We present a Bayesian model and an incremental learning algorithm which sequentially integrates newly observed data. We evaluate our model output against gold standard categories (elicited experimentally from human participants), and show that high-quality categories are learnt both from child-directed data and from large, thematically unrestricted text corpora. We find that the model performs well even under constrained memory resources, resembling human cognitive limitations. While lists of representative features for categories emerge from this model, they are neither structured nor jointly optimized with the categories. We address these shortcomings in the second part of the thesis, and present a Bayesian model which jointly learns categories and structured featural representations. We present both batch and incremental learning algorithms, and demonstrate the model’s effectiveness on both encyclopedic and child-directed data. We show that high-quality categories and features emerge in the joint learning process, and that the structured features are intuitively interpretable through human plausibility judgment evaluation. In the third part of the thesis we turn to the dynamic nature of meaning: categories and their featural representations change over time, e.g., children distinguish some types of features (such as size and shade) less clearly than adults, and word meanings adapt to our ever changing environment and its structure. We present a dynamic Bayesian model of meaning change, which infers time-specific concept representations as a set of feature types and their prevalence, and captures their development as a smooth process. We analyze the development of concept representations in their complexity over time from child-directed data, and show that our model captures established patterns of child concept learning. We also apply our model to diachronic change of word meaning, modeling how word senses change internally and in prevalence over centuries. The contributions of this thesis are threefold. Firstly, we show that a variety of experimental results on the acquisition and representation of categories can be captured with computational models within the framework of Bayesian modeling. Secondly, we show that natural language text is an appropriate source of information for modeling categorization-related phenomena suggesting that the environmental structure that drives category formation is encoded in this data. Thirdly, we show that the experimental findings hold on a larger scale. Our models are trained and tested on a larger set of concepts and categories than is common in behavioral experiments and the categories and featural representations they can learn from linguistic text are in principle unrestricted

Paper in architecture: Research by design, engineering and prototyping

Paper is a fascinating material that we encounter every day in different variants: tissues, paper towels, packaging material, wall paper or even fillers of doors. Despite radical changes in production technology, the material, which has been known to mankind for almost two thousand years, still has a natural composition, being made up of fibres of plant origin (particularly wood fibres). Thanks to its unique properties, relatively high compression strength and bending stiffness, low production costs and ease of recycling, paper is becoming more and more popular in many types of industry. Mass-produced paper products such as special paper, paperboard, corrugated cardboard, honeycomb panels, tubes and L- and U-shapes are suitable for use as a building material in the broad sense of these words – i.e., in design and architecture. Objects for everyday use, furniture, interior design elements and partitions are just a few examples of things in which paper can be employed. Temporary events such as festivals, exhibitions or sporting events like the Olympics require structures that only need to last for a limited period of time. When they are demolished after a few days or months, their leftovers can have a significant impact on the local environment. In the context of growing awareness of environmental threats and the efforts undertaken by local and international organisations and governments to counter these threats, the use of natural materials that can be recycled after their lifespan is becoming increasingly widespread. Paper and its derivatives fascinate designers and architects, who are always looking for new challenges and trying to meet the market’s demands for innovative and proecological solutions. Being a low-cost and readily available material, paper is suited to the production of emergency shelters for victims of natural and man-made disasters, as well as homeless persons. In order to gain a better understanding of paper’s potential in terms of architecture, its material properties were researched on a micro, meso and macro level. This research of the possible applications of paper in architecture was informed by two main research questions: What is paper and to what extent can it be used in architecture? What is the most suitable way to use paper in emergency architecture? To answer the first research question, fundamental and material research on paper and paper products had to be conducted. The composition of the material, production methods and properties of paper were researched. Then paper products with the potential to be used in architecture were examined. The history of the development of paper and its influence on civilisation helped the author gain a better understanding of the nature of this material, which we encounter in our lives every day. Research on objects for everyday use, furniture, pavilions and architecture realised in the last 150 years allowed the author to distinguish various types of paper design and paper architecture. Analysis of realised buildings in which paper products were used as structural elements and parts of the building envelope resulted in a wide array of possible solutions. Structural systems, types of connections between the various elements, impregnation methods and the functionalities and lifespan of different types of buildings were systematised. The knowledge thus collected allowed the author to conduct a further exploration of paper architecture in the form of designs and prototypes. To answer the second research question, the analysed case studies were translated into designs and prototypes of emergency shelters. During the research-by-design, engineering and prototyping phases, more than a dozen prototypes were built. The prototypes differed in terms of structural systems, used materials, connections between structural elements, impregnation methods, functionality and types of building. The three versions of the Transportable Emergency Cardboard House project presented in the final chapter form the author’s final answer to the second research question. Paper will never replace traditional building materials such as timber, concrete, steel, glass or plastic. It can, however, complement them to a significant degree. &nbsp