1,001 research outputs found
Bayesian learning of noisy Markov decision processes
We consider the inverse reinforcement learning problem, that is, the problem
of learning from, and then predicting or mimicking a controller based on
state/action data. We propose a statistical model for such data, derived from
the structure of a Markov decision process. Adopting a Bayesian approach to
inference, we show how latent variables of the model can be estimated, and how
predictions about actions can be made, in a unified framework. A new Markov
chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior
distribution. This step includes a parameter expansion step, which is shown to
be essential for good convergence properties of the MCMC sampler. As an
illustration, the method is applied to learning a human controller
On particle Gibbs sampling
The particle Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm to
sample from the full posterior distribution of a state-space model. It does so
by executing Gibbs sampling steps on an extended target distribution defined on
the space of the auxiliary variables generated by an interacting particle
system. This paper makes the following contributions to the theoretical study
of this algorithm. Firstly, we present a coupling construction between two
particle Gibbs updates from different starting points and we show that the
coupling probability may be made arbitrarily close to one by increasing the
number of particles. We obtain as a direct corollary that the particle Gibbs
kernel is uniformly ergodic. Secondly, we show how the inclusion of an
additional Gibbs sampling step that reselects the ancestors of the particle
Gibbs' extended target distribution, which is a popular approach in practice to
improve mixing, does indeed yield a theoretically more efficient algorithm as
measured by the asymptotic variance. Thirdly, we extend particle Gibbs to work
with lower variance resampling schemes. A detailed numerical study is provided
to demonstrate the efficiency of particle Gibbs and the proposed variants.Comment: Published at http://dx.doi.org/10.3150/14-BEJ629 in the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Sequential quasi-Monte Carlo: Introduction for Non-Experts, Dimension Reduction, Application to Partly Observed Diffusion Processes
SMC (Sequential Monte Carlo) is a class of Monte Carlo algorithms for
filtering and related sequential problems. Gerber and Chopin (2015) introduced
SQMC (Sequential quasi-Monte Carlo), a QMC version of SMC. This paper has two
objectives: (a) to introduce Sequential Monte Carlo to the QMC community, whose
members are usually less familiar with state-space models and particle
filtering; (b) to extend SQMC to the filtering of continuous-time state-space
models, where the latent process is a diffusion. A recurring point in the paper
will be the notion of dimension reduction, that is how to implement SQMC in
such a way that it provides good performance despite the high dimension of the
problem.Comment: To be published in the proceedings of MCMQMC 201
Transcriptional Regulation of Dendritic Cell Diversity
Dendritic cells (DCs) are specialized antigen presenting cells that are exquisitely adapted to sense pathogens and induce the development of adaptive immune responses. They form a complex network of phenotypically and functionally distinct subsets. Within this network, individual DC subsets display highly specific roles in local immunosurveillance, migration, and antigen presentation. This division of labor amongst DCs offers great potential to tune the immune response by harnessing subset-specific attributes of DCs in the clinical setting. Until recently, our understanding of DC subsets has been limited and paralleled by poor clinical translation and efficacy. We have now begun to unravel how different DC subsets develop within a complex multilayered system. These findings open up exciting possibilities for targeted manipulation of DC subsets. Furthermore, ground-breaking developments overcoming a major translational obstacle – identification of similar DC populations in mouse and man – now sets the stage for significant advances in the field. Here we explore the determinants that underpin cellular and transcriptional heterogeneity within the DC network, how these influence DC distribution and localization at steady-state, and the capacity of DCs to present antigens via direct or cross-presentation during pathogen infection
Forest resampling for distributed sequential Monte Carlo
This paper brings explicit considerations of distributed computing
architectures and data structures into the rigorous design of Sequential Monte
Carlo (SMC) methods. A theoretical result established recently by the authors
shows that adapting interaction between particles to suitably control the
Effective Sample Size (ESS) is sufficient to guarantee stability of SMC
algorithms. Our objective is to leverage this result and devise algorithms
which are thus guaranteed to work well in a distributed setting. We make three
main contributions to achieve this. Firstly, we study mathematical properties
of the ESS as a function of matrices and graphs that parameterize the
interaction amongst particles. Secondly, we show how these graphs can be
induced by tree data structures which model the logical network topology of an
abstract distributed computing environment. Thirdly, we present efficient
distributed algorithms that achieve the desired ESS control, perform resampling
and operate on forests associated with these trees
Harold Jeffreys's Theory of Probability Revisited
Published exactly seventy years ago, Jeffreys's Theory of Probability (1939)
has had a unique impact on the Bayesian community and is now considered to be
one of the main classics in Bayesian Statistics as well as the initiator of the
objective Bayes school. In particular, its advances on the derivation of
noninformative priors as well as on the scaling of Bayes factors have had a
lasting impact on the field. However, the book reflects the characteristics of
the time, especially in terms of mathematical rigor. In this paper we point out
the fundamental aspects of this reference work, especially the thorough
coverage of testing problems and the construction of both estimation and
testing noninformative priors based on functional divergences. Our major aim
here is to help modern readers in navigating in this difficult text and in
concentrating on passages that are still relevant today.Comment: This paper commented in: [arXiv:1001.2967], [arXiv:1001.2968],
[arXiv:1001.2970], [arXiv:1001.2975], [arXiv:1001.2985], [arXiv:1001.3073].
Rejoinder in [arXiv:0909.1008]. Published in at
http://dx.doi.org/10.1214/09-STS284 the Statistical Science
(http://www.imstat.org/sts/) by the Institute of Mathematical Statistics
(http://www.imstat.org
Some discussions of D. Fearnhead and D. Prangle's Read Paper "Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation"
This report is a collection of comments on the Read Paper of Fearnhead and
Prangle (2011), to appear in the Journal of the Royal Statistical Society
Series B, along with a reply from the authors.Comment: 10 page
Chronic wounds consultation by telemedicine between a rehabilitation healthcare center and nursing home or home
Saint-Hélier Rehabilitation Center (pôle MPR Saint-Hélier), located in Rennes, has been selected for a regional telemedicine project in 2014 about chronic wounds.AimTo make care access easier for heavy disabilities patients in nursing homes or at home with chronic wounds.MethodThe members of TLM Pl@ies chronic team are specialist doctors and nurses for wounds. On request, the occupational therapist or dietician involve in the consultation (multidisciplinary approach). A secure videoconference (web) is used.ResultsSince July 2014, over 100 teleconsultations have been done. Targeted population is constituted by patients:– whose access to care is decreased due to moving difficulties;– of which the health care team is crossing difficulties in the care process (wound care but also disability, nutrition..).Seventy percent of requests come from the nursing home, 30% from homes (pressure ulcers stages 3 and 4, arterial ulcers, venous or mixed). Middle age: 78 years (20–101 years). Only 3 patients refused. Time to organize the teleconsultation is on average 13 days. Consultations last on average 25 minutes. In 30% of cases the teleconsultation is extended by a real live training time for the nurse at home guided by the TLM Pl@ies chroniques team. We evaluate professional satisfaction and technical satisfaction. Without teleconsultation, in 77% of cases transportation request for consultation would be made, in 5% hospitalization. In 18% no request would be done.Discussion/conclusionThese first results, encouraging, confirms the interest of specialized consultations in medico-social settings, and telemedicine can be an effective solution
- …