To conduct Bayesian inference with large data sets, it is often convenient or
necessary to distribute the data across multiple machines. We consider a
likelihood function expressed as a product of terms, each associated with a
subset of the data. Inspired by global variable consensus optimisation, we
introduce an instrumental hierarchical model associating auxiliary statistical
parameters with each term, which are conditionally independent given the
top-level parameters. One of these top-level parameters controls the
unconditional strength of association between the auxiliary parameters. This
model leads to a distributed MCMC algorithm on an extended state space yielding
approximations of posterior expectations. A trade-off between computational
tractability and fidelity to the original model can be controlled by changing
the association strength in the instrumental model. We further propose the use
of a SMC sampler with a sequence of association strengths, allowing both the
automatic determination of appropriate strengths and for a bias correction
technique to be applied. In contrast to similar distributed Monte Carlo
algorithms, this approach requires few distributional assumptions. The
performance of the algorithms is illustrated with a number of simulated
examples
