1,740 research outputs found
Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach
Bayesian inference typically requires the computation of an approximation to
the posterior distribution. An important requirement for an approximate
Bayesian inference algorithm is to output high-accuracy posterior mean and
uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain
Monte Carlo, remain the gold standard for approximate Bayesian inference
because they have a robust finite-sample theory and reliable convergence
diagnostics. However, alternative methods, which are more scalable or apply to
problems where Markov Chain Monte Carlo cannot be used, lack the same
finite-data approximation theory and tools for evaluating their accuracy. In
this work, we develop a flexible new approach to bounding the error of mean and
uncertainty estimates of scalable inference algorithms. Our strategy is to
control the estimation errors in terms of Wasserstein distance, then bound the
Wasserstein distance via a generalized notion of Fisher distance. Unlike
computing the Wasserstein distance, which requires access to the normalized
posterior distribution, the Fisher distance is tractable to compute because it
requires access only to the gradient of the log posterior density. We
demonstrate the usefulness of our Fisher distance approach by deriving bounds
on the Wasserstein error of the Laplace approximation and Hilbert coresets. We
anticipate that our approach will be applicable to many other approximate
inference methods such as the integrated Laplace approximation, variational
inference, and approximate Bayesian computationComment: 22 pages, 2 figure
Approximate Consensus in Highly Dynamic Networks: The Role of Averaging Algorithms
In this paper, we investigate the approximate consensus problem in highly
dynamic networks in which topology may change continually and unpredictably. We
prove that in both synchronous and partially synchronous systems, approximate
consensus is solvable if and only if the communication graph in each round has
a rooted spanning tree, i.e., there is a coordinator at each time. The striking
point in this result is that the coordinator is not required to be unique and
can change arbitrarily from round to round. Interestingly, the class of
averaging algorithms, which are memoryless and require no process identifiers,
entirely captures the solvability issue of approximate consensus in that the
problem is solvable if and only if it can be solved using any averaging
algorithm. Concerning the time complexity of averaging algorithms, we show that
approximate consensus can be achieved with precision of in a
coordinated network model in synchronous
rounds, and in rounds when
the maximum round delay for a message to be delivered is . While in
general, an upper bound on the time complexity of averaging algorithms has to
be exponential, we investigate various network models in which this exponential
bound in the number of nodes reduces to a polynomial bound. We apply our
results to networked systems with a fixed topology and classical benign fault
models, and deduce both known and new results for approximate consensus in
these systems. In particular, we show that for solving approximate consensus, a
complete network can tolerate up to 2n-3 arbitrarily located link faults at
every round, in contrast with the impossibility result established by Santoro
and Widmayer (STACS '89) showing that exact consensus is not solvable with n-1
link faults per round originating from the same node
Brief Announcement: Lower Bounds for Asymptotic Consensus in Dynamic Networks
In this work we study the performance of asymptotic and approximate consensus algorithms in dynamic networks. The asymptotic consensus problem requires a set of agents to repeatedly set their outputs such that the outputs converge to a common value within the convex hull of initial values. This problem, and the related approximate consensus problem, are fundamental building blocks in distributed systems where exact consensus among agents is not required, e.g., man-made distributed control systems, and have applications in the analysis of natural distributed systems, such as flocking and opinion dynamics. We prove new nontrivial lower bounds on the contraction rates of asymptotic consensus algorithms, from which we deduce lower bounds on the time complexity of approximate consensus algorithms. In particular, the obtained bounds show optimality of asymptotic and approximate consensus algorithms presented in [Charron-Bost et al., ICALP\u2716] for certain classes of networks that include classical failure assumptions, and confine the search for optimal bounds in the general case.
Central to our lower bound proofs is an extended notion of valency, the set of reachable limits of an asymptotic consensus algorithm starting from a given configuration. We further relate topological properties of valencies to the solvability of exact consensus, shedding some light on the relation of these three fundamental problems in dynamic networks
Distributed multi-agent Gaussian regression via finite-dimensional approximations
We consider the problem of distributedly estimating Gaussian processes in
multi-agent frameworks. Each agent collects few measurements and aims to
collaboratively reconstruct a common estimate based on all data. Agents are
assumed with limited computational and communication capabilities and to gather
noisy measurements in total on input locations independently drawn from a
known common probability density. The optimal solution would require agents to
exchange all the input locations and measurements and then invert an matrix, a non-scalable task. Differently, we propose two suboptimal
approaches using the first orthonormal eigenfunctions obtained from the
\ac{KL} expansion of the chosen kernel, where typically . The benefits
are that the computation and communication complexities scale with and not
with , and computing the required statistics can be performed via standard
average consensus algorithms. We obtain probabilistic non-asymptotic bounds
that determine a priori the desired level of estimation accuracy, and new
distributed strategies relying on Stein's unbiased risk estimate (SURE)
paradigms for tuning the regularization parameters and applicable to generic
basis functions (thus not necessarily kernel eigenfunctions) and that can again
be implemented via average consensus. The proposed estimators and bounds are
finally tested on both synthetic and real field data
- …