5 research outputs found
Multi-Party Set Reconciliation Using Characteristic Polynomials
In the standard set reconciliation problem, there are two parties and
, each respectively holding a set of elements and . The goal is
for both parties to obtain the union . In many distributed
computing settings the sets may be large but the set difference
is small. In these cases one aims to achieve
reconciliation efficiently in terms of communication; ideally, the
communication should depend on the size of the set difference, and not on the
size of the sets.
Recent work has considered generalizations of the reconciliation problem to
multi-party settings, using a framework based on a specific type of linear
sketch called an Invertible Bloom Lookup Table. Here, we consider multi-party
set reconciliation using the alternative framework of characteristic
polynomials, which have previously been used for efficient pairwise set
reconciliation protocols, and compare their performance with Invertible Bloom
Lookup Tables for these problems.Comment: 6 page
Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Models
We introduce a two-stage probabilistic framework for statistical downscaling
using unpaired data. Statistical downscaling seeks a probabilistic map to
transform low-resolution data from a biased coarse-grained numerical scheme to
high-resolution data that is consistent with a high-fidelity scheme. Our
framework tackles the problem by composing two transformations: (i) a debiasing
step via an optimal transport map, and (ii) an upsampling step achieved by a
probabilistic diffusion model with a posteriori conditional sampling. This
approach characterizes a conditional distribution without needing paired data,
and faithfully recovers relevant physical statistics from biased samples. We
demonstrate the utility of the proposed approach on one- and two-dimensional
fluid flow problems, which are representative of the core difficulties present
in numerical simulations of weather and climate. Our method produces realistic
high-resolution outputs from low-resolution inputs, by upsampling resolutions
of 8x and 16x. Moreover, our procedure correctly matches the statistics of
physical quantities, even when the low-frequency content of the inputs and
outputs do not match, a crucial but difficult-to-satisfy assumption needed by
current state-of-the-art alternatives. Code for this work is available at:
https://github.com/google-research/swirl-dynamics/tree/main/swirl_dynamics/projects/probabilistic_diffusion.Comment: NeurIPS 2023 (spotlight
Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations
We introduce a data-driven learning framework that assimilates two powerful
ideas: ideal large eddy simulation (LES) from turbulence closure modeling and
neural stochastic differential equations (SDE) for stochastic modeling. The
ideal LES models the LES flow by treating each full-order trajectory as a
random realization of the underlying dynamics, as such, the effect of
small-scales is marginalized to obtain the deterministic evolution of the LES
state. However, ideal LES is analytically intractable. In our work, we use a
latent neural SDE to model the evolution of the stochastic process and an
encoder-decoder pair for transforming between the latent space and the desired
ideal flow field. This stands in sharp contrast to other types of neural
parameterization of closure models where each trajectory is treated as a
deterministic realization of the dynamics. We show the effectiveness of our
approach (niLES - neural ideal LES) on a challenging chaotic dynamical system:
Kolmogorov flow at a Reynolds number of 20,000. Compared to competing methods,
our method can handle non-uniform geometries using unstructured meshes
seamlessly. In particular, niLES leads to trajectories with more accurate
statistics and enhances stability, particularly for long-horizon rollouts.Comment: 18 page