Random Bit Quadrature and Approximation of Distributions on Hilbert Spaces

We study the approximation of expectations \E(f(X)) for Gaussian random elements $X$ with values in a separable Hilbert space $H$ and Lipschitz continuous functionals $f \colon H \to \R$. We consider restricted Monte Carlo algorithms, which may only use random bits instead of random numbers. We determine the asymptotics (in some cases sharp up to multiplicative constants, in the other cases sharp up to logarithmic factors) of the corresponding $n$-th minimal error in terms of the decay of the eigenvalues of the covariance operator of $X$. It turns out that, within the margins from above, restricted Monte Carlo algorithms are not inferior to arbitrary Monte Carlo algorithms, and suitable random bit multilevel algorithms are optimal. The analysis of this problem leads to a variant of the quantization problem, namely, the optimal approximation of probability measures on $H$ by uniform distributions supported by a given, finite number of points. We determine the asymptotics (up to multiplicative constants) of the error of the best approximation for the one-dimensional standard normal distribution, for Gaussian measures as above, and for scalar autonomous SDEs

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by HPC applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are the two leading lossy compressors available to compress scientific data sets. However, their performance is not consistent across different data sets and across different fields of some data sets: for some fields SZ provides better compression performance, while other fields are better compressed with ZFP. This situation raises the need for an automatic online (during compression) selection between SZ and ZFP, with a minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. We implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70% with the same level of data distortion because of very accurate selection (around 99%) of the best-fit compressor, with little overhead (less than 7% in the experiments).Comment: 14 pages, 9 figures, first revisio