Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization

Today's HPC applications are producing extremely large amounts of data, such that data storage and analysis are becoming more challenging for scientific research. In this work, we design a new error-controlled lossy compression algorithm for large-scale scientific data. Our key contribution is significantly improving the prediction hitting rate (or prediction accuracy) for each data point based on its nearby data values along multiple dimensions. We derive a series of multilayer prediction formulas and their unified formula in the context of data compression. One serious challenge is that the data prediction has to be performed based on the preceding decompressed values during the compression in order to guarantee the error bounds, which may degrade the prediction accuracy in turn. We explore the best layer for the prediction by considering the impact of compression errors on the prediction accuracy. Moreover, we propose an adaptive error-controlled quantization encoder, which can further improve the prediction hitting rate considerably. The data size can be reduced significantly after performing the variable-length encoding because of the uneven distribution produced by our quantization encoder. We evaluate the new compressor on production scientific data sets and compare it with many other state-of-the-art compressors: GZIP, FPZIP, ZFP, SZ-1.1, and ISABELA. Experiments show that our compressor is the best in class, especially with regard to compression factors (or bit-rates) and compression errors (including RMSE, NRMSE, and PSNR). Our solution is better than the second-best solution by more than a 2x increase in the compression factor and 3.8x reduction in the normalized root mean squared error on average, with reasonable error bounds and user-desired bit-rates.Comment: Accepted by IPDPS'17, 11 pages, 10 figures, double colum

A review of attendances at Paediatric Accident and Emergency Department at Mater Dei Hospital for neurological complaints patient

Aims: Attendances at paediatric accident and emergency department (A&E) during a six month period were reviewed to determine the proportion of children with neurological complaints, type of symptoms and the outcomes in terms of admissions, discharges and out-patient referrals. Methods: Neurological complaints were classified as (a) febrile convulsions, (b) unprovoked seizures, (c) status epilepticus, (d) headaches, (e) altered consciousness, (f) acute ataxia, (g) flaccid weakness, (h) visual loss, or (i) others. Outcomes of these attendances were also recorded as either admission, referrals to out-patient clinics or discharges from A&E. Results: A total of 7670 children attended paediatric A&E during the study time of which 352 (4.5%) presented with neurological complaints. 173 children (49%) presented with headache, 54 (15.3%) presented with unprovoked seizures, 51 (14.4%) presented with febrile convulsions, 34 (9.6%) presented with altered consciousness and the remaining 40 children (11.7%) presented with various other complaints. 24.8% of children who presented with headache were admitted, 34.1% were referred to out-patient clinics and 41% were discharged. In contrast, 75.5% of children you presented with unprovoked seizures were admitted, 22.2% were referred to out-patient clinics and 3.7% were discharged. There were no deaths. Conclusion: 1 in 20 children who attended paediatric A&E presented with neurological complaints. One half of these children presented with headache, around one third presented with seizures (febrile and unprovoked), around 10% presented with altered consciousness. Around a half of these children were admitted, a quarter were discharged home and the other quarter were referred to out-patient clinics.peer-reviewe

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by HPC applications, significantly reducing data size is critical because of limited capacity of storage space and potential bottlenecks on I/O or networks in writing/reading or transferring data. SZ and ZFP are the two leading lossy compressors available to compress scientific data sets. However, their performance is not consistent across different data sets and across different fields of some data sets: for some fields SZ provides better compression performance, while other fields are better compressed with ZFP. This situation raises the need for an automatic online (during compression) selection between SZ and ZFP, with a minimal overhead. In this paper, the automatic selection optimizes the rate-distortion, an important statistical quality metric based on the signal-to-noise ratio. To optimize for rate-distortion, we investigate the principles of SZ and ZFP. We then propose an efficient online, low-overhead selection algorithm that predicts the compression quality accurately for two compressors in early processing stages and selects the best-fit compressor for each data field. We implement the selection algorithm into an open-source library, and we evaluate the effectiveness of our proposed solution against plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results on three data sets representing about 100 fields show that our selection algorithm improves the compression ratio up to 70% with the same level of data distortion because of very accurate selection (around 99%) of the best-fit compressor, with little overhead (less than 7% in the experiments).Comment: 14 pages, 9 figures, first revisio

Damping of liquid sloshing by foams: from everyday observations to liquid transport

We perform experiments on the sloshing dynamics of liquids in a rectangular container submitted to an impulse. We show that when foam is placed on top of the liquid the oscillations of the free interface are significantly damped. The ability to reduce sloshing and associated splashing could find applications in numerous industrial processes involving liquid transport.Comment: Accepted for publication in Journal of Visualizatio