1,286 research outputs found
Full-State Quantum Circuit Simulation by Using Data Compression
Quantum circuit simulations are critical for evaluating quantum algorithms
and machines. However, the number of state amplitudes required for full
simulation increases exponentially with the number of qubits. In this study, we
leverage data compression to reduce memory requirements, trading computation
time and fidelity for memory space. Specifically, we develop a hybrid solution
by combining the lossless compression and our tailored lossy compression method
with adaptive error bounds at each timestep of the simulation. Our approach
optimizes for compression speed and makes sure that errors due to lossy
compression are uncorrelated, an important property for comparing simulation
output with physical machines. Experiments show that our approach reduces the
memory requirement of simulating the 61-qubit Grover's search algorithm from 32
exabytes to 768 terabytes of memory on Argonne's Theta supercomputer using
4,096 nodes. The results suggest that our techniques can increase the
simulation size by 2 to 16 qubits for general quantum circuits.Comment: Published in SC2019. Please cite the SC versio
CEAZ: Accelerating Parallel I/O via Hardware-Algorithm Co-Design of Efficient and Adaptive Lossy Compression
As supercomputers continue to grow to exascale, the amount of data that needs
to be saved or transmitted is exploding. To this end, many previous works have
studied using error-bounded lossy compressors to reduce the data size and
improve the I/O performance. However, little work has been done for effectively
offloading lossy compression onto FPGA-based SmartNICs to reduce the
compression overhead. In this paper, we propose a hardware-algorithm co-design
of efficient and adaptive lossy compressor for scientific data on FPGAs (called
CEAZ) to accelerate parallel I/O. Our contribution is fourfold: (1) We propose
an efficient Huffman coding approach that can adaptively update Huffman
codewords online based on codewords generated offline (from a variety of
representative scientific datasets). (2) We derive a theoretical analysis to
support a precise control of compression ratio under an error-bounded
compression mode, enabling accurate offline Huffman codewords generation. This
also helps us create a fixed-ratio compression mode for consistent throughput.
(3) We develop an efficient compression pipeline by adopting cuSZ's
dual-quantization algorithm to our hardware use case. (4) We evaluate CEAZ on
five real-world datasets with both a single FPGA board and 128 nodes from
Bridges-2 supercomputer. Experiments show that CEAZ outperforms the second-best
FPGA-based lossy compressor by 2X of throughput and 9.6X of compression ratio.
It also improves MPI_File_write and MPI_Gather throughputs by up to 25.8X and
24.8X, respectively.Comment: 14 pages, 17 figures, 8 table
Implementation issues in source coding
An edge preserving image coding scheme which can be operated in both a lossy and a lossless manner was developed. The technique is an extension of the lossless encoding algorithm developed for the Mars observer spectral data. It can also be viewed as a modification of the DPCM algorithm. A packet video simulator was also developed from an existing modified packet network simulator. The coding scheme for this system is a modification of the mixture block coding (MBC) scheme described in the last report. Coding algorithms for packet video were also investigated
- …