4,770 research outputs found
Online compression of cache-filtered address traces
International audienceTrace-driven simulation is potentially much faster than cycle-accurate simulation. However, one drawback is the large amount of storage that may be necessary to store traces. Trace compression techniques are useful for decreasing the storage space requirement. But the compression ratio of existing trace compressors is limited because they implement lossless compression. We propose two new methods for compressing cachefiltered address traces. The first method, bytesort, is a lossless compression method that achieves high compression ratios on cache-filtered address traces. The second method is a lossy one, based on the concept of phase. We have combined these two methods in a trace compressor called ATC. Our experimental results show that ATC gives high compression ratio while keeping the memory-locality characteristics of the original trace
Real-Time Lossless Compression of SoC Trace Data
Nowadays, with the increasing complexity of System-on-Chip (SoC), traditional debugging approaches are not enough in multi-core architecture systems. Hardware tracing becomes necessary for performance analysis in these systems. The problem is that the size of collected trace data through hardware-based tracing techniques is usually extremely large due to the increasing complexity of System-on-Chips. Hence on-chip trace compression performed in hardware is needed to reduce the amount of transferred or stored data. In this dissertation, the feasibility of different types of lossless data compression algorithms in hardware implementation are investigated and examined. A lossless data compression algorithm LZ77 is selected, analyzed, and optimized to Nexus traces data. In order to meet the hardware cost and compression performances requirements for the real-time compression, an optimized LZ77 compression algorithm is proposed based on the characteristics of Nexus trace data. This thesis presents a hardware implementation of LZ77 encoder described in Very High Speed Integrated Circuit Hardware Description Language (VHDL). Test results demonstrate that the compression speed can achieve16 bits/clock cycle and the average compression ratio is 1.35 for the minimal hardware cost case, which is a suitable trade-off between the hardware cost and the compression performances effectively
Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP
With ever-increasing volumes of scientific data produced by HPC applications,
significantly reducing data size is critical because of limited capacity of
storage space and potential bottlenecks on I/O or networks in writing/reading
or transferring data. SZ and ZFP are the two leading lossy compressors
available to compress scientific data sets. However, their performance is not
consistent across different data sets and across different fields of some data
sets: for some fields SZ provides better compression performance, while other
fields are better compressed with ZFP. This situation raises the need for an
automatic online (during compression) selection between SZ and ZFP, with a
minimal overhead. In this paper, the automatic selection optimizes the
rate-distortion, an important statistical quality metric based on the
signal-to-noise ratio. To optimize for rate-distortion, we investigate the
principles of SZ and ZFP. We then propose an efficient online, low-overhead
selection algorithm that predicts the compression quality accurately for two
compressors in early processing stages and selects the best-fit compressor for
each data field. We implement the selection algorithm into an open-source
library, and we evaluate the effectiveness of our proposed solution against
plain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation results
on three data sets representing about 100 fields show that our selection
algorithm improves the compression ratio up to 70% with the same level of data
distortion because of very accurate selection (around 99%) of the best-fit
compressor, with little overhead (less than 7% in the experiments).Comment: 14 pages, 9 figures, first revisio
Lossless and near-lossless source coding for multiple access networks
A multiple access source code (MASC) is a source code designed for the following network configuration: a pair of correlated information sequences {X-i}(i=1)(infinity), and {Y-i}(i=1)(infinity) is drawn independent and identically distributed (i.i.d.) according to joint probability mass function (p.m.f.) p(x, y); the encoder for each source operates without knowledge of the other source; the decoder jointly decodes the encoded bit streams from both sources. The work of Slepian and Wolf describes all rates achievable by MASCs of infinite coding dimension (n --> infinity) and asymptotically negligible error probabilities (P-e((n)) --> 0). In this paper, we consider the properties of optimal instantaneous MASCs with finite coding dimension (n 0) performance. The interest in near-lossless codes is inspired by the discontinuity in the limiting rate region at P-e((n)) = 0 and the resulting performance benefits achievable by using near-lossless MASCs as entropy codes within lossy MASCs. Our central results include generalizations of Huffman and arithmetic codes to the MASC framework for arbitrary p(x, y), n, and P-e((n)) and polynomial-time design algorithms that approximate these optimal solutions
Optimality in Quantum Data Compression using Dynamical Entropy
In this article we study lossless compression of strings of pure quantum
states of indeterminate-length quantum codes which were introduced by
Schumacher and Westmoreland. Past work has assumed that the strings of quantum
data are prepared to be encoded in an independent and identically distributed
way. We introduce the notion of quantum stochastic ensembles, allowing us to
consider strings of quantum states prepared in a more general way. For any
identically distributed quantum stochastic ensemble we define an associated
quantum Markov chain and prove that the optimal average codeword length via
lossless coding is equal to the quantum dynamical entropy of the associated
quantum Markov chain
JPEG2000 Image Compression on Solar EUV Images
For future solar missions as well as ground-based telescopes, efficient ways
to return and process data have become increasingly important. Solar Orbiter,
e.g., which is the next ESA/NASA mission to explore the Sun and the
heliosphere, is a deep-space mission, which implies a limited telemetry rate
that makes efficient onboard data compression a necessity to achieve the
mission science goals. Missions like the Solar Dynamics Observatory (SDO) and
future ground-based telescopes such as the Daniel K. Inouye Solar Telescope, on
the other hand, face the challenge of making petabyte-sized solar data archives
accessible to the solar community. New image compression standards address
these challenges by implementing efficient and flexible compression algorithms
that can be tailored to user requirements. We analyse solar images from the
Atmospheric Imaging Assembly (AIA) instrument onboard SDO to study the effect
of lossy JPEG2000 (from the Joint Photographic Experts Group 2000) image
compression at different bit rates. To assess the quality of compressed images,
we use the mean structural similarity (MSSIM) index as well as the widely used
peak signal-to-noise ratio (PSNR) as metrics and compare the two in the context
of solar EUV images. In addition, we perform tests to validate the scientific
use of the lossily compressed images by analysing examples of an on-disk and
off-limb coronal-loop oscillation time-series observed by AIA/SDO.Comment: 25 pages, published in Solar Physic
Entropy Encoding, Hilbert Space and Karhunen-Loeve Transforms
By introducing Hilbert space and operators, we show how probabilities,
approximations and entropy encoding from signal and image processing allow
precise formulas and quantitative estimates. Our main results yield orthogonal
bases which optimize distinct measures of data encoding.Comment: 25 pages, 1 figur
Compression of spectral meteorological imagery
Data compression is essential to current low-earth-orbit spectral sensors with global coverage, e.g., meteorological sensors. Such sensors routinely produce in excess of 30 Gb of data per orbit (over 4 Mb/s for about 110 min) while typically limited to less than 10 Gb of downlink capacity per orbit (15 minutes at 10 Mb/s). Astro-Space Division develops spaceborne compression systems for compression ratios from as little as three to as much as twenty-to-one for high-fidelity reconstructions. Current hardware production and development at Astro-Space Division focuses on discrete cosine transform (DCT) systems implemented with the GE PFFT chip, a 32x32 2D-DCT engine. Spectral relations in the data are exploited through block mean extraction followed by orthonormal transformation. The transformation produces blocks with spatial correlation that are suitable for further compression with any block-oriented spatial compression system, e.g., Astro-Space Division's Laplacian modeler and analytic encoder of DCT coefficients
- …