102,841 research outputs found
Value-Compressed Sparse Column (VCSC): Sparse Matrix Storage for Redundant Data
Compressed Sparse Column (CSC) and Coordinate (COO) are popular compression
formats for sparse matrices. However, both CSC and COO are general purpose and
cannot take advantage of any of the properties of the data other than sparsity,
such as data redundancy. Highly redundant sparse data is common in many machine
learning applications, such as genomics, and is often too large for in-core
computation using conventional sparse storage formats. In this paper, we
present two extensions to CSC: (1) Value-Compressed Sparse Column (VCSC) and
(2) Index- and Value-Compressed Sparse Column (IVCSC). VCSC takes advantage of
high redundancy within a column to further compress data up to 3-fold over COO
and 2.25-fold over CSC, without significant negative impact to performance
characteristics. IVCSC extends VCSC by compressing index arrays through delta
encoding and byte-packing, achieving a 10-fold decrease in memory usage over
COO and 7.5-fold decrease over CSC. Our benchmarks on simulated and real data
show that VCSC and IVCSC can be read in compressed form with little added
computational cost. These two novel compression formats offer a broadly useful
solution to encoding and reading redundant sparse data
Grammar Boosting: A New Technique for Proving Lower Bounds for Computation over Compressed Data
Grammar compression is a general compression framework in which a string
of length is represented as a context-free grammar of size whose
language contains only . In this paper, we focus on studying the limitations
of algorithms and data structures operating on strings in grammar-compressed
form. Previous work focused on proving lower bounds for grammars constructed
using algorithms that achieve the approximation ratio
. Unfortunately, for the majority of
grammar compressors, is either unknown or satisfies
. In their seminal paper, Charikar et al. [IEEE
Trans. Inf. Theory 2005] studied seven popular grammar compression algorithms:
RePair, Greedy, LongestMatch, Sequential, Bisection, LZ78, and
-Balanced. Only one of them (-Balanced) is known to achieve
.
We develop the first technique for proving lower bounds for data structures
and algorithms on grammars that is fully general and does not depend on the
approximation ratio of the used grammar compressor. Using this
technique, we first prove that time is required
for random access on RePair, Greedy, LongestMatch, Sequential, and Bisection,
while time is required for random access to LZ78. All
these lower bounds hold within space and
match the existing upper bounds. We also generalize this technique to prove
several conditional lower bounds for compressed computation. For example, we
prove that unless the Combinatorial -Clique Conjecture fails, there is no
combinatorial algorithm for CFG parsing on Bisection (for which it holds
) that runs in time for all constants and . Previously,
this was known only for
Computing NP-Hard Repetitiveness Measures via MAX-SAT
Repetitiveness measures reveal profound characteristics of datasets, and give rise to compressed data structures and algorithms working in compressed space. Alas, the computation of some of these measures is NP-hard, and straight-forward computation is infeasible for datasets of even small sizes. Three such measures are the smallest size of a string attractor, the smallest size of a bidirectional macro scheme, and the smallest size of a straight-line program. While a vast variety of implementations for heuristically computing approximations exist, exact computation of these measures has received little to no attention. In this paper, we present MAX-SAT formulations that provide the first non-trivial implementations for exact computation of smallest string attractors, smallest bidirectional macro schemes, and smallest straight-line programs. Computational experiments show that our implementations work for texts of length up to a few hundred for straight-line programs and bidirectional macro schemes, and texts even over a million for string attractors
CONCISE: Compressed 'n' Composable Integer Set
Bit arrays, or bitmaps, are used to significantly speed up set operations in
several areas, such as data warehousing, information retrieval, and data
mining, to cite a few. However, bitmaps usually use a large storage space, thus
requiring compression. Nevertheless, there is a space-time tradeoff among
compression schemes. The Word Aligned Hybrid (WAH) bitmap compression trades
some space to allow for bitwise operations without first decompressing bitmaps.
WAH has been recognized as the most efficient scheme in terms of computation
time. In this paper we present CONCISE (Compressed 'n' Composable Integer Set),
a new scheme that enjoys significatively better performances than those of WAH.
In particular, when compared to WAH, our algorithm is able to reduce the
required memory up to 50%, by having similar or better performance in terms of
computation time. Further, we show that CONCISE can be efficiently used to
manipulate bitmaps representing sets of integral numbers in lieu of well-known
data structures such as arrays, lists, hashtables, and self-balancing binary
search trees. Extensive experiments over synthetic data show the effectiveness
of our approach.Comment: Preprint submitted to Information Processing Letters, 7 page
Indexes and Computation over Compressed Structured Data (Dagstuhl Seminar 13232)
This report documents the program and the outcomes of Dagstuhl Seminar
13232 "Indexes and Computation over Compressed Structured Data"
SCV-GNN: Sparse Compressed Vector-based Graph Neural Network Aggregation
Graph neural networks (GNNs) have emerged as a powerful tool to process
graph-based data in fields like communication networks, molecular interactions,
chemistry, social networks, and neuroscience. GNNs are characterized by the
ultra-sparse nature of their adjacency matrix that necessitates the development
of dedicated hardware beyond general-purpose sparse matrix multipliers. While
there has been extensive research on designing dedicated hardware accelerators
for GNNs, few have extensively explored the impact of the sparse storage format
on the efficiency of the GNN accelerators. This paper proposes SCV-GNN with the
novel sparse compressed vectors (SCV) format optimized for the aggregation
operation. We use Z-Morton ordering to derive a data-locality-based computation
ordering and partitioning scheme. The paper also presents how the proposed
SCV-GNN is scalable on a vector processing system. Experimental results over
various datasets show that the proposed method achieves a geometric mean
speedup of and over CSC and CSR aggregation
operations, respectively. The proposed method also reduces the memory traffic
by a factor of and over compressed sparse column
(CSC) and compressed sparse row (CSR), respectively. Thus, the proposed novel
aggregation format reduces the latency and memory access for GNN inference
Stochastic Controlled Averaging for Federated Learning with Communication Compression
Communication compression, a technique aiming to reduce the information
volume to be transmitted over the air, has gained great interests in Federated
Learning (FL) for the potential of alleviating its communication overhead.
However, communication compression brings forth new challenges in FL due to the
interplay of compression-incurred information distortion and inherent
characteristics of FL such as partial participation and data heterogeneity.
Despite the recent development, the performance of compressed FL approaches has
not been fully exploited. The existing approaches either cannot accommodate
arbitrary data heterogeneity or partial participation, or require stringent
conditions on compression.
In this paper, we revisit the seminal stochastic controlled averaging method
by proposing an equivalent but more efficient/simplified formulation with
halved uplink communication costs. Building upon this implementation, we
propose two compressed FL algorithms, SCALLION and SCAFCOM, to support unbiased
and biased compression, respectively. Both the proposed methods outperform the
existing compressed FL methods in terms of communication and computation
complexities. Moreover, SCALLION and SCAFCOM accommodates arbitrary data
heterogeneity and do not make any additional assumptions on compression errors.
Experiments show that SCALLION and SCAFCOM can match the performance of
corresponding full-precision FL approaches with substantially reduced uplink
communication, and outperform recent compressed FL methods under the same
communication budget.Comment: 45 pages, 4 figure
Top Tree Compression of Tries
We present a compressed representation of tries based on top tree compression [ICALP 2013] that works on a standard, comparison-based, pointer machine model of computation and supports efficient prefix search queries. Namely, we show how to preprocess a set of strings of total length n over an alphabet of size sigma into a compressed data structure of worst-case optimal size O(n/log_sigma n) that given a pattern string P of length m determines if P is a prefix of one of the strings in time O(min(m log sigma,m + log n)). We show that this query time is in fact optimal regardless of the size of the data structure.
Existing solutions either use Omega(n) space or rely on word RAM techniques, such as tabulation, hashing, address arithmetic, or word-level parallelism, and hence do not work on a pointer machine. Our result is the first solution on a pointer machine that achieves worst-case o(n) space. Along the way, we develop several interesting data structures that work on a pointer machine and are of independent interest. These include an optimal data structures for random access to a grammar-compressed string and an optimal data structure for a variant of the level ancestor problem
- …