Fast compression of transportable Tcl scripts

An information agent is charged with the task of searching a collection of electronic resources for information that is relevant to the user\u27s current needs. These resources are often distributed across a network and can contain tremendous quantities of data. One of the paradigms that has been suggested for allowing efficient access to such resources is transportable agents -- the agent is sent to the machine that maintains the information resource; the agent executes on this remote machine and then returns its results to the local machine. We have implemented a transportable agent system that uses the Tool Command Language (Tcl) as the agent language. Each Tcl script can suspend its execution at an arbitrary point, transport itself to another machine and resume execution on the new machine. The execution state of the script -- which includes the commands that have not been executed -- must be transmitted to the new machine. Although the execution state tends to be small, there will be a large number of agents moving across the network in a large-scale system. Thus it is desirable to compress the execution state as much as possible. Furthermore any compression scheme must be fast so that it does not become a bottleneck between the transportable agent system and the network routines. In this paper we explore several fast compression methods

Compressing computer programs

This paper describes a scheme for compressing programs written in a particular programming language—which can be any language that has a formal lexical and syntactic description—in such a way that they can be reproduced exactly. Only syntactically correct programs can be compressed. The scheme is illustrated on the Pascal language, and compression results are given for a corpus of Pascal programs; but it is by no means restricted to Pascal. In fact, we discuss how a "compressor-generator" program can be constructed that creates a compressor automatically from a formal specification of a programming language, in much the same way as a parser generator creates a syntactic parser from a formal language description

Grammar-Based Representations of Large Sparse Binary Matrices

Large sparse matrices representation is a fundamental problem in big data processing and analysis. In some applications dealing with large sparse matrices, the I/O of these sparse matrices is the bottleneck of the whole system. To reduce the requirement of memory bandwidth in this scenario, it is important to develop alternative compact representations of large sparse matrices, while facilitating, if possible, matrix operations. In this thesis, we propose two grammar-based methods to compactly represent a sparse binary matrix with the capability of random accessing an element in the matrix. The first approach combines dimension coding (proposed by Yang[12]) with one of raster scan or Hilbert scan, where the so-called directionless grammar is applied. With the power of scanning, dimension coding’s capability of representing 1-D sparse signals can be extended to 2-D sparse matrices. This approach inherits the random accessibility of dimension coding. In the second approach, we will introduce a new concept called Context-free Bipartite Grammar (CFBG) and present a framework wherein large sparse binary matrices can be represented by CFBG. Similar to the traditional concept of Context-free Grammar (CFG), a CFBG consists of a set of production rules. Unlike CFGs, however, the right member of each production rule in a CFBG is a labeled bipartite graph with each edge labeled either as a variable or terminal symbol. As the right-hand side of a production rule is an ordered edge set, CFBG is also directionless. Two bipartite grammar transforms, a Sequential D-Neighborhood Pairing Transform (SNPT) and an Iterative Pairing Transform (IPT), are further presented to convert any binary matrix into a CFBG representing it. Experiments show that compared with popular sparse matrix storage methods such as compressed row storage and quadtree, grammar-based sparse binary matrix representations can reduce the storage requirement of sparse matrices significantly (by a factor of as much as 70)

Proceedings of the Third Symposium on Programming Languages and Software Tools : Kääriku, Estonia, August 23-24 1993

http://www.ester.ee/record=b1064507*es