Transform Based And Search Aware Text Compression Schemes And Compressed Domain Text Retrieval

In recent times, we have witnessed an unprecedented growth of textual information via the Internet, digital libraries and archival text in many applications. While a good fraction of this information is of transient interest, useful information of archival value will continue to accumulate. We need ways to manage, organize and transport this data from one point to the other on data communications links with limited bandwidth. We must also have means to speedily find the information we need from this huge mass of data. Sometimes, a single site may also contain large collections of data such as a library database, thereby requiring an efficient search mechanism even to search within the local data. To facilitate the information retrieval, an emerging ad hoc standard for uncompressed text is XML which preprocesses the text by putting additional user defined metadata such as DTD or hyperlinks to enable searching with better efficiency and effectiveness. This increases the file size considerably, underscoring the importance of applying text compression. On account of efficiency (in terms of both space and time), there is a need to keep the data in compressed form for as much as possible. Text compression is concerned with techniques for representing the digital text data in alternate representations that takes less space. Not only does it help conserve the storage space for archival and online data, it also helps system performance by requiring less number of secondary storage (disk or CD Rom) accesses and improves the network transmission bandwidth utilization by reducing the transmission time. Unlike static images or video, there is no international standard for text compression, although compressed formats like .zip, .gz, .Z files are increasingly being used. In general, data compression methods are classified as lossless or lossy. Lossless compression allows the original data to be recovered exactly. Although used primarily for text data, lossless compression algorithms are useful in special classes of images such as medical imaging, finger print data, astronomical images and data bases containing mostly vital numerical data, tables and text information. Many lossy algorithms use lossless methods at the final stage of the encoding stage underscoring the importance of lossless methods for both lossy and lossless compression applications. In order to be able to effectively utilize the full potential of compression techniques for the future retrieval systems, we need efficient information retrieval in the compressed domain. This means that techniques must be developed to search the compressed text without decompression or only with partial decompression independent of whether the search is done on the text or on some inversion table corresponding to a set of key words for the text. In this dissertation, we make the following contributions: (1) Star family compression algorithms: We have proposed an approach to develop a reversible transformation that can be applied to a source text that improves existing algorithm\u27s ability to compress. We use a static dictionary to convert the English words into predefined symbol sequences. These transformed sequences create additional context information that is superior to the original text. Thus we achieve some compression at the preprocessing stage. We have a series of transforms which improve the performance. Star transform requires a static dictionary for a certain size. To avoid the considerable complexity of conversion, we employ the ternary tree data structure that efficiently converts the words in the text to the words in the star dictionary in linear time. (2) Exact and approximate pattern matching in Burrows-Wheeler transformed (BWT) files: We proposed a method to extract the useful context information in linear time from the BWT transformed text. The auxiliary arrays obtained from BWT inverse transform brings logarithm search time. Meanwhile, approximate pattern matching can be performed based on the results of exact pattern matching to extract the possible candidate for the approximate pattern matching. Then fast verifying algorithm can be applied to those candidates which could be just small parts of the original text. We present algorithms for both k-mismatch and k-approximate pattern matching in BWT compressed text. A typical compression system based on BWT has Move-to-Front and Huffman coding stages after the transformation. We propose a novel approach to replace the Move-to-Front stage in order to extend compressed domain search capability all the way to the entropy coding stage. A modification to the Move-to-Front makes it possible to randomly access any part of the compressed text without referring to the part before the access point. (3) Modified LZW algorithm that allows random access and partial decoding for the compressed text retrieval: Although many compression algorithms provide good compression ratio and/or time complexity, LZW is the first one studied for the compressed pattern matching because of its simplicity and efficiency. Modifications on LZW algorithm provide the extra advantage for fast random access and partial decoding ability that is especially useful for text retrieval systems. Based on this algorithm, we can provide a dynamic hierarchical semantic structure for the text, so that the text search can be performed on the expected level of granularity. For example, user can choose to retrieve a single line, a paragraph, or a file, etc. that contains the keywords. More importantly, we will show that parallel encoding and decoding algorithm is trivial with the modified LZW. Both encoding and decoding can be performed with multiple processors easily and encoding and decoding process are independent with respect to the number of processors

AN EXTENDED GREEN-SASAO HIERARCHY OF CANONICAL TERNARY GALOIS FORMS AND UNIVERSAL LOGIC MODULES

A new extended Green-Sasao hierarchy of families and forms with a new sub-family for many-valued Reed-Muller logic is introduced. Recently, two families of binary canonical Reed-Muller forms, called Inclusive Forms (IFs) and Generalized Inclusive Forms (GIFs) have been proposed, where the second family was the first to include all minimum Exclusive Sum-Of-Products (ESOPs). In this paper, we propose, analogously to the binary case, two general families of canonical ternary Reed-Muller forms, called Ternary Inclusive Forms (TIFs) and their generalization of Ternary Generalized Inclusive Forms (TGIFs), where the second family includes minimum Galois Field Sum-Of-Products (GFSOPs) over ternary Galois field GF(3). One of the basic motivations in this work is the application of these TIFs and TGIFs to find the minimum GFSOP for many-valued input-output functions within logic synthesis, where a GFSOP minimizer based on IF polarity can be used to minimize the many-valued GFSOP expression for any given function. The realization of the presented S/D trees using Universal Logic Modules (ULMs) is also introduced, whereULMs are complete systems that can implement all possible logic functions utilizing the corresponding S/D expansions of many-valuedShannon and Davio spectral transforms.   

Definability of linear equation systems over groups and rings

Motivated by the quest for a logic for PTIME and recent insights that the descriptive complexity of problems from linear algebra is a crucial aspect of this problem, we study the solvability of linear equation systems over finite groups and rings from the viewpoint of logical (inter-)definability. All problems that we consider are decidable in polynomial time, but not expressible in fixed-point logic with counting. They also provide natural candidates for a separation of polynomial time from rank logics, which extend fixed-point logics by operators for determining the rank of definable matrices and which are sufficient for solvability problems over fields. Based on the structure theory of finite rings, we establish logical reductions among various solvability problems. Our results indicate that all solvability problems for linear equation systems that separate fixed-point logic with counting from PTIME can be reduced to solvability over commutative rings. Moreover, we prove closure properties for classes of queries that reduce to solvability over rings, which provides normal forms for logics extended with solvability operators. We conclude by studying the extent to which fixed-point logic with counting can express problems in linear algebra over finite commutative rings, generalising known results on the logical definability of linear-algebraic problems over finite fields

Generating functions for generating trees

Certain families of combinatorial objects admit recursive descriptions in terms of generating trees: each node of the tree corresponds to an object, and the branch leading to the node encodes the choices made in the construction of the object. Generating trees lead to a fast computation of enumeration sequences (sometimes, to explicit formulae as well) and provide efficient random generation algorithms. We investigate the links between the structural properties of the rewriting rules defining such trees and the rationality, algebraicity, or transcendence of the corresponding generating function.Comment: This article corresponds, up to minor typo corrections, to the article submitted to Discrete Mathematics (Elsevier) in Nov. 1999, and published in its vol. 246(1-3), March 2002, pp. 29-5

Enhancement of Exon Regions Recognition in Gene Sequences Using a Radix -4 Multi-valued Logic with DSP Approach

Numerous levels of concepts perform logical designand logical representations in an efficient manner. In typical and quantum theories of computation, Binary logic and Boolean algebra occupies an imperative place. But they havethe limitation of representing signals or sequences by using either binary ‘1’ or ‘0’. This has major drawbacks that the neutralities or any intermediate values are ignored which are essential in most of the applications. Because of the occurrence of such situations it is the need of the hour to look into other alternative logics in order to fulfill the necessities of the user in their respective applications. The binary logic can be replaced by Multi-Valued Logic (MVL), which grabs the positions of the major applications because of the ability to provide representation by using more than two values.As most of the significant applications are based on the logical sequences, the multi-valued logic shines because of its thriving feature. Genomic signal processing, a novel research area in bioinformatics,is one of the foremost applications which involve the operations of logical sequences. It is concerned with the digital signal representations and analysis of genomic data.Determination of the coding region in DNA sequence is one of the genomic operations.This leads to the identification of the characteristics of the gene which in turn finds out an individual’s behavior. In order to extract the coding regions on the basis of logical sequences a number of techniques have been proposed by researchers. But most of the works utilized binary logic, which lead to the problem of losing some of the coding regions and incorrectly recognizing non-coding regions as the coding regions. Hereby,we are proposing an approach for recognizing the exon regions from a gene sequence based on the multi-valued logic. In this approach, we have utilized fourlevel logical system, termed as quaternary logic for the representation of gene sequences and so that we recognize theexon regions from the DNA sequence