Lower Bounds on the Redundancy of Huffman Codes with Known and Unknown Probabilities

In this paper we provide a method to obtain tight lower bounds on the minimum redundancy achievable by a Huffman code when the probability distribution underlying an alphabet is only partially known. In particular, we address the case where the occurrence probabilities are unknown for some of the symbols in an alphabet. Bounds can be obtained for alphabets of a given size, for alphabets of up to a given size, and for alphabets of arbitrary size. The method operates on a Computer Algebra System, yielding closed-form numbers for all results. Finally, we show the potential of the proposed method to shed some light on the structure of the minimum redundancy achievable by the Huffman code

Comparison Of Sparse Coding And Jpeg Coding Schemes For Blurred Retinal Images.

Overcomplete representations are currently one of the highly researched areas especially in the field of signal processing due to their strong potential to generate sparse representation of signals. Sparse representation implies that given signal can be represented with components that are only rarely significantly active. It has been strongly argued that the mammalian visual system is highly related towards sparse and overcomplete representations. The primary visual cortex has overcomplete responses in representing an input signal which leads to the use of sparse neuronal activity for further processing. This work investigates the sparse coding with an overcomplete basis set representation which is believed to be the strategy employed by the mammalian visual system for efficient coding of natural images. This work analyzes the Sparse Code Learning algorithm in which the given image is represented by means of linear superposition of sparse statistically independent events on a set of overcomplete basis functions. This algorithm trains and adapts the overcomplete basis functions such as to represent any given image in terms of sparse structures. The second part of the work analyzes an inhibition based sparse coding model in which the Gabor based overcomplete representations are used to represent the image. It then applies an iterative inhibition algorithm based on competition between neighboring transform coefficients to select subset of Gabor functions such as to represent the given image with sparse set of coefficients. This work applies the developed models for the image compression applications and tests the achievable levels of compression of it. The research towards these areas so far proves that sparse coding algorithms are inefficient in representing high frequency sharp image features. So this work analyzes the performance of these algorithms only on the natural images which does not have sharp features and compares the compression results with the current industrial standard coding schemes such as JPEG and JPEG 2000. It also models the characteristics of an image falling on the retina after the distortion effects of the eye and then applies the developed algorithms towards these images and tests compression results

Neural Mechanisms for Information Compression by Multiple Alignment, Unification and Search

This article describes how an abstract framework for perception and cognition may be realised in terms of neural mechanisms and neural processing. This framework — called information compression by multiple alignment, unification and search (ICMAUS) — has been developed in previous research as a generalized model of any system for processing information, either natural or artificial. It has a range of applications including the analysis and production of natural language, unsupervised inductive learning, recognition of objects and patterns, probabilistic reasoning, and others. The proposals in this article may be seen as an extension and development of Hebb’s (1949) concept of a ‘cell assembly’. The article describes how the concept of ‘pattern’ in the ICMAUS framework may be mapped onto a version of the cell assembly concept and the way in which neural mechanisms may achieve the effect of ‘multiple alignment’ in the ICMAUS framework. By contrast with the Hebbian concept of a cell assembly, it is proposed here that any one neuron can belong in one assembly and only one assembly. A key feature of present proposals, which is not part of the Hebbian concept, is that any cell assembly may contain ‘references’ or ‘codes’ that serve to identify one or more other cell assemblies. This mechanism allows information to be stored in a compressed form, it provides a robust mechanism by which assemblies may be connected to form hierarchies and other kinds of structure, it means that assemblies can express abstract concepts, and it provides solutions to some of the other problems associated with cell assemblies. Drawing on insights derived from the ICMAUS framework, the article also describes how learning may be achieved with neural mechanisms. This concept of learning is significantly different from the Hebbian concept and appears to provide a better account of what we know about human learning

Study and simulation of low rate video coding schemes

The semiannual report is included. Topics covered include communication, information science, data compression, remote sensing, color mapped images, robust coding scheme for packet video, recursively indexed differential pulse code modulation, image compression technique for use on token ring networks, and joint source/channel coder design

Shape representation and coding of visual objets in multimedia applications — An overview

Emerging multimedia applications have created the need for new functionalities in digital communications. Whereas existing compression standards only deal with the audio-visual scene at a frame level, it is now necessary to handle individual objects separately, thus allowing scalable transmission as well as interactive scene recomposition by the receiver. The future MPEG-4 standard aims at providing compression tools addressing these functionalities. Unlike existing frame-based standards, the corresponding coding schemes need to encode shape information explicitly. This paper reviews existing solutions to the problem of shape representation and coding. Region and contour coding techniques are presented and their performance is discussed, considering coding efficiency and rate-distortion control capability, as well as flexibility to application requirements such as progressive transmission, low-delay coding, and error robustnes

Some basic properties of fix-free codes.

by Chunxuan Ye.Thesis (M.Phil.)--Chinese University of Hong Kong, 2000.Includes bibliographical references (leaves 74-[78]).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Information Theory --- p.1Chapter 1.2 --- Source Coding --- p.2Chapter 1.3 --- Fixed Length Codes and Variable Length Codes --- p.4Chapter 1.4 --- Prefix Codes --- p.5Chapter 1.4.1 --- Kraft Inequality --- p.7Chapter 1.4.2 --- Huffman Coding --- p.9Chapter 2 --- Existence of Fix-Free Codes --- p.13Chapter 2.1 --- Introduction --- p.13Chapter 2.2 --- Previous Results --- p.14Chapter 2.2.1 --- Complete Fix-Free Codes --- p.14Chapter 2.2.2 --- Ahlswede's Results --- p.16Chapter 2.3 --- Two Properties of Fix-Free Codes --- p.17Chapter 2.4 --- A Sufficient Condition --- p.20Chapter 2.5 --- Other Sufficient Conditions --- p.33Chapter 2.6 --- A Necessary Condition --- p.37Chapter 2.7 --- A Necessary and Sufficient Condition --- p.42Chapter 3 --- Redundancy of Optimal Fix-Free Codes --- p.44Chapter 3.1 --- Introduction --- p.44Chapter 3.2 --- An Upper Bound in Terms of q --- p.46Chapter 3.3 --- An Upper Bound in Terms of p1 --- p.48Chapter 3.4 --- An Upper Bound in Terms of pn --- p.51Chapter 4 --- Two Applications of the Probabilistic Method --- p.54Chapter 4.1 --- An Alternative Proof for the Kraft Inequality --- p.54Chapter 4.2 --- A Characteristic Inequality for ´ب1´ة-ended Codes --- p.59Chapter 5 --- Summary and Future Work --- p.69Appendix --- p.71A Length Assignment for Upper Bounding the Redundancy of Fix-Free Codes --- p.71Bibliography --- p.7