The Rényi Redundancy of Generalized Huffman Codes

Huffman's algorithm gives optimal codes, as measured by average codeword length, and the redundancy can be measured as the difference between the average codeword length and Shannon's entropy. If the objective function is replaced by an exponentially weighted average, then a simple modification of Huffman's algorithm gives optimal codes. The redundancy can now be measured as the difference between this new average and A. Renyi's (1961) generalization of Shannon's entropy. By decreasing some of the codeword lengths in a Shannon code, the upper bound on the redundancy given in the standard proof of the noiseless source coding theorem is improved. The lower bound is improved by randomizing between codeword lengths, allowing linear programming techniques to be used on an integer programming problem. These bounds are shown to be asymptotically equal. The results are generalized to the Renyi case and are related to R.G. Gallager's (1978) bound on the redundancy of Huffman codes

Descriptive Complexity Approaches to Inductive Inference

We present a critical review of descriptive complexity approaches to inductive inference. Inductive inference is defined as any process by which a model of the world is formed from observations. The descriptive complexity approach is a formalization of Occam\u27s razor: choose the simplest model consistent with the data. Descriptive complexity as defined by Kolmogorov, Chaitin and Solomonoff is presented as a generalization of Shannon\u27s entropy. We discuss its relationship with randomness and present examples. However, a major result of the theory is negative: descriptive complexity is uncomputable. Rissanen\u27s minimum description length (MDL) principle is presented as a restricted form of the descriptive complexity which avoids the uncomputability problem. We demonstrate the effectiveness of MDL through its application to AR processes. Lastly, we present and discuss LeClerc\u27s application of MDL to the problem of image segmentation

Coding Theorems on a Non-Additive Generalized Entropy of Havrda-Charvat And Tsallis

A new measure Lβα, called average code word length of order α and type β is defined and its relationship with a generalized information measure of order α and type β is discussed. Using Lβα , some coding theorems are proved

A discrete theory of search. II

summary:The first part of this paper has been published in the preceding number of this volume. In this part asymptotic estimates and formulas for the Bayes error are given, mostly under the assumption that the sampling is governed by a random strategy, and relations between random strategies of search and random coding are studied

Capacity and coding in digital communications

+164hlm.;24c

Langattoman viestinnän kanavakapasiteetin optimointi keilanmuodostustekniikalla

Many modern devices rely on wireless Internet connection. This has increased the demand for high performance wireless networks. However, the traditional single-transmitting single-receiving antenna systems cannot provide the desired data rates and latencies in the future. The capacity of wireless communication systems can be increased by using more than one transmitting and receiving antennas. These multiple-input multiple-output (MIMO) systems allow the use of advanced signal precoding methods including beamforming. This method enables transmission of several concurrent data streams by controlling the electromagnetic wave interference. In this thesis, we review various beamforming techniques for single user and multi-user MIMO systems. In the single user case, the well-known eigen-beamforming method is derived, which achieves the optimal channel capacity. In the multi-user MIMO context, we focus on zero-forcing techniques that are based on a zero inter-user interference constraint. Finally, we define a new geometry-based MIMO channel model that is used for beamforming simulations. Results indicate that using beamforming can yield substantial increase in performance compared to the traditional communication systems.Monet nykyajan laitteet vaativat toimiakseen langattoman internet-yhteyden. Lisääntyvä älylaitteiden määrä on kasvattanut tarvetta korkean suorituskyvyn langattomille tietoverkoille. Perinteiset yhden lähetys- ja vastaanottoantennin viestintäsysteemit eivät kuitenkaan pysty tulevaisuudessa tuottamaan tarvittavia datansiirtonopeuksia ja viiveaikoja. Langattomassa viestinnässä kapasiteettia voidaan kasvattaa käyttämällä useita lähetys- ja vastaanottoantenneja. Nämä niin kutsutut MIMO-systeemit mahdollistavat edistyneiden koodaustekniikoiden käyttämisen. Yksi näistä tekniikoista on keilanmuodostus, jonka avulla voidaan lähettää useita datapaketteja samanaikaisesti. Menetelmä perustuu sähkömagneettisten aaltojen superpositioperiaatteen hyödyntämiseen. Tässä diplomityössä tarkastellaan useita keilanmuodostustekniikoita sekä yhden että monen käyttäjän MIMO-systeemeissä. Yhden käyttäjän tapauksessa johdetaan tunnettu menetelmä, joka saavuttaa optimaalisen kanavakapasiteetin. Usean käyttäjän MIMO-systeemien yhteydessä taas keskitytään tekniikoihin, joissa käyttäjien väliset häiriöt minimoidaan. Lopuksi määritellään uusi geometriaperusteinen kanavamalli, jota käytetään keilanmuodostussimulaatioissa. Tulokset osoittavat, että keilanmuodostusta käyttämällä voidaan parantaa merkittävästi langattomien viestintäsysteemien suorituskykyä

Fidelity criteria for visual image transmission over noisy communication channels

Imperial Users onl