Combining the Burrows-Wheeler Transform and RCM-LDGM Codes for the Transmission of Sources with Memory at High Spectral Efficiencies

In this paper, we look at the problem of implementing high-throughput Joint SourceChannel (JSC) coding schemes for the transmission of binary sources with memory over AWGN channels. The sources are modeled either by a Markov chain (MC) or a hidden Markov model (HMM). We propose a coding scheme based on the Burrows-Wheeler Transform (BWT) and the parallel concatenation of Rate-Compatible Modulation and Low-Density Generator Matrix (RCM-LDGM) codes. The proposed scheme uses the BWT to convert the original source with memory into a set of independent non-uniform Discrete Memoryless (DMS) binary sources, which are then separately encoded, with optimal rates, using RCM-LDGM codes

Source and channel coding using Fountain codes

The invention of Fountain codes is a major advance in the field of error correcting codes. The goal of this work is to study and develop algorithms for source and channel coding using a family of Fountain codes known as Raptor codes. From an asymptotic point of view, the best currently known sum-product decoding algorithm for non binary alphabets has a high complexity that limits its use in practice. For binary channels, sum-product decoding algorithms have been extensively studied and are known to perform well. In the first part of this work, we develop a decoding algorithm for binary codes on non-binary channels based on a combination of sum-product and maximum-likelihood decoding. We apply this algorithm to Raptor codes on both symmetric and non-symmetric channels. Our algorithm shows the best performance in terms of complexity and error rate per symbol for blocks of finite length for symmetric channels. Then, we examine the performance of Raptor codes under sum-product decoding when the transmission is taking place on piecewise stationary memoryless channels and on channels with memory corrupted by noise. We develop algorithms for joint estimation and detection while simultaneously employing expectation maximization to estimate the noise, and sum-product algorithm to correct errors. We also develop a hard decision algorithm for Raptor codes on piecewise stationary memoryless channels. Finally, we generalize our joint LT estimation-decoding algorithms for Markov-modulated channels. In the third part of this work, we develop compression algorithms using Raptor codes. More specifically we introduce a lossless text compression algorithm, obtaining in this way competitive results compared to the existing classical approaches. Moreover, we propose distributed source coding algorithms based on the paradigm proposed by Slepian and Wolf

Intra-Key-Frame Coding and Side Information Generation Schemes in Distributed Video Coding

In this thesis investigation has been made to propose improved schemes for intra-key-frame coding and side information (SI) generation in a distributed video coding (DVC) framework. From the DVC developments in last few years it has been observed that schemes put more thrust on intra-frame coding and better quality side information (SI) generation. In fact both are interrelated as SI generation is dependent on decoded key frame quality. Hence superior quality key frames generated through intra-key frame coding will in turn are utilized to generate good quality SI frames. As a result, DVC needs less number of parity bits to reconstruct the WZ frames at the decoder. Keeping this in mind, we have proposed two schemes for intra-key frame coding namely, (a) Borrows Wheeler Transform based H.264/AVC (Intra) intra-frame coding (BWT-H.264/AVC(Intra)) (b) Dictionary based H.264/AVC (Intra) intra-frame coding using orthogonal matching pursuit (DBOMP-H.264/AVC (Intra)) BWT-H.264/AVC (Intra) scheme is a modified version of H.264/AVC (Intra) scheme where a regularized bit stream is generated prior to compression. This scheme results in higher compression efficiency as well as high quality decoded key frames. DBOMP-H.264/AVC (Intra) scheme is based on an adaptive dictionary and H.264/AVC (Intra) intra-frame coding. The traditional transform is replaced with a dictionary trained with K-singular value decomposition (K-SVD) algorithm. The dictionary elements are coded using orthogonal matching pursuit (OMP). Further, two side information generation schemes have been suggested namely, (a) Multilayer Perceptron based side information generation (MLP - SI) (b) Multivariable support vector regression based side information generation (MSVR-SI) MLP-SI scheme utilizes a multilayer perceptron (MLP) to estimate SI frames from the decoded key frames block-by-block. The network is trained offline using training patterns from different frames collected from standard video sequences. MSVR-SI scheme uses an optimized multi variable support vector regression (M-SVR) to generate SI frames from decoded key frames block-by-block. Like MLP, the training for M-SVR is made offline with known training patterns apriori. Both intra-key-frame coding and SI generation schemes are embedded in the Stanford based DVC architecture and studied individually to compare performances with their competitive schemes. Visual as well as quantitative evaluations have been made to show the efficacy of the schemes. To exploit the usefulness of intra-frame coding schemes in SI generation, four hybrid schemes have been formulated by combining the aforesaid suggested schemes as follows: (a) BWT-MLP scheme that uses BWT-H.264/AVC (Intra) intra-frame coding scheme and MLP-SI side information generation scheme. (b) BWT-MSVR scheme, where we utilize BWT-H.264/AVC (Intra) for intra-frame coding followed by MSVR-SI based side information generation. (c) DBOMP-MLP scheme is an outcome of putting DBOMP-H.264/AVC (Intra) intra-frame coding and MLP-SI side information generation schemes. (d) DBOMP-MSVR scheme deals with DBOMP-H.264/AVC (Intra) intra-frame coding and MSVR-SI side information generation together. The hybrid schemes are also incorporated into the Stanford based DVC architecture and simulation has been carried out on standard video sequences. The performance analysis with respect to overall rate distortion, number requests per SI frame, temporal evaluation, and decoding time requirement has been made to derive an overall conclusion

When Machine Learning Meets Information Theory: Some Practical Applications to Data Storage

Machine learning and information theory are closely inter-related areas. In this dissertation, we explore topics in their intersection with some practical applications to data storage. Firstly, we explore how machine learning techniques can be used to improve data reliability in non-volatile memories (NVMs). NVMs, such as flash memories, store large volumes of data. However, as devices scale down towards small feature sizes, they suffer from various kinds of noise and disturbances, thus significantly reducing their reliability. This dissertation explores machine learning techniques to design decoders that make use of natural redundancy (NR) in data for error correction. By NR, we mean redundancy inherent in data, which is not added artificially for error correction. This work studies two different schemes for NR-based error-correcting decoders. In the first scheme, the NR-based decoding algorithm is aware of the data representation scheme (e.g., compression, mapping of symbols to bits, meta-data, etc.), and uses that information for error correction. In the second scenario, the NR-decoder is oblivious of the representation scheme and uses deep neural networks (DNNs) to recognize the file type as well as perform soft decoding on it based on NR. In both cases, these NR-based decoders can be combined with traditional error correction codes (ECCs) to substantially improve their performance. Secondly, we use concepts from ECCs for designing robust DNNs in hardware. Non-volatile memory devices like memristors and phase-change memories are used to store the weights of hardware implemented DNNs. Errors and faults in these devices (e.g., random noise, stuck-at faults, cell-level drifting etc.) might degrade the performance of such DNNs in hardware. We use concepts from analog error-correcting codes to protect the weights of noisy neural networks and to design robust neural networks in hardware. To summarize, this dissertation explores two important directions in the intersection of information theory and machine learning. We explore how machine learning techniques can be useful in improving the performance of ECCs. Conversely, we show how information-theoretic concepts can be used to design robust neural networks in hardware

Modified belief propagation decoders applied to non-CSS QLDGM codes.

Quantum technology is becoming increasingly popular, and big companies are starting to invest huge amounts of money to ensure they do not get left behind in this technological race. Presently, qubits and operational quantum channels may be thought of as far-fetched ideas, but in the future, quantum computing will be of critical importance. In this project, it is provided a concise overview of the basics of coding theory and how they can be used in the design of quantum computers. Specifically, Low Density Parity Check (LDPC) codes are focused, as they can be integrated within the stabilizer construction to build effective quantum codes. Following this, it is introduced the specifics of the quantum paradigm and present the most common family of quantum codes: stabilizer codes. Finally, it is explained the codes that have been used in this project, discussing what type of code they are and how they are designed. In this last section, it is also presented the ultimate goal of the project: using modified belief propagation decoders that had previously been tested for QLDPCs, for the proposed non-CSS QLDGM codes of this project

Low-complexity approaches to distributed data dissemination

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 145-153).In this thesis we consider practical ways of disseminating information from multiple senders to multiple receivers in an optimal or provably close-to-optimal fashion. The basis for our discussion of optimal transmission of information is mostly information theoretic - but the methods that we apply to do so in a low-complexity fashion draw from a number of different engineering disciplines. The three canonical multiple-input, multiple-output problems we focus our attention upon are: * The Slepian-Wolf problem where multiple correlated sources must be distributedly compressed and recovered with a common receiver. * The discrete memoryless multiple access problem where multiple senders communicate across a common channel to a single receiver. * The deterministic broadcast channel problem where multiple messages are sent from a common sender to multiple receivers through a deterministic medium. Chapter 1 serves as an introduction and provides models, definitions, and a discussion of barriers between theory and practice for the three canonical data dissemination problems we will discuss. Here we also discuss how these three problems are all in different senses 'dual' to each other, and use this as a motivating force to attack them with unifying themes.(cont.) Chapter 2 discusses the Slepian-Wolf problem of distributed near-lossless compression of correlated sources. Here we consider embedding any achievable rate in an M-source problem to a corner point in a 2M - 1-source problem. This allows us to employ practical iterative decoding techniques and achieve rates near the boundary with legitimate empirical performance. Both synthetic data and real correlated data from sensors at the International Space Station are used to successfully test our approach. Chapter 3 generalizes the investigation of practical and provably good decoding algorithms for multiterminal systems to the case where the statistical distribution of the memoryless system is unknown. It has been well-established in the theoretical literature that such 'universal' decoders exist and do not suffer a performance penalty, but their proposed structure is highly nonlinear and therefore believed to be complex. For this reason, most discussion of such decoders has been limited to the realm of ontology and proof of existence. By exploiting recently derived results in other engineering disciplines (i.e. expander graphs, linear programming relaxations, etc), we discuss a code construction and two decoding algorithms that have polynomial complexity and admit provably good performance (exponential error probability decay).(cont.) Because there is no need for a priori statistical knowledge in decoding (which in many settings - for instance a sensor network - might be difficult to repeatedly acquire without significant cost), this approach has very attractive robustness, energy efficiency, and stand-alone practical implications. Finally, Chapter 4 walks away from the multiple-sender, single-receiver setting and steps into the single-sender-multiple receiver setting. We focus our attention here on the deterministic broadcast channel, which is dual to the Slepian-Wolf and multiple access problems in a number of ways - including how the difficulty of practical implementation lies in the encoding rather than decoding. Here we illustrate how again a splitting approach can be applied, and how the same properties from the Slepian-Wolf and multiple access splitting settings remain. We also discuss practical coding strategies for some problems motivated by wireless, and show how by properly 'dualizing' provably good decoding strategies for some channel coding problems, we admit provably good encoding for this setting.by Todd Prentice Coleman.Ph.D

Joint source-channel-network coding in wireless mesh networks with temporal reuse

Technological innovation that empowers tiny low-cost transceivers to operate with a high degree of utilisation efficiency in multihop wireless mesh networks is contributed in this dissertation. Transmission scheduling and joint source-channel-network coding are two of the main aspects that are addressed. This work focuses on integrating recent enhancements such as wireless network coding and temporal reuse into a cross-layer optimisation framework, and to design a joint coding scheme that allows for space-optimal transceiver implementations. Link-assigned transmission schedules with timeslot reuse by multiple links in both the space and time domains are investigated for quasi-stationary multihop wireless mesh networks with both rate and power adaptivity. Specifically, predefined cross-layer optimised schedules with proportionally fair end-to-end flow rates and network coding capability are constructed for networks operating under the physical interference model with single-path minimum hop routing. Extending transmission rights in a link-assigned schedule allows for network coding and temporal reuse, which increases timeslot usage efficiency when a scheduled link experiences packet depletion. The schedules that suffer from packet depletion are characterised and a generic temporal reuse-aware achievable rate region is derived. Extensive computational experiments show improved schedule capacity, quality of service, power efficiency and benefit from opportunistic bidirectional network coding accrued with schedules optimised in the proposed temporal reuse-aware convex capacity region. The application of joint source-channel coding, based on fountain codes, in the broadcast timeslot of wireless two-way network coding is also investigated. A computationally efficient subroutine is contributed to the implementation of the fountain compressor, and an error analysis is done. Motivated to develop a true joint source-channel-network code that compresses, adds robustness against channel noise and network codes two packets on a single bipartite graph and iteratively decodes the intended packet on the same Tanner graph, an adaptation of the fountain compressor is presented. The proposed code is shown to outperform a separated joint source-channel and network code in high source entropy and high channel noise regions, in anticipated support of dense networks that employ intelligent signalling. AFRIKAANS : Tegnologiese innovasie wat klein lae-koste kommunikasie toestelle bemagtig om met ’n hoë mate van benuttings doeltreffendheid te werk word bygedra in hierdie proefskrif. Transmissie-skedulering en gesamentlike bron-kanaal-netwerk kodering is twee van die belangrike aspekte wat aangespreek word. Hierdie werk fokus op die integrasie van onlangse verbeteringe soos draadlose netwerk kodering en temporêre herwinning in ’n tussen-laag optimaliserings raamwerk, en om ’n gesamentlike kodering skema te ontwerp wat voorsiening maak vir spasie-optimale toestel implementerings. Skakel-toegekende transmissie skedules met tydgleuf herwinning deur veelvuldige skakels in beide die ruimte en tyd domeine word ondersoek vir kwasi-stilstaande, veelvuldige-sprong draadlose rooster netwerke met beide transmissie-spoed en krag aanpassings. Om spesifiek te wees, word vooraf bepaalde tussen-laag geoptimiseerde skedules met verhoudings-regverdige punt-tot-punt vloei tempo’s en netwerk kodering vermoë saamgestel vir netwerke wat bedryf word onder die fisiese inmengings-model met enkel-pad minimale sprong roetering. Die uitbreiding van transmissie-regte in ’n skakel-toegekende skedule maak voorsiening vir netwerk kodering en temporêre herwinning, wat tydgleuf gebruiks-doeltreffendheid verhoog wanneer ’n geskeduleerde skakel pakkie-uitputting ervaar. Die skedules wat ly aan pakkie-uitputting word gekenmerk en ’n generiese temporêre herwinnings-bewuste haalbare transmissie-spoed gebied word afgelei. Omvattende berekenings-eksperimente toon verbeterde skedulerings kapasiteit, diensgehalte, krag doeltreffendheid asook verbeterde voordeel wat getrek word uit opportunistiese tweerigting netwerk kodering met die skedules wat geoptimiseer word in die temporêre herwinnings-bewuste konvekse transmissie-spoed gebied. Die toepassing van gesamentlike bron-kanaal kodering, gebaseer op fontein kodes, in die uitsaai-tydgleuf van draadlose tweerigting netwerk kodering word ook ondersoek. ’n Berekenings-effektiewe subroetine word bygedra in die implementering van die fontein kompressor, en ’n foutanalise word gedoen. Gemotiveer om ’n ware gesamentlike bron-kanaal-netwerk kode te ontwikkel, wat robuustheid byvoeg teen kanaal geraas en twee pakkies netwerk kodeer op ’n enkele bipartiete grafiek en die beoogde pakkie iteratief dekodeer op dieselfde Tanner grafiek, word ’n aanpassing van die fontein kompressor aangebied. Dit word getoon dat die voorgestelde kode ’n geskeide gesamentlike bron-kanaal en netwerk kode in hoë bron-entropie en ho¨e kanaal-geraas gebiede oortref in verwagte ondersteuning van digte netwerke wat van intelligente sein-metodes gebruik maak.Dissertation (MEng)--University of Pretoria, 2011.Electrical, Electronic and Computer Engineeringunrestricte

Novel linear and nonlinear optical signal processing for ultra-high bandwidth communications

The thesis is articulated around the theme of ultra-wide bandwidth single channel signals. It focuses on the two main topics of transmission and processing of information by techniques compatible with high baudrates. The processing schemes introduced combine new linear and nonlinear optical platforms such as Fourier-domain programmable optical processors and chalcogenide chip waveguides, as well as the concept of neural network. Transmission of data is considered in the context of medium distance links of Optical Time Division Multiplexed (OTDM) data subject to environmental fluctuations. We experimentally demonstrate simultaneous compensation of differential group delay and multiple orders of dispersion at symbol rates of 640 Gbaud and 1.28 Tbaud. Signal processing at high bandwidth is envisaged both in the case of elementary post-transmission analog error mitigation and in the broader field of optical computing for high level operations (“optical processor”). A key innovation is the introduction of a novel four-wave mixing scheme implementing a dot-product operation between wavelength multiplexed channels. In particular, it is demonstrated for low-latency hash-key based all-optical error detection in links encoded with advanced modulation formats. Finally, the work presents groundbreaking concepts for compact implementation of an optical neural network as a programmable multi-purpose processor. The experimental architecture can implement neural networks with several nodes on a single optical nonlinear transfer function implementing functions such as analog-to-digital conversion. The particularity of the thesis is the new approaches to optical signal processing that potentially enable high level operations using simple optical hardware and limited cascading of components

Ecology and Physiology of Aerobic Aromatic Catabolism in Roseobacters

Roseobacters are an abundant and trophically versatile lineage of marine bacteria that are especially dominant in coastal salt marshes. Central to understanding of how members of the Roseobacter clade contribute to biogeochemical cycling in the world’s oceans is how these bacteria physiologically respond to mixtures of usable growth substrates present in their environmental niches. A prior study provided evidence that bacterioplankton transcripts most closely related to the Roseobacter Sagitulla stellata E-37 are among the most abundant in coastal systems for biogeochemically significant processes of N, P, and S cycling. Thus, this strain was used throughout this dissertation as an environmentally relevant model. Most roseobacter isolates contain multiple aerobic ring-cleaving pathways for the degradation of aromatic compounds, yet it was unknown whether cross-regulation occurred between different parallel pathways. In S. stellata E-37, benzoate is catabolized through the aerobic benzoyl-CoA oxidation (box) pathway, and p-hydroxybenzoate proceeds through the protocatechuate (pca) branch of the beta-ketoadipate pathway. Temporal analysis of S. stellata E-37 was performed in a mixed-substrate environment containing both benzoate and p-hydroxybenzoate and showed that both substrates were simultaneously catabolized at the same rate. Computational and further experimental studies suggest this phenotype appears unique to roseobacters and is anticipated to confer an ecological growth advantage. Additional studies focusing on the environmental relevance of the box pathway revealed it is an abundant yet taxonomically constrained pathway to Betaproteobacteria and Roseobacter. Finally, a study was performed to examine the response of S. stellata E-37 to a more chemically complex environment. In coastal salt marshes where roseobacters are dominant, Spartina alterniflora (cordgrass) is the most abundant primary producer and Phaeosphaeria spartinicola is an abundant and active lignocellulose-degrading fungus. Cordgrass degradation products (mediated by P. spartinicola), representing a salt marsh-like dissolved organic carbon (DOC) mixture, were provided to S. stellata E-37 to link metabolic processes to mineralization of cordgrass. Multiple ring-cleaving and carbohydrate metabolism genes were induced during growth on the DOC mixture. Furthermore, the overexpression of central metabolism pathways such as the tricarboxylic acid (TCA) cycle, ATPase, and NADH dehydrogenase suggest a hyperactive trophic strategy for this metabolically versatile roseobacter