A Continuous-Time Recurrent Neural Network for Joint Equalization and Decoding – Analog Hardware Implementation Aspects

Equalization and channel decoding are “traditionally” two cascade processes at the receiver side of a digital transmission. They aim to achieve a reliable and efficient transmission. For high data rates, the energy consumption of their corresponding algorithms is expected to become a limiting factor. For mobile devices with limited battery’s size, the energy consumption, mirrored in the lifetime of the battery, becomes even more crucial. Therefore, an energy-efficient implementation of equalization and decoding algorithms is desirable. The prevailing way is by increasing the energy efficiency of the underlying digital circuits. However, we address here promising alternatives offered by mixed (analog/digital) circuits. We are concerned with modeling joint equalization and decoding as a whole in a continuous-time framework. In doing so, continuous-time recurrent neural networks play an essential role because of their nonlinear characteristic and special suitability for analog very-large-scale integration (VLSI). Based on the proposed model, we show that the superiority of joint equalization and decoding (a well-known fact from the discrete-time case) preserves in analog. Additionally, analog circuit design related aspects such as adaptivity, connectivity and accuracy are discussed and linked to theoretical aspects of recurrent neural networks such as Lyapunov stability and simulated annealing

Efficient soft decoding techniques for reed-solomon codes

The main focus of this thesis is on finding efficient decoding methods for Reed-Solomon (RS) codes, i.e., algorithms with acceptable performance and affordable complexity. Three classes of decoders are considered including sphere decoding, belief propagation decoding and interpolation-based decoding. Originally proposed for finding the exact solution of least-squares problems, sphere decoding (SD) is used along with the most reliable basis (MRB) to design an efficient soft decoding algorithm for RS codes. For an (N, K ) RS code, given the received vector and the lattice of all possible transmitted vectors, we propose to look for only those lattice points that fall within a sphere centered at the received vector and also are valid codewords. To achieve this goal, we use the fact that RS codes are maximum distance separable (MDS). Therefore, we use sphere decoding in order to find tentative solutions consisting of the K most reliable code symbols that fall inside the sphere. The acceptable values for each of these symbols are selected from an ordered set of most probable transmitted symbols. Based on the MDS property, K code symbols of each tentative solution can he used to find the rest of codeword symbols. If the resulting codeword is within the search radius, it is saved as a candidate transmitted codeword. Since we first find the most reliable code symbols and for each of them we use an ordered set of most probable transmitted symbols, candidate codewords are found quickly resulting in reduced complexity. Considerable coding gains are achieved over the traditional hard decision decoders with moderate increase in complexity. Due to their simplicity and good performance when used for decoding low density parity check (LDPC) codes, iterative decoders based on belief propagation (BP) have also been considered for RS codes. However, the parity check matrix of RS codes is very dense resulting in lots of short cycles in the factor graph and consequently preventing the reliability updates (using BP) from converging to a codeword. In this thesis, we propose two BP based decoding methods. In both of them, a low density extended parity check matrix is used because of its lower number of short cycles. In the first method, the cyclic structure of RS codes is taken into account and BP algorithm is applied on different cyclically shifted versions of received reliabilities, capable of detecting different error patterns. This way, some deterministic errors can be avoided. The second method is based on information correction in BP decoding where all possible values are tested for selected bits with low reliabilities. This way, the chance of BP iterations to converge to a codeword is improved significantly. Compared to the existing iterative methods for RS codes, our proposed methods provide a very good trade-off between the performance and the complexity. We also consider interpolation based decoding of RS codes. We specifically focus on Guruswami-Sudan (GS) interpolation decoding algorithm. Using the algebraic structure of RS codes and bivariate interpolation, the GS method has shown improved error correction capability compared to the traditional hard decision decoders. Based on the GS method, a multivariate interpolation decoding method is proposed for decoding interleaved RS (IRS) codes. Using this method all the RS codewords of the interleaved scheme are decoded simultaneously. In the presence of burst errors, the proposed method has improved correction capability compared to the GS method. This method is applied for decoding IRS codes when used as outer codes in concatenated code

Multilevel Coding and Unequal Error Protection for Multiple-Access Communications and Ultra-Wideband Communications in the Presence of Interference.

Interference is one of the major factors that degrade the performance of a communication system. Various types of interference cause di Kerent impact on the system performance. In this thesis, we consider interference management at the physical layer. In order to enhance the performance, the receiver needs to have the knowledge about the interference. By exploiting the knowledge about interference, such as statistical properties, it can be suppressed to enhance the link quality. This thesis contains two main topics: multilevel coding (MLC) for unequal error protection (UEP) and receiver design for ultra-wideband (UWB) communications to suppress interference. Both topics deal with interference in di Kerent ways, and face di Kerent design challenges. MLC is a way to provide UEP for different streams of information with different levels of importance in a communication system. It combines coding and modulation schemes to optimize the system performance. The idea is to protect each bit in the modulation constellation point by an individual binary code. We designed and analyzed a DS-CDMA system with asymmetric PSK modulation and MLC using BCH codes in an AWGN channel. The analysis includes probability of bit error of the system, and the capacity and throughput of the MLC scheme combined with 8-PSK modulation. The results show that the MLC scheme can have a higher throughput than the regular coding scheme in the low SNR region in the AWGN channel. We also analyzed the performance of UWB communications in the presence of MAI and jamming interference. We considered a nonlinear interference suppression technique for impulse radio based UWB systems in the AWGN channel. The technique is based on the locally optimum Bayes detection (LOBD) algorithm, which utilizes the interference probability density function (PDF) for receiver design. This type of receiver has low complexity, and numerical results show that its performance asymptotically approaches that of the optimum receiver. Lastly, we discussed the implementation of the proposed receiver by adaptively monitor and update the interference PDF. The adaptive LOBD algorithm makes the proposed receiver implementation practical to deal with different types of interference.Ph.D.Electrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/75955/1/wangcw_1.pd

Bandwidth-efficient communication systems based on finite-length low density parity check codes

Low density parity check (LDPC) codes are linear block codes constructed by pseudo-random parity check matrices. These codes are powerful in terms of error performance and, especially, have low decoding complexity. While infinite-length LDPC codes approach the capacity of communication channels, finite-length LDPC codes also perform well, and simultaneously meet the delay requirement of many communication applications such as voice and backbone transmissions. Therefore, finite-length LDPC codes are attractive to employ in low-latency communication systems. This thesis mainly focuses on the bandwidth-efficient communication systems using finite-length LDPC codes. Such bandwidth-efficient systems are realized by mapping a group of LDPC coded bits to a symbol of a high-order signal constellation. Depending on the systems' infrastructure and knowledge of the channel state information (CSI), the signal constellations in different coded modulation systems can be two-dimensional multilevel/multiphase constellations or multi-dimensional space-time constellations. In the first part of the thesis, two basic bandwidth-efficient coded modulation systems, namely LDPC coded modulation and multilevel LDPC coded modulation, are investigated for both additive white Gaussian noise (AWGN) and frequency-flat Rayleigh fading channels. The bounds on the bit error rate (BER) performance are derived for these systems based on the maximum likelihood (ML) criterion. The derivation of these bounds relies on the union bounding and combinatoric techniques. In particular, for the LDPC coded modulation, the ML bound is computed from the Hamming distance spectrum of the LDPC code and the Euclidian distance profile of the two-dimensional constellation. For the multilevel LDPC coded modulation, the bound of each decoding stage is obtained for a generalized multilevel coded modulation, where more than one coded bit is considered for level. For both systems, the bounds are confirmed by the simulation results of ML decoding and/or the performance of the ordered-statistic decoding (OSD) and the sum-product decoding. It is demonstrated that these bounds can be efficiently used to evaluate the error performance and select appropriate parameters (such as the code rate, constellation and mapping) for the two communication systems.The second part of the thesis studies bandwidth-efficient LDPC coded systems that employ multiple transmit and multiple receive antennas, i.e., multiple-input multiple-output (MIMO) systems. Two scenarios of CSI availability considered are: (i) the CSI is unknown at both the transmitter and the receiver; (ii) the CSI is known at both the transmitter and the receiver. For the first scenario, LDPC coded unitary space-time modulation systems are most suitable and the ML performance bound is derived for these non-coherent systems. To derive the bound, the summation of chordal distances is obtained and used instead of the Euclidean distances. For the second case of CSI, adaptive LDPC coded MIMO modulation systems are studied, where three adaptive schemes with antenna beamforming and/or antenna selection are investigated and compared in terms of the bandwidth efficiency. For uncoded discrete-rate adaptive modulation, the computation of the bandwidth efficiency shows that the scheme with antenna selection at the transmitter and antenna combining at the receiver performs the best when the number of antennas is small. For adaptive LDPC coded MIMO modulation systems, an achievable threshold of the bandwidth efficiency is also computed from the ML bound of LDPC coded modulation derived in the first part

Serial-Turbo-Trellis-Coded Modulation with Rate-1 Inner Code

Serially concatenated turbo codes have been proposed to satisfy requirements for low bit- and word-error rates and for low (in comparison with related previous codes) complexity of coding and decoding algorithms and thus low complexity of coding and decoding circuitry. These codes are applicable to such high-level modulations as octonary phase-shift keying (8PSK) and 16-state quadrature amplitude modulation (16QAM); the signal product obtained by applying one of these codes to one of these modulations is denoted, generally, as serially concatenated trellis-coded modulation (SCTCM). These codes could be particularly beneficial for communication systems that must be designed and operated subject to limitations on bandwidth and power. Some background information is prerequisite to a meaningful summary of this development. Trellis-coded modulation (TCM) is now a well-established technique in digital communications. A turbo code combines binary component codes (which typically include trellis codes) with interleaving. A turbo code of the type that has been studied prior to this development is composed of parallel concatenated convolutional codes (PCCCs) implemented by two or more constituent systematic encoders joined through one or more interleavers. The input information bits feed the first encoder and, after having been scrambled by the interleaver, enter the second encoder. A code word of a parallel concatenated code consists of the input bits to the first encoder followed by the parity check bits of both encoders. The suboptimal iterative decoding structure for such a code is modular, and consists of a set of concatenated decoding modules one for each constituent code connected through an interleaver identical to the one in the encoder side. Each decoder performs weighted soft decoding of the input sequence. PCCCs yield very large coding gains at the cost of a reduction in the data rate and/or an increase in bandwidth

An identity of Chernoff bounds with an interpretation in statistical physics and applications in information theory

An identity between two versions of the Chernoff bound on the probability a certain large deviations event, is established. This identity has an interpretation in statistical physics, namely, an isothermal equilibrium of a composite system that consists of multiple subsystems of particles. Several information--theoretic application examples, where the analysis of this large deviations probability naturally arises, are then described from the viewpoint of this statistical mechanical interpretation. This results in several relationships between information theory and statistical physics, which we hope, the reader will find insightful.Comment: 29 pages, 1 figure. Submitted to IEEE Trans. on Information Theor

On multi-user EXIT chart analysis aided turbo-detected MBER beamforming designs

Abstract—This paper studies the mutual information transfer characteristics of a novel iterative soft interference cancellation (SIC) aided beamforming receiver communicating over both additive white Gaussian noise (AWGN) and multipath slow fading channels. Based on the extrinsic information transfer (EXIT) chart technique, we investigate the convergence behavior of an iterative minimum bit error rate (MBER) multiuser detection (MUD) scheme as a function of both the system parameters and channel conditions in comparison to the SIC aided minimum mean square error (SIC-MMSE) MUD. Our simulation results show that the EXIT chart analysis is sufficiently accurate for the MBER MUD. Quantitatively, a two-antenna system was capable of supporting up to K=6 users at Eb/N0=3dB, even when their angular separation was relatively low, potentially below 20?. Index Terms—Minimum bit error rate, beamforming, multiuser detection, soft interference cancellation, iterative processing, EXIT chart

Design of serially-concatenated LDGM codes

[Resumen] Since Shannon demonstrated in 1948 the feasibility of achieving an arbitrarily low error probability in a communications system provided that the transmission rate was kept below a certain limit, one of the greatest challenges in the realm of digital communications and, more specifically, in the channel coding field, has been finding codes that are able to approach this limit as much as possible with a reasonable encoding and decoding complexity, However, it was not until 1993, when Berrou et al. presented the turbo codes, that a coding scheme capable of performing at less than 1dB from Shannon's limit with an extremely low error probability was found. The idea on which these codes are based is the iterative decoding of concatenated components that exchange information about the transmitted bits, which is known as the "turbo principle". The generalization of this idea led in 1995 to the rediscovery of LDPC (Low Density Parity Check) codes, proposed for the first time by Gallager in the 60s. LDPC codes are linear block codes with a sparse parity check matrix that are able to surpass the performance of turbo codes with a smaller decoding complexity. However, due to the fact that the generator matrix of general LDPC codes is not sparse, their encoding complexity can be excessively high. LDGM (Low Density Generator Matrix) codes, a particular case of LDPC codes, are codes with a sparse generator matrix, thanks to which they present a lower encoding complexity. However, except for the case of very high rate codes, LDGM codes are "bad", i.e., they have a non-zero error probability that is independent of the code block length. More recently, IRA (Irregular Repeat-Accumulated) codes, consisting of the serial concatenation of a LDGM code and an accumulator, have been proposed. IRA codes are able to get close to the performance of LDPC codes with an encoding complexity similar to that of LDGM codes. In this thesis we explore an alternative to IRA codes consisting in the serial concatenation of two LDGM codes, a scheme that we will denote SCLDGM (Serially-Concatenated Low-Density Generator Matrix). The basic premise of SCLDGM codes is that an inner code of rate close to the desired transmission rate fixes most of the errors, and an external code of rate close to one corrects the few errors that result from decoding the inner code. For any of these schemes to perform as close as possible to the capacity limit it is necessary to determine the code parameters that best fit the channel over which the transmission will be done. The two techniques most commonly used in the literature to optimize LDPC codes are Density Evolution (DE) and EXtrinsic Information Transfer (EXIT) charts, which have been employed to obtain optimized codes that perform at a few tenths of a decibel of the AWGN channel capacity. However, no optimization techniques have been presented for SCLDGM codes, which so far have been designed heuristically and therefore their performance is far from the performance achieved by IRA and LDPC codes. Other of the most important advances that have occurred in recent years is the utilization of multiple antennas at the trasmitter and the receiver, which is known as MIMO (Multiple-Input Multiple-Output) systems. Telatar showed that the channel capacity in these kind of systems scales linearly with the minimum number of transmit and receive antennas, which enables us to achieve spectral efficiencies far greater than with systems with a single transmit and receive antenna (or Single Input Single Output (SISO) systems). This important advantage has attracted a lot of attention from the research community, and has caused that many of the new standards, such as WiMax 802.16e or WiFi 802.11n, as well as future 4G systems are based on MIMO systems. The main problem of MIMO systems is the high complexity of optimum detection, which grows exponentially with the number of transmit antennas and the number of modulation levels. Several suboptimum algorithms have been proposed to reduce this complexity, most notably the SIC-MMSE (Soft-Interference Cancellation Minimum Mean Square Error) and spherical detectors. Another major issue is the high complexity of the channel estimation, due to the large number of coefficients which determine it. There are techniques, such as Maximum-Likelihood-Expectation-Maximization (ML-EM), that have been successfully applied to estimate MIMO channels but, as in the case of detection, they suffer from the problem of a very high complexity when the number of transmit antennas or the size of the constellation increase. The main objective of this work is the study and optimization of SCLDGM codes in SISO and MIMO channels. To this end, we propose an optimization method for SCLDGM codes based on EXIT charts that allow these codes to exceed the performance of IRA codes existing in the literature and get close to the performance of LDPC codes, with the advantage over the latter of a lower encoding complexity. We also propose optimized SCLDGM codes for both spherical and SIC-MMSE suboptimal MIMO detectors, constituting a system that is capable of approaching the capacity limits of MIMO channels with a low complexity encoding, detection and decoding. We analyze the BICM (Bit-Interleaved Coded Modulation) scheme and the concatenation of SCLDGM codes with Space-Time Codes (STC) in ergodic and quasi-static MIMO channels. Furthermore, we explore the combination of these codes with different channel estimation algorithms that will take advantage of the low complexity of the suboptimum detectors to reduce the complexity of the estimation process while keeping a low distance to the capacity limit. Finally, we propose coding schemes for low rates involving the serial concatenation of several LDGM codes, reducing the complexity of recently proposed schemes based on Hadamard codes