Iterative Decoding of K/N Convolutional Codes based on Recurrent Neural Network with Stopping Criterion

This paper outlines a novel iterative decoding technique for a rate K/N convolutional code based on recurrent neural network (RNN) with stopping criterion. The algorithm is introduced by describing the theoretical models of the encoder and decoder. In particular this paper focuses on the investigation of a stopping criterion on the iterating procedure in order to minimize the decoding time yet still obtain an optimal BER performance. The simulation results of a rate 1/2 and 2/3 encoders respectively in comparison with the conventional Viterbi decoder are also presente

When Machine Learning Meets Information Theory: Some Practical Applications to Data Storage

Machine learning and information theory are closely inter-related areas. In this dissertation, we explore topics in their intersection with some practical applications to data storage. Firstly, we explore how machine learning techniques can be used to improve data reliability in non-volatile memories (NVMs). NVMs, such as flash memories, store large volumes of data. However, as devices scale down towards small feature sizes, they suffer from various kinds of noise and disturbances, thus significantly reducing their reliability. This dissertation explores machine learning techniques to design decoders that make use of natural redundancy (NR) in data for error correction. By NR, we mean redundancy inherent in data, which is not added artificially for error correction. This work studies two different schemes for NR-based error-correcting decoders. In the first scheme, the NR-based decoding algorithm is aware of the data representation scheme (e.g., compression, mapping of symbols to bits, meta-data, etc.), and uses that information for error correction. In the second scenario, the NR-decoder is oblivious of the representation scheme and uses deep neural networks (DNNs) to recognize the file type as well as perform soft decoding on it based on NR. In both cases, these NR-based decoders can be combined with traditional error correction codes (ECCs) to substantially improve their performance. Secondly, we use concepts from ECCs for designing robust DNNs in hardware. Non-volatile memory devices like memristors and phase-change memories are used to store the weights of hardware implemented DNNs. Errors and faults in these devices (e.g., random noise, stuck-at faults, cell-level drifting etc.) might degrade the performance of such DNNs in hardware. We use concepts from analog error-correcting codes to protect the weights of noisy neural networks and to design robust neural networks in hardware. To summarize, this dissertation explores two important directions in the intersection of information theory and machine learning. We explore how machine learning techniques can be useful in improving the performance of ECCs. Conversely, we show how information-theoretic concepts can be used to design robust neural networks in hardware

Toward the implementation of analog LDPC decoders for long codewords

Error control codes are used in virtually every digital communication system. Traditionally, decoders have been implemented digitally. Analog decoders have been recently shown to have the potential to outperform digital decoders in terms of area and power/speed ratio. Analog designers have attempted to fully understand and exploit this potential for large decoders. However, large codes are generally still implemented with digital circuits. Nevertheless, in this thesis a number of aspects of analog decoder implementation are investigated with the hope of enabling the design of large analog decoders. In this thesis, we study and modify analog circuits used in a decoding algorithm known as the sum-product algorithm for implementation in a CMOS 90 nm technology. We apply a current-mode approach at the input nodes of these circuits and show through simulations that the power/speed ratio will be improved. Interested in studying the dynamics of decoders, we model an LDPC code in MATLAB's Simulink. We then apply the linearization technique on the modeled LDPC code in order to linearize the decoder about an initial state as its solution point. Challenges associated with decoder linearization are discussed. We also design and implement a chip comprised of the sum-product circuits with different configurations and sizes in order to study the effect of mismatch on the accuracy of the outputs. Unfortunately, testing of the chip fails as a result of errors in either the packaging process or fabrication

Application of wavelets and artificial neural network for indoor optical wireless communication systems

Abstract This study investigates the use of error control code, discrete wavelet transform (DWT) and artificial neural network (ANN) to improve the link performance of an indoor optical wireless communication in a physical channel. The key constraints that barricade the realization of unlimited bandwidth in optical wavelengths are the eye-safety issue, the ambient light interference and the multipath induced intersymbol interference (ISI). Eye-safety limits the maximum average transmitted optical power. The rational solution is to use power efficient modulation techniques. Further reduction in transmitted power can be achieved using error control coding. A mathematical analysis of retransmission scheme is investigated for variable length modulation techniques and verified using computer simulations. Though the retransmission scheme is simple to implement, the shortfall in terms of reduced throughput will limit higher code gain. Due to practical limitation, the block code cannot be applied to the variable length modulation techniques and hence the convolutional code is the only possible option. The upper bound for slot error probability of the convolutional coded dual header pulse interval modulation (DH-PIM) and digital pulse interval modulation (DPIM) schemes are calculated and verified using simulations. The power penalty due to fluorescent light interference (FL I) is very high in indoor optical channel making the optical link practically infeasible. A denoising method based on a DWT to remove the FLI from the received signal is devised. The received signal is first decomposed into different DWT levels; the FLI is then removed from the signal before reconstructing the signal. A significant reduction in the power penalty is observed using DWT. Comparative study of DWT based denoising scheme with that of the high pass filter (HPF) show that DWT not only can match the best performance obtain using a HPF, but also offers a reduced complexity and design simplicity. The high power penalty due to multipath induced ISI makes a diffuse optical link practically infeasible at higher data rates. An ANN based linear and DF architectures are investigated to compensation the ISI. Unlike the unequalized cases, the equalized schemes don‘t show infinite power penalty and a significant performance improvement is observed for all modulation schemes. The comparative studies substantiate that ANN based equalizers match the performance of the traditional equalizers for all channel conditions with a reduced training data sequence. The study of the combined effect of the FLI and ISI shows that DWT-ANN based receiver perform equally well in the present of both interference. Adaptive decoding of error control code can offer flexibility of selecting the best possible encoder in a given environment. A suboptimal ?soft‘ sliding block convolutional decoder based on the ANN and a 1/2 rate convolutional code with a constraint length is investigated. Results show that the ANN decoder can match the performance of optimal Viterbi decoder for hard decision decoding but with slightly inferior performance compared to soft decision decoding. This provides a foundation for further investigation of the ANN decoder for convolutional code with higher constraint length values. Finally, the proposed DWT-ANN receiver is practically realized in digital signal processing (DSP) board. The output from the DSP board is compared with the computer simulations and found that the difference is marginal. However, the difference in results doesn‘t affect the overall error probability and identical error probability is obtained for DSP output and computer simulations