3 research outputs found
Soft-Output Deep Neural Network-Based Decoding
Deep neural network (DNN)-based channel decoding is widely considered in the
literature. The existing solutions are investigated for the case of hard
output, i.e. when the decoder returns the estimated information word. At the
same time, soft-output decoding is of critical importance for iterative
receivers and decoders. In this paper, we focus on the soft-output DNN-based
decoding problem. We start with the syndrome-based approach proposed by
Bennatan et al. (2018) and modify it to provide soft output in the AWGN
channel. The new decoder can be considered as an approximation of the MAP
decoder with smaller computation complexity. We discuss various regularization
functions for joint DNN-MAP training and compare the resulting distributions
for [64, 45] BCH code. Finally, to demonstrate the soft-output quality we
consider the turbo-product code with [64, 45] BCH codes as row and column
codes. We show that the resulting DNN-based scheme is very close to the
MAP-based performance and significantly outperforms the solution based on the
Chase decoder. We come to the conclusion that the new method is prospective for
the challenging problem of DNN-based decoding of long codes consisting of short
component codes.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
Dynamic Mathematics for Automated Machine Learning Techniques
Machine Learning and Neural Networks have been gaining popularity and are widely considered as the driving force of the Fourth Industrial Revolution. However, modern machine learning techniques such as backpropagation training was firmly established in 1986 while computer vision was revolutionised in 2012 with the introduction of AlexNet. Given all these accomplishments, why are neural networks still not an integral part of our society? ``Because they are difficult to implement in practice.'' I'd like to use machine learning, but I can't invest much time. The concept of Automated Machine Learning (AutoML) was first proposed by Professor Frank Hutter of the University of Freiburg. Machine learning is not simple; it requires a practitioner to have thorough understanding on the attributes of their data and the components which their model entails. AutoML is the effort to automate all tedious aspects of machine learning to form a clean data analysis pipeline. This thesis is our effort to develop and to understand ways to automate machine learning. Specifically, we focused on Recurrent Neural Networks (RNNs), Meta-Learning, and Continual Learning. We studied continual learning to enable a network to sequentially acquire skills in a dynamic environment; we studied meta-learning to understand how a network can be configured efficiently; and we studied RNNs to understand the consequences of consecutive actions. Our RNN-study focused on mathematical interpretability. We described a large variety of RNNs as one mathematical class to understand their core network mechanism. This enabled us to extend meta-learning beyond network configuration for network pruning and continual learning. This also provided insights for us to understand how a single network should be consecutively configured and led us to the creation of a simple generic patch that is compatible to several existing continual learning archetypes. This patch enhanced the robustness of continual learning techniques and allowed them to generalise data better. By and large, this thesis presented a series of extensions to enable AutoML to be made simple, efficient, and robust. More importantly, all of our methods are motivated with mathematical understandings through the lens of dynamical systems. Thus, we also increased the interpretability of AutoML concepts