Spatially Coupled Sparse Regression Codes for Single- and Multi-user Communications

Sparse regression codes (SPARCs) are a class of channel codes for efficient communication over the single-user additive white Gaussian noise (AWGN) channel at rates approaching the channel capacity. In a standard SPARC, codewords are sparse linear combinations of columns of an i.i.d. Gaussian design matrix, and the user message is encoded in the indices of those columns. Techniques such as power allocation and spatial coupling have been proposed to improve the performance of low-complexity iterative decoding algorithms such as approximate message passing (AMP). In this thesis we investigate spatially coupled SPARCs, where the design matrix has a block- wise band-diagonal structure, and modulated SPARCs, which generalise standard SPARCs by introducing modulation to the encoding of user messages. We introduce a base matrix framework which provides a unified way to construct power allocated and spatially coupled design matrices, and propose AMP decoders for modulated SPARCs constructed using base matrices. We prove that phase shift keying modulated and spatially coupled SPARCs with AMP decoding asymptotically achieve the capacity of the (complex) AWGN channel. We also show via numerical simulations that they can achieve lower error rates than standard coded modulation schemes at finite code lengths. A sliding window AMP decoder is proposed for spatially coupled SPARCs that significantly reduces the decoding latency and complexity. We then investigate coding schemes based on random linear models and AMP decoding for the multi-user Gaussian multiple access channel in the asymptotic regime where the number of users grows linearly with the code length. For a fixed target error rate and message size per user (in bits), we obtain the exact trade-off between energy-per-bit and the user density achievable in the large system limit. We show that a coding scheme based on spatially coupled Gaussian matrices and AMP decoding achieves near-optimal trade-off for a large range of user densities. To the best of our knowledge, this is the first efficient coding scheme to do so in this multiple access regime. Moreover, the spatially coupled coding scheme has a practical interpretation: it can be viewed as block-wise time-division with overlap.Funded by a Doctoral Training Partnership Award from the Engineering and Physical Sciences Research Council

Channel Detection and Decoding With Deep Learning

In this thesis, we investigate the designs of pragmatic data detectors and channel decoders with the assistance of deep learning. We focus on three emerging and fundamental research problems, including the designs of message passing algorithms for data detection in faster-than-Nyquist (FTN) signalling, soft-decision decoding algorithms for high-density parity-check codes and user identification for massive machine-type communications (mMTC). These wireless communication research problems are addressed by the employment of deep learning and an outline of the main contributions are given below. In the first part, we study a deep learning-assisted sum-product detection algorithm for FTN signalling. The proposed data detection algorithm works on a modified factor graph which concatenates a neural network function node to the variable nodes of the conventional FTN factor graph to compensate any detrimental effects that degrade the detection performance. By investigating the maximum-likelihood bit-error rate performance of a finite length coded FTN system, we show that the error performance of the proposed algorithm approaches the maximum a posterior performance, which might not be approachable by employing the sum-product algorithm on conventional FTN factor graph. After investigating the deep learning-assisted message passing algorithm for data detection, we move to the design of an efficient channel decoder. Specifically, we propose a node-classified redundant decoding algorithm based on the received sequence’s channel reliability for Bose-Chaudhuri-Hocquenghem (BCH) codes. Two preprocessing steps are proposed prior to decoding, to mitigate the unreliable information propagation and to improve the decoding performance. On top of the preprocessing, we propose a list decoding algorithm to augment the decoder’s performance. Moreover, we show that the node-classified redundant decoding algorithm can be transformed into a neural network framework, where multiplicative tuneable weights are attached to the decoding messages to optimise the decoding performance. We show that the node-classified redundant decoding algorithm provides a performance gain compared to the random redundant decoding algorithm. Additional decoding performance gain can be obtained by both the list decoding method and the neural network “learned” node-classified redundant decoding algorithm. Finally, we consider one of the practical services provided by the fifth-generation (5G) wireless communication networks, mMTC. Two separate system models for mMTC are studied. The first model assumes that low-resolution digital-to-analog converters are equipped by the devices in mMTC. The second model assumes that the devices' activities are correlated. In the first system model, two rounds of signal recoveries are performed. A neural network is employed to identify a suspicious device which is most likely to be falsely alarmed during the first round of signal recovery. The suspicious device is enforced to be inactive in the second round of signal recovery. The proposed scheme can effectively combat the interference caused by the suspicious device and thus improve the user identification performance. In the second system model, two deep learning-assisted algorithms are proposed to exploit the user activity correlation to facilitate channel estimation and user identification. We propose a deep learning modified orthogonal approximate message passing algorithm to exploit the correlation structure among devices. In addition, we propose a neural network framework that is dedicated for the user identification. More specifically, the neural network aims to minimise the missed detection probability under a pre-determined false alarm probability. The proposed algorithms substantially reduce the mean squared error between the estimate and unknown sequence, and largely improve the trade-off between the missed detection probability and the false alarm probability compared to the conventional orthogonal approximate message passing algorithm. All the aforementioned three parts of research works demonstrate that deep learning is a powerful technology in the physical layer designs of wireless communications

Representational dynamics across multiple timescales in human cortical networks

Human cognition occurs at multiple timescales, including immediate processing of the ongoing experiences and slowly drifting higher-level thoughts. To understand how the brain selects and represents these various types of information to guide behavior, this thesis examined representational content within sensory regions, multiple demand (MD) network, and default mode network (DMN). Chapter 1 provides a background review of the current literature. It begins by reviewing experimental investigations of component visual processes that unfold over time. Next, the MD network is introduced as a collection of frontal and parietal regions involved in implementing cognitive control by assembling the required operations for task-relevant behavior. Finally, the DMN is introduced in the context of temporal processing hierarchies, with focus on its representation of situation models summarizing interactions among entities and the environment. The first experiment, presented in Chapter 2, used EEG/MEG to track multiple component processes of selective attention. Five distinct processing operations with different time-courses were quantified, including representation of visual display properties, target location, target identity, behavioral significance, and finally, possible reactivation of the attentional template. Chapter 3 used fMRI to examine neural representations of task episodes, which are temporally organized sequences of steps that occur within a given context. It was found that MD and visual regions showed sensitivity to the fine structure of the contents within a task. DMN regions showed gradual change throughout the entire task, with increased activation at the offset of the entire episode. Chapter 4 analyzed activation profiles of DMN regions using six diverse tasks to examine their functional convergence during social, episodic, and self-referential thought. Results supported proposals of separate subsystems, yet also suggest integration within the DMN. The final chapter, Chapter 5, provides an extended discussion of theoretical concepts related to the three experiments and proposes possible avenues for further research

Generalizable deep learning based medical image segmentation

Deep learning is revolutionizing medical image analysis and interpretation. However, its real-world deployment is often hindered by the poor generalization to unseen domains (new imaging modalities and protocols). This lack of generalization ability is further exacerbated by the scarcity of labeled datasets for training: Data collection and annotation can be prohibitively expensive in terms of labor and costs because label quality heavily dependents on the expertise of radiologists. Additionally, unreliable predictions caused by poor model generalization pose safety risks to clinical downstream applications. To mitigate labeling requirements, we investigate and develop a series of techniques to strengthen the generalization ability and the data efficiency of deep medical image computing models. We further improve model accountability and identify unreliable predictions made on out-of-domain data, by designing probability calibration techniques. In the first and the second part of thesis, we discuss two types of problems for handling unexpected domains: unsupervised domain adaptation and single-source domain generalization. For domain adaptation we present a data-efficient technique that adapts a segmentation model trained on a labeled source domain (e.g., MRI) to an unlabeled target domain (e.g., CT), using a small number of unlabeled training images from the target domain. For domain generalization, we focus on both image reconstruction and segmentation. For image reconstruction, we design a simple and effective domain generalization technique for cross-domain MRI reconstruction, by reusing image representations learned from natural image datasets. For image segmentation, we perform causal analysis of the challenging cross-domain image segmentation problem. Guided by this causal analysis we propose an effective data-augmentation-based generalization technique for single-source domains. The proposed method outperforms existing approaches on a large variety of cross-domain image segmentation scenarios. In the third part of the thesis, we present a novel self-supervised method for learning generic image representations that can be used to analyze unexpected objects of interest. The proposed method is designed together with a novel few-shot image segmentation framework that can segment unseen objects of interest by taking only a few labeled examples as references. Superior flexibility over conventional fully-supervised models is demonstrated by our few-shot framework: it does not require any fine-tuning on novel objects of interest. We further build a publicly available comprehensive evaluation environment for few-shot medical image segmentation. In the fourth part of the thesis, we present a novel probability calibration model. To ensure safety in clinical settings, a deep model is expected to be able to alert human radiologists if it has low confidence, especially when confronted with out-of-domain data. To this end we present a plug-and-play model to calibrate prediction probabilities on out-of-domain data. It aligns the prediction probability in line with the actual accuracy on the test data. We evaluate our method on both artifact-corrupted images and images from an unforeseen MRI scanning protocol. Our method demonstrates improved calibration accuracy compared with the state-of-the-art method. Finally, we summarize the major contributions and limitations of our works. We also suggest future research directions that will benefit from the works in this thesis.Open Acces

Multimedia

The nowadays ubiquitous and effortless digital data capture and processing capabilities offered by the majority of devices, lead to an unprecedented penetration of multimedia content in our everyday life. To make the most of this phenomenon, the rapidly increasing volume and usage of digitised content requires constant re-evaluation and adaptation of multimedia methodologies, in order to meet the relentless change of requirements from both the user and system perspectives. Advances in Multimedia provides readers with an overview of the ever-growing field of multimedia by bringing together various research studies and surveys from different subfields that point out such important aspects. Some of the main topics that this book deals with include: multimedia management in peer-to-peer structures & wireless networks, security characteristics in multimedia, semantic gap bridging for multimedia content and novel multimedia applications

25th Annual Computational Neuroscience Meeting: CNS-2016

Abstracts of the 25th Annual Computational Neuroscience Meeting: CNS-2016 Seogwipo City, Jeju-do, South Korea. 2–7 July 201

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

Deep Learning Designs for Physical Layer Communications

Wireless communication systems and their underlying technologies have undergone unprecedented advances over the last two decades to assuage the ever-increasing demands for various applications and emerging technologies. However, the traditional signal processing schemes and algorithms for wireless communications cannot handle the upsurging complexity associated with fifth-generation (5G) and beyond communication systems due to network expansion, new emerging technologies, high data rate, and the ever-increasing demands for low latency. This thesis extends the traditional downlink transmission schemes to deep learning-based precoding and detection techniques that are hardware-efficient and of lower complexity than the current state-of-the-art. The thesis focuses on: precoding/beamforming in massive multiple-inputs-multiple-outputs (MIMO), signal detection and lightweight neural network (NN) architectures for precoder and decoder designs. We introduce a learning-based precoder design via constructive interference (CI) that performs the precoding on a symbol-by-symbol basis. Instead of conventionally training a NN without considering the specifics of the optimisation objective, we unfold a power minimisation symbol level precoding (SLP) formulation based on the interior-point-method (IPM) proximal ‘log’ barrier function. Furthermore, we propose a concept of NN compression, where the weights are quantised to lower numerical precision formats based on binary and ternary quantisations. We further introduce a stochastic quantisation technique, where parts of the NN weight matrix are quantised while the remaining is not. Finally, we propose a systematic complexity scaling of deep neural network (DNN) based MIMO detectors. The model uses a fraction of the DNN inputs by scaling their values through weights that follow monotonically non-increasing functions. Furthermore, we investigate performance complexity tradeoffs via regularisation constraints on the layer weights such that, at inference, parts of network layers can be removed with minimal impact on the detection accuracy. Simulation results show that our proposed learning-based techniques offer better complexity-vs-BER (bit-error-rate) and complexity-vs-transmit power performances compared to the state-of-the-art MIMO detection and precoding techniques