Nonlinear consensus+innovations under correlated heavy-tailed noises: Mean square convergence rate and asymptotics

We consider distributed recursive estimation of consensus+innovations type in the presence of heavy-tailed sensing and communication noises. We allow that the sensing and communication noises are mutually correlated while independent identically distributed (i.i.d.) in time, and that they may both have infinite moments of order higher than one (hence having infinite variances). Such heavy-tailed, infinite-variance noises are highly relevant in practice and are shown to occur, e.g., in dense internet of things (IoT) deployments. We develop a consensus+innovations distributed estimator that employs a general nonlinearity in both consensus and innovations steps to combat the noise. We establish the estimator's almost sure convergence, asymptotic normality, and mean squared error (MSE) convergence. Moreover, we establish and explicitly quantify for the estimator a sublinear MSE convergence rate. We then quantify through analytical examples the effects of the nonlinearity choices and the noises correlation on the system performance. Finally, numerical examples corroborate our findings and verify that the proposed method works in the simultaneous heavy-tail communication-sensing noise setting, while existing methods fail under the same noise conditions

Nonlocal Myriad Filters for Cauchy Noise Removal

The contribution of this paper is two-fold. First, we introduce a generalized myriad filter, which is a method to compute the joint maximum likelihood estimator of the location and the scale parameter of the Cauchy distribution. Estimating only the location parameter is known as myriad filter. We propose an efficient algorithm to compute the generalized myriad filter and prove its convergence. Special cases of this algorithm result in the classical myriad filtering, respective an algorithm for estimating only the scale parameter. Based on an asymptotic analysis, we develop a second, even faster generalized myriad filtering technique. Second, we use our new approaches within a nonlocal, fully unsupervised method to denoise images corrupted by Cauchy noise. Special attention is paid to the determination of similar patches in noisy images. Numerical examples demonstrate the excellent performance of our algorithms which have moreover the advantage to be robust with respect to the parameter choice

Analysis of low-density parity-check codes on impulsive noise channels

PhD ThesisCommunication channels can severely degrade a signal, not only due to fading effects but also interference in the form of impulsive noise. In conventional communication systems, the additive noise at the receiver is usually assumed to be Gaussian distributed. However, this assumption is not always valid and examples of non-Gaussian distributed noise include power line channels, underwater acoustic channels and manmade interference. When designing a communication system it is useful to know the theoretical performance in terms of bit-error probability (BEP) on these types of channels. However, the effect of impulses on the BEP performance has not been well studied, particularly when error correcting codes are employed. Today, advanced error-correcting codes with very long block lengths and iterative decoding algorithms, such as Low-Density Parity-Check (LDPC) codes and turbo codes, are popular due to their capacity-approaching performance. However, very long codes are not always desirable, particularly in communications systems where latency is a serious issue, such as in voice and video communication between multiple users. This thesis focuses on the analysis of short LDPC codes. Finite length analyses of LDPC codes have already been presented for the additive white Gaussian noise channel in the literature, but the analysis of short LDPC codes for channels that exhibit impulsive noise has not been investigated. The novel contributions in this thesis are presented in three sections. First, uncoded and LDPC-coded BEP performance on channels exhibiting impulsive noise modelled by symmetric -stable (S S) distributions are examined. Different sub-optimal receivers are compared and a new low-complexity receiver is proposed that achieves near-optimal performance. Density evolution is then used to derive the threshold signal-tonoise ratio (SNR) of LDPC codes that employ these receivers. In order to accurately predict the waterfall performance of short LDPC codes, a nite length analysis is proposed with the aid of the threshold SNRs of LDPC codes and the derived uncoded BEPs for impulsive noise channels. Second, to investigate the e ect of impulsive noise on wireless channels, the analytic BEP on generalized fading channels with S S noise is derived. However, it requires the evaluation of a double integral to obtain the analytic BEP, so to reduce the computational cost, the Cauchy- Gaussian mixture model and the asymptotic property of S S process are used to derive upper bounds of the exact BEP. Two closed-form expressions are derived to approximate the exact BEP on a Rayleigh fading channel with S S noise. Then density evolution of different receivers is derived for these channels to nd the asymptotic performance of LDPC codes. Finally, the waterfall performance of LDPC codes is again estimated for generalized fading channels with S S noise by utilizing the derived uncoded BEP and threshold SNRs. Finally, the addition of spatial diversity at the receiver is investigated. Spatial diversity is an effective method to mitigate the effects of fading and when used in conjunction with LDPC codes and can achieve excellent error-correcting performance. Hence, the performance of conventional linear diversity combining techniques are derived. Then the SNRs of these linear combiners are compared and the relationship of the noise power between different linear combiners is obtained. Nonlinear detectors have been shown to achieve better performance than linear combiners hence, optimal and sub-optimal detectors are also presented and compared. A non-linear detector based on the bi-parameter Cauchy-Gaussian mixture model is used and shows near-optimal performance with a significant reduction in complexity when compared with the optimal detector. Furthermore, we show how to apply density evolution of LDPC codes for different combining techniques on these channels and an estimation of the waterfall performance of LDPC codes is derived that reduces the gap between simulated and asymptotic performance. In conclusion, the work presented in this thesis provides a framework to evaluate the performance of communication systems in the presence of additive impulsive noise, with and without spatial diversity at the receiver. For the first time, bounds on the BEP performance of LDPC codes on channels with impulsive noise have been derived for optimal and sub-optimal receivers, allowing other researchers to predict the performance of LDPC codes in these type of environments without needing to run lengthy computer simulations

A flexible statistical framework for the characterization and modelling of noise in powerline communication channels.

Doctor of Philosophy in Electronic Engineering.One communication medium that has received a lot of interest in recent years is the power line channel, especially for the delivery of broadband content. This channel has been traditionally used to carry electrical power only. But with the recent advancements in digital signal processing, it is now possible to realize communications through the power grid, both in narrowband and broadband. The use of the power line network for telecommunication purposes constitutes what is referred to as powerline carrier communications or simply powerline communications (PLC). The biggest incentive for PLC technology use is the fact that the power line network is already in place, which greatly reduces the communication network set up cost, since no new cabling layout is required. PLC technology is widely applied in home networking, broadband internet provision and smart grid solutions. However, the PLC channel presents a very hostile communication environment. And as such, no consideration has been made in the design of traditional power line network to accommodate communication services. Of all the PLC channel impairments which include frequency-dependent attenuation, frequency selectivity, multipath and noise, noise is the biggest threat to communication signals. This noise manifests itself in form of coloured background noise, narrowband interference and impulsive noise. A thorough understanding of this noise distribution is therefore crucial for the design of a reliable and high performing PLC system. A proper understanding of the noise characteristics in the PLC channel can only be realized through noise measurements in live power networks, and then analyzing and modeling the noise appropriately. Moreover, the noise scenario in power line networks is very complex and therefore cannot be modeled through mere analytical methods. Additionally, most of the models that have been proposed for the PLC noise previously are mere adaptations of the measured noise to some existing impulsive noise models. These earlier modeling approaches are also rigid and model the noise via a fixed set of parameters. In the introductory work in this thesis, a study of orthogonal frequency division multiplexing (OFDM) as the modulation of choice for PLC systems is presented. A thorough survey of the salient features of this modulation scheme that make it the perfect candidate for PLC modulation needs is presented. In the end, a performance analysis study on the impact of impulsive noise on an OFDM based binary phase shift keying (BPSK) system is done. This study differs from earlier ones in that its focus is on how the elementary parameters that define the impulsive noise affect the system, a departure from the usual norm of considering the overall noise distribution. This study focuses on the impact of interarrival times (IAT), pulse amplitudes as well as pulse widths, among other parameters. In the first part of the main work in this thesis, results of an intensive noise measurement campaign for indoor low voltage power line noise carried out in various power line networks, in the Department of Electrical, Electronic and Computer Engineering buildings at the University of KwaZulu-Natal, Howard campus are presented. The noise measurements are carried out in both time and frequency domains. Next, the noise measurements are then analyzed and modeled using two very flexible data modeling tools; nonparametric kernel density estimators and parametric alpha stable (α-stable) distributions. The kernel method’s ability to overcome all the shortcomings of the primitive histogram method makes it very attractive. In this method, the noise data structure is derived straight from the data itself, with no prior assumptions or restrictions on the data structure, thus effectively overcoming the rigidity associated with previous noise models for power line channels. As such, it results in density estimates that “hug” the measured density as much as possible. The models obtained using the kernel methods are therefore better than any parametric equivalent; something that can always be proven through goodness of fit tests. These models therefore form an excellent reference for parametric modeling of the power line noise. This work forms the author’s first main contribution to PLC research. As a demonstration of the kernel models suitability to act as a reference, parametric models of the noise distribution using the alpha stable (α-stable) distribution are also developed. This distribution is chosen due to its flexibility and ability to capture impulsiveness (long-tailed behaviour), such as the one found in power line noise. Stable distributions are characterized by long/fat tails than those of the Gaussian distribution, and that is the main reason why they are preferable here since the noise characteritics obtained in the kernel technique show visible long/heavy tailed behavior. A parameter estimation technique that is based on quantiles and another on the empirical characteristic function are employed in the extraction of the four parameters that define the characteristic function of the α-stable distribution. The application of the α-stable distribution in other signal processing problems has often been over-simplied by considering the symmetric alpha stable distribution, but in this thesis, the general α-stable distribution is used to model the power line noise. This is necessary so as to ensure that no features of the noise distribution are missed. All the models obtained are validated through error analysis and Chi-square fitness tests. This work forms the author’s second main contribution to PLC research. The author’s last contribution in this thesis is the development of an algorithm for the synthesis of the power line as a Levy stable stochastic process. The algorithm developed is then used to generate the PLC noise process for a random number of alpha stable noise samples using the alpha stable noise parameters obtained in the parametric modeling using stable distributions. This algorithm is generalized for all admissible values of alpha stable noise parameters and therefore results for a Levy stable Gaussian process are also presented for the same number of random noise samples for comparison purposes

Investigation of non-binary trellis codes designed for impulsive noise environments

PhD ThesisIt is well known that binary codes with iterative decoders can achieve near Shannon limit performance on the additive white Gaussian noise (AWGN) channel, but their performance on more realistic wired or wireless channels can become degraded due to the presence of burst errors or impulsive noise. In such extreme environments, error correction alone cannot combat the serious e ect of the channel and must be combined with the signal processing techniques such as channel estimation, channel equalisation and orthogonal frequency division multiplexing (OFDM). However, even after the received signal has been processed, it can still contain burst errors, or the noise present in the signal maybe non Gaussian. In these cases, popular binary coding schemes such as Low-Density Parity-Check (LDPC) or turbo codes may not perform optimally, resulting in the degradation of performance. Nevertheless, there is still scope for the design of new non-binary codes that are more suitable for these environments, allowing us to achieve further gains in performance. In this thesis, an investigation into good non-binary trellis error-correcting codes and advanced noise reduction techniques has been carried out with the aim of enhancing the performance of wired and wireless communication networks in di erent extreme environments. These environments include, urban, indoor, pedestrian, underwater, and powerline communication (PLC). This work includes an examination of the performance of non-binary trellis codes in harsh scenarios such as underwater communications when the noise channel is additive S S noise. Similar work was also conducted for single input single output (SISO) power line communication systems for single carrier (SC) and multi carrier (MC) over realistic multi-path frequency selective channels. A further examination of multi-input multi-output (MIMO) wired and wireless systems on Middleton class A noise channel was carried out. The main focus of the project was non-binary coding schemes as it is well-known that they outperform their binary counterparts when the channel is bursty. However, few studies have investigated non-binary codes for other environments. The major novelty of this work is the comparison of the performance of non-binary trellis codes with binary trellis codes in various scenarios, leading to the conclusion that non-binary codes are, in most cases, superior in performance to binary codes. Furthermore, the theoretical bounds of SISO and MIMO binary and non-binary convolutional coded OFDM-PLC systems have been investigated for the rst time. In order to validate our results, the implementation of simulated and theoretical results have been obtained for di erent values of noise parameters and on di erent PLC channels. The results show a strong agreement between the simulated and theoretical analysis for all cases.University of Thi-Qar for choosing me for their PhD scholarship and the Iraqi Ministry of Higher Education and Scienti c Research (MOHESR) for granting me the funds to study in UK. In addition, there was ample support towards my stay in the UK from the Iraqi Cultural Attach e in Londo

Estimation of stability index for symmetric {\alpha}-stable distribution using quantile conditional variance ratios

The class of $\alpha$-stable distributions is widely used in various applications, especially for modelling heavy-tailed data. Although the $\alpha$-stable distributions have been used in practice for many years, new methods for identification, testing, and estimation are still being refined and new approaches are being proposed. The constant development of new statistical methods is related to the low efficiency of existing algorithms, especially when the underlying sample is small or the underlying distribution is close to Gaussian. In this paper we propose a new estimation algorithm for stability index, for samples from the symmetric $\alpha$-stable distribution. The proposed approach is based on quantile conditional variance ratio. We study the statistical properties of the proposed estimation procedure and show empirically that our methodology often outperforms other commonly used estimation algorithms. Moreover, we show that our statistic extracts unique sample characteristics that can be combined with other methods to refine existing methodologies via ensamble methods. Although our focus is set on the symmetric $\alpha$-stable case, we demonstrate that the considered statistic is insensitive to the skewness parameter change, so that our method could be also used in a more generic framework. For completeness, we also show how to apply our method on real data linked to plasma physics

Adaptive Equalisation for Impulsive Noise Environments

This thesis addresses the problem of adaptive channel equalisation in environments where the interfering noise exhibits non–Gaussian behaviour due to impulsive phenomena. The family of alpha-stable distributions has proved to be a suitable and flexible tool for the modelling of signals with impulsive nature. However,non–Gaussian alpha–stable signals have infinite variance, and signal processing techniques based on second order moments are meaningless in such environments. In order to exploit the flexibility of the stable family and still take advantage of the existing signal processing tools, a novel framework for the integration of the stable model in a communications context is proposed, based on a finite dynamic range receiver. The performance of traditional signal processing algorithms designed under the Gaussian assumption may degrade seriously in impulsive environments. When this degradation cannot be tolerated, the traditional signal processing methods must be revisited and redesigned taking into account the non–Gaussian noise statistics. In this direction, the optimum feed–forward and decision feedback Bayesian symbol–by–symbol equalisers for stable noise environments are derived. Then, new analytical tools for the evaluation of systems in infinite variance environments are presented. For the centers estimation of the proposed Bayesian equaliser, a unified framework for a family of robust recursive linear estimation techniques is presented and the underlying relationships between them are identified. Furthermore, the direct clustering technique is studied and robust variants of the existing algorithms are proposed. A novel clustering algorithm is also derived based on robust location estimation. The problem of estimating the stable parameters has been addressed in the literature and a variety of algorithms can be found. Some of these algorithms are assessed in terms of efficiency, simplicity and performance and the most suitable is chosen for the equalisation problem. All the building components of an adaptive Bayesian equaliser are then put together and the performance of the equaliser is evaluated experimentally. The simulation results suggest that the proposed adaptive equaliser offers a significant performance benefit compared with a traditional equaliser, designed under the Gaussian assumption. The implementation of the proposed Bayesian equaliser is simple but the computational complexity can be unaffordable. However, this thesis proposes certain approximations which enable the computationally efficient implementation of the optimum equaliser with negligible loss in performance

Time domain classification of transient RFI

Since the emergence of radio astronomy as a field, it has been afflicted by radio frequency interference (RFI). RFI continues to present a problem despite increasingly sophisticated countermeasures developed over the decades. Due to technological improvements, radio telescopes have become more sensitive (for example, MeerKAT’s L-band receiver). Existing RFI has become more prominent as a result. At the same time, the prevalence of RFI-generating devices has increased as new technologies have been adopted by society. Many approaches have been developed for mitigating RFI, which are typically used in concert. New telescope arrays are often built far from human habitation in radio-quiet reserves. In South Africa, a radio-quiet reserve has been established in which several world class instruments are under construction. Despite the remote location of the reserve, careful attention is paid to the possibility of RFI. For example, some instruments will begin observations while others are still under construction. The infrastructure and equipment related to the construction work may increase the risk of RFI, especially transient RFI. A number of mitigation strategies have been employed, including the use of fixed and mobile RFI monitoring stations. Such stations operate independently of the main telescope arrays and continuously monitor a wide bandwidth in all directions. They are capable of recording spectra and high resolution time domain captures of transient RFI. Once detected, and if identified, an RFI source can be found and dealt with. The ability to identify the sources of detected RFI would be highly beneficial. Continuous wave intentional transmissions (telecommunication signals for example) are easily identified as they are required to adhere to allocated frequency bands. Transient RFI signals, however, are significantly more challenging to identify since they are generally broadband and highly intermittent. Transient RFI can be generated as a by-product of the normal operation of devices such as relays, AC machines and fluorescent lights, for example. Such devices may be present near radio telescope arrays as part of the infrastructure or equipment involved in the construction of new instruments. Other than contaminating observation data, transient RFI can also appear to have genuine astronomical origins. In one case, transient signals received from a microwave oven exhibited dispersion, suggesting a distant source. Therefore, the ability to identify transient RFI by source would be enormously valuable. Once identified, such sources may be removed or replaced where possible. Despite this need, there is a paucity of work on classifying transient RFI in the literature. This thesis focusses on the problem of identifying transient RFI by source in time domain data of the type captured by remote monitoring stations. Several novel approaches are explored in this thesis. If used with independent RFI monitoring stations, these approaches may aid in tracking down nearby RFI sources at a radio telescope array. They may also be useful for improving RFI flagging in data from radio telescopes themselves. Distinguishing between transient RFI and natural astronomical signals is likely to be an easier prospect than classifying transient RFI by source. Furthermore, these approaches may be better able to avoid excising genuine astronomical transients that nevertheless share some characteristics with RFI signals. The radio telescopes themselves are significantly more sensitive than RFI monitoring stations, and would thus be able to detect RFI sources more easily. However, terrestrial RFI would likely enter via sidelobes, tempering this advantage somewhat. In this thesis, transient RFI is first characterised, prior to classification by source. Labelled time-domain recordings of a number of transient RFI sources are acquired and statistically examined. Second, components analysis techniques are considered for feature selection. Cluster separation is analysed for principal components analysis (PCA) and kernel PCA, the latter proving most suitable. The effect of the supply voltage of certain RFI sources on cluster separation in the principal components domain is also explored. Several na¨ıve classification algorithms are tested, using kernel PCA for feature selection A more sophisticated dictionary-based approach is developed next. While there are variations in repeated recordings of the same RFI source, the signals tend to adhere to a common overarching structure. Full RFI signals are observed to consist of sequences of individual transients. An algorithm is presented to extract individual transients from full recordings, after which they are labelled using unsupervised clustering methods. This procedure results in a dictionary of archetypal transients, from which any full RFI sequence may be represented. Some approaches in Automated Speech Recognition (ASR) are similar: spoken words are divided into individual labelled phonemes. Representing RFI signals as sequences enables the use of hidden Markov models (HMMs) for identification. HMMs are well suited to sequence identification problems, and are known for their robustness to variation. For example, in ASR, HMMs are able to handle the variations in repeated utterances of the same word. When classifying the recorded RFI signals, good accuracy is achieved, improving on the results obtained using the more na¨ıve methods. Finally, a strategy involving deep learning techniques is explored. Recurrent neural networks and convolutional neural networks (CNNs) have shown great promise in a wide variety of classification tasks. Here, a model is developed that includes a pre-trained CNN layer followed by a bidirectional long short-term memory (BLSTM) layer. Special attention is paid to mitigating class imbalance when the model is used with individual transients extracted from full recordings. High classification accuracy is achieved, improving on the dictionary-based approach and the other na¨ıve methods. Recommendations are made for future work on developing these approaches further for practical use with remote monitoring stations. Other possibilities for future research are also discussed, including testing the robustness of the proposed approaches. They may also prove useful for RFI excision in observation data from radio telescopes