New Methods for MLE of Toeplitz Structured Covariance Matrices with Applications to RADAR Problems

This work considers Maximum Likelihood Estimation (MLE) of a Toeplitz structured covariance matrix. In this regard, an equivalent reformulation of the MLE problem is introduced and two iterative algorithms are proposed for the optimization of the equivalent statistical learning framework. Both the strategies are based on the Majorization Minimization (MM) paradigm and hence enjoy nice properties such as monotonicity and ensured convergence to a stationary point of the equivalent MLE problem. The proposed framework is also extended to deal with MLE of other practically relevant covariance structures, namely, the banded Toeplitz, block Toeplitz, and Toeplitz-block-Toeplitz. Through numerical simulations, it is shown that the new methods provide excellent performance levels in terms of both mean square estimation error (which is very close to the benchmark Cram\'er-Rao Bound (CRB)) and signal-to-interference-plus-noise ratio, especially in comparison with state of the art strategies.Comment: submitted to IEEE Transactions on Signal Processing. arXiv admin note: substantial text overlap with arXiv:2110.1217

Beyond the noise : high fidelity MR signal processing

This thesis describes a variety of methods developed to increase the sensitivity and resolution of liquid state nuclear magnetic resonance (NMR) experiments. NMR is known as one of the most versatile non-invasive analytical techniques yet often suffers from low sensitivity. The main contribution to this low sensitivity issue is a presence of noise and level of noise in the spectrum is expressed numerically as “signal-to-noise ratio”. NMR signal processing involves sensitivity and resolution enhancement achieved by noise reduction using mathematical algorithms. A singular value decomposition based reduced rank matrix method, composite property mapping, in particular is studied extensively in this thesis to present its advantages, limitations, and applications. In theory, when the sum of k noiseless sinusoidal decays is formatted into a specific matrix form (i.e., Toeplitz), the matrix is known to possess k linearly independent columns. This information becomes apparent only after a singular value decomposition of the matrix. Singular value decomposition factorises the large matrix into three smaller submatrices: right and left singular vector matrices, and one diagonal matrix containing singular values. Were k noiseless sinusoidal decays involved, there would be only k nonzero singular values appearing in the diagonal matrix in descending order providing the information of the amplitude of each sinusoidal decay. The number of non-zero singular values or the number of linearly independent columns is known as the rank of the matrix. With real NMR data none of the singular values equals zero and the matrix has full rank. The reduction of the rank of the matrix and thus the noise in the reconstructed NMR data can be achieved by replacing all the singular values except the first k values with zeroes. This noise reduction process becomes difficult when biomolecular NMR data is to be processed due to the number of resonances being unknown and the presence of a large solvent peak

Multiple contrast tests with repeated and multiple endpoints : with biological applications

[no abstract

Advanced methods for analysing and modelling multivariate palaeoclimatic time series

The separation of natural and anthropogenically caused climatic changes is an important task of contemporary climate research. For this purpose, a detailed knowledge of the natural variability of the climate during warm stages is a necessary prerequisite. Beside model simulations and historical documents, this knowledge is mostly derived from analyses of so-called climatic proxy data like tree rings or sediment as well as ice cores. In order to be able to appropriately interpret such sources of palaeoclimatic information, suitable approaches of statistical modelling as well as methods of time series analysis are necessary, which are applicable to short, noisy, and non-stationary uni- and multivariate data sets. Correlations between different climatic proxy data within one or more climatological archives contain significant information about the climatic change on longer time scales. Based on an appropriate statistical decomposition of such multivariate time series, one may estimate dimensions in terms of the number of significant, linear independent components of the considered data set. In the presented work, a corresponding approach is introduced, critically discussed, and extended with respect to the analysis of palaeoclimatic time series. Temporal variations of the resulting measures allow to derive information about climatic changes ...thesi

Distinguishing cause from effect using observational data: methods and benchmarks

The discovery of causal relationships from purely observational data is a fundamental problem in science. The most elementary form of such a causal discovery problem is to decide whether X causes Y or, alternatively, Y causes X, given joint observations of two variables X, Y. An example is to decide whether altitude causes temperature, or vice versa, given only joint measurements of both variables. Even under the simplifying assumptions of no confounding, no feedback loops, and no selection bias, such bivariate causal discovery problems are challenging. Nevertheless, several approaches for addressing those problems have been proposed in recent years. We review two families of such methods: Additive Noise Methods (ANM) and Information Geometric Causal Inference (IGCI). We present the benchmark CauseEffectPairs that consists of data for 100 different cause-effect pairs selected from 37 datasets from various domains (e.g., meteorology, biology, medicine, engineering, economy, etc.) and motivate our decisions regarding the "ground truth" causal directions of all pairs. We evaluate the performance of several bivariate causal discovery methods on these real-world benchmark data and in addition on artificially simulated data. Our empirical results on real-world data indicate that certain methods are indeed able to distinguish cause from effect using only purely observational data, although more benchmark data would be needed to obtain statistically significant conclusions. One of the best performing methods overall is the additive-noise method originally proposed by Hoyer et al. (2009), which obtains an accuracy of 63+-10 % and an AUC of 0.74+-0.05 on the real-world benchmark. As the main theoretical contribution of this work we prove the consistency of that method.Comment: 101 pages, second revision submitted to Journal of Machine Learning Researc

Inference in high-dimensional time series models

Today’s world provides us with great potential in terms of data availability: “big data” is a term that very much circulates and many came across with. While having loads of data is a great opportunity to better understand the complexity of the real world, designing reliable statistical inference in such data-dense contexts requires careful modelling. Furthermore, when data such as time series is considered, the matter gets further complicated, given the inherent time dependency one needs to account for. This research develops statistical techniques aimed at both testing causal hypothesis and obtain forecasts in high-dimensional time series models. Applications of these techniques are provided in both finance, macroeconomics and climate econometrics, thus demonstrating the relevance of such tools across various sub-disciplines

An Integrated Approach to Performance Monitoring and Fault Diagnosis of Nuclear Power Systems

In this dissertation an integrated framework of process performance monitoring and fault diagnosis was developed for nuclear power systems using robust data driven model based methods, which comprises thermal hydraulic simulation, data driven modeling, identification of model uncertainty, and robust residual generator design for fault detection and isolation. In the applications to nuclear power systems, on the one hand, historical data are often not able to characterize the relationships among process variables because operating setpoints may change and thermal fluid components such as steam generators and heat exchangers may experience degradation. On the other hand, first-principle models always have uncertainty and are often too complicated in terms of model structure to design residual generators for fault diagnosis. Therefore, a realistic fault diagnosis method needs to combine the strength of first principle models in modeling a wide range of anticipated operation conditions and the strength of data driven modeling in feature extraction. In the developed robust data driven model-based approach, the changes in operation conditions are simulated using the first principle models and the model uncertainty is extracted from plant operation data such that the fault effects on process variables can be decoupled from model uncertainty and normal operation changes. It was found that the developed robust fault diagnosis method was able to eliminate false alarms due to model uncertainty and deal with changes in operating conditions throughout the lifetime of nuclear power systems. Multiple methods of robust data driven model based fault diagnosis were developed in this dissertation. A complete procedure based on causal graph theory and data reconciliation method was developed to investigate the causal relationships and the quantitative sensitivities among variables so that sensor placement could be optimized for fault diagnosis in the design phase. Reconstruction based Principal Component Analysis (PCA) approach was applied to deal with both simple faults and complex faults for steady state diagnosis in the context of operation scheduling and maintenance management. A robust PCA model-based method was developed to distinguish the differences between fault effects and model uncertainties. In order to improve the sensitivity of fault detection, a hybrid PCA model based approach was developed to incorporate system knowledge into data driven modeling. Subspace identification was proposed to extract state space models from thermal hydraulic simulations and a robust dynamic residual generator design algorithm was developed for fault diagnosis for the purpose of fault tolerant control and extension to reactor startup and load following operation conditions. The developed robust dynamic residual generator design algorithm is unique in that explicit identification of model uncertainty is not necessary. Finally, it was demonstrated that the developed new methods for the IRIS Helical Coil Steam Generator (HCSG) system. A simulation model was first developed for this system. It was revealed through steady state simulation that the primary coolant temperature profile could be used to indicate the water inventory inside the HCSG tubes. The performance monitoring and fault diagnosis module was then developed to monitor sensor faults, flow distribution abnormality, and heat performance degradation for both steady state and dynamic operation conditions. This dissertation bridges the gap between the theoretical research on computational intelligence and the engineering design in performance monitoring and fault diagnosis for nuclear power systems. The new algorithms have the potential of being integrated into the Generation III and Generation IV nuclear reactor I&C design after they are tested on current nuclear power plants or Generation IV prototype reactors

Longitudinal clinical covariates influence on CD4+ cell count after seroconversion.

Doctoral Degree. University of KwaZulu-Natal, Durban.The Acquired Immunodeficiency Syndrome (AIDS) pandemic is a global challenge. The human immunodeficiency virus (HIV) is notoriously known for weakening the immune system and opening channels for opportunistic infections. The Cluster of Difference 4 (CD4+) cells are mainly killed by the HIV and hence used as a health indicator for HIV infected patients. In the past, the CD4+ count diagnostics were very expensive and therefore beyond the reach of many in resource-limited settings. Accordingly, the CD4+ count’s clinical covariates were the potential diagnostic tools. From a different angle, it is essential to examine a trail of the clinical covariates effecting the CD4+ cell response. That is, inasmuch as the immune system regulates the CD4+ count fluctuations in reaction to the viral invasion, the body’s other complex functional systems are bound to adjust too. However, little is known about the corresponding adaptive behavioural patterns of the clinical covariates influence on the CD4+ cell count. The investigation in this study was carried out on data obtained from the Centre for the Programme of AIDS research in South Africa (CAPRISA), where initially, HIV negative patients were enrolled into different cohorts, for different objectives. These HIV negative patients were then followed up in their respective cohort studies. As soon as a patient seroconverted in any of the cohort studies, the patient was then enrolled again, into a new cohort of HIV positive patients only. The follow-up on the seroconvertants involved a simultaneous recording of repeated measurements of the CD4+ count and 46 clinical covariates. An extensive exploratory analysis was consequently performed with three variable reduction methods for high-dimensional longitudinal data to identify the strongest clinical covariates. The sparse partial least squares approach proved to be the most appropriate and a robust technique to adopt. It identified 18 strongest clinical covariates which were subsequently used to fit other sophisticated statistical models including the longitudinal multilevel models for assessing inter-individual variation in the CD4+ count due to each clinical covariate. Generalised additive mixed models were then used to gain insight into the CD4+ count trends and possible adaptive optimal set-points of the clinical covariates. To single out break-points in the change of linear relationships between the CD4+ count and the covariates, segmented regression models were employed. In getting to grips with the understanding of the highly complex and intertwined relationships between the CD4+ count, clinical covariates and the time lagged effects during the HIV disease progression, a Structural Equation Model (SEM) was constructed and fitted. The results showed that sodium consistently changed its effects at 132mEq/L and 140 mEq/L across all the post HIV infection phases. Generally, the covariate influence on the CD4+ count varied with infection phase and widely between individuals during the anti-retroviral therapy (ART). We conlude that there is evidence of covariate set-point adaptive behaviour to positively influence the CD4+ cell count during the HIV disease progression