A comprehensive expectation identification framework for multirate time-delayed systems

The expectation maximization (EM) algorithm has been extensively used to solve system identification problems with hidden variables. It needs to calculate a derivative equation and perform a matrix inversion in the EM-M step. The equations related to the EM algorithm may be unsolvable for some complex nonlinear systems, and the matrix inversion has heavy computational costs for large-scale systems. This article provides two expectation-based algorithms with the aim of constructing a comprehensive expectation framework concerning different kinds of time-delayed systems: 1) for a small-scale linear system, the classical EM algorithm can quickly obtain the parameter and time-delay estimates; 2) for a complex nonlinear system with low order, the proposed expectation gradient descent algorithm can avoid derivative function calculation; 3) for a large-scale system, the proposed expectation multidirection algorithm does not require eigenvalue calculation and has less computational costs. These two algorithms are developed based on the gradient descent and multidirection methods. Under such an expectation framework, different kinds of models are identified on a case-by-case basis. The convergence analysis and simulation examples show the effectiveness of the algorithms

Finite Impulse Response Errors-in-Variables system identification utilizing Approximated Likelihood and Gaussian Mixture Models

In this paper a Maximum likelihood estimation algorithm for Finite Impulse Response Errors-in-Variables systems is developed. We consider that the noise-free input signal is Gaussian-mixture distributed. We propose an Expectation-Maximization-based algorithm to estimate the system model parameters, the input and output noise variances, and the Gaussian mixture noise-free input parameters. The benefits of our proposal are illustrated via numerical simulation

Two iterative reweighted algorithms for systems contaminated by outliers

This study proposes two iterative reweighted (IRE) algorithms for systems whose data are contaminated by outliers. For the negative effect caused by the outliers, traditional least squares (LSs) and gradient descent (GD) algorithms cannot obtain unbiased estimates, while the variational Bayesian (VB) and expectation–maximization (EM) algorithms have the assumption that the prior knowledge of the outlier is available. To deal with these dilemmas, two IRE algorithms are developed. By assigning suitable weights for each dataset, unbiased parameter estimates can be obtained. In addition, the weights of the corrupted datasets become smaller and smaller with the increased number of iterations, and then, the contaminated data can be picked out from the datasets. The proposed algorithms do not require the prior knowledge of the outliers. Convergence analysis and numerical experiments show the effectiveness of the IRE algorithms

Inference techniques for stochastic nonlinear system identification with application to the Wiener-Hammerstein models

Stochastic nonlinear systems are a specific class of nonlinear systems where unknown disturbances affect the system\u27s output through a nonlinear transformation. In general, the identification of parametric models for this kind of systems can be very challenging. A main statistical inference technique for parameter estimation is the Maximum Likelihood estimator. The central object of this technique is the likelihood function, i.e. a mathematical expression describing the probability of obtaining certain observations for given values of the parameter. For many stochastic nonlinear systems, however, the likelihood function is not available in closed-form. Several methods have been developed to obtain approximate solutions to the Maximum Likelihood problem, mainly based on the Monte Carlo method. However, one of the main difficulties of these methods is that they can be computationally expensive, especially when they are combined with numerical optimization techniques for likelihood maximisation.This thesis can be divided in three parts. In the first part, a background on the main statistical techniques for parameter estimation is presented. In particular, two iterative methods for finding the Maximum Likelihood estimator are introduced. They are the gradient-based and the Expectation-Maximisation algorithms.In the second part, the main Monte Carlo methods for approximating the Maximum Likelihood problem are analysed. Their combinations with gradient-based and Expectation-Maximisation algorithms is considered. For ensuring convergence, these algorithms require the use of enormous Monte Carlo effort, i.e. the number of random samples used to build the Monte Carlo estimates. In order to reduce this effort and make the algorithms usable in practice, iterative solutions solutions alternating \emph{local} Monte Carlo approximations and maximisation steps are derived. In particular, a procedure implementing an efficient samples simulation across the steps of a Newton\u27s method is developed. The procedure is based on the sensitivity of the parameter search with respect to the Monte Carlo samples and it results into an accurate and fast algorithm for solving the MLE problem.The considered Maximum Likelihood estimation methods proceed through local explorations of the parameter space. Hence, they have guaranteed convergence only to a local optimizer of the likelihood function. In the third part of the thesis, this issue is addressed by deriving initialization algorithms. The purpose is to generate initial guesses that increase the chances of converging to the global maximum. In particular, initialization algorithms are derived for the Wiener-Hammerstein model, i.e. a nonlinear model where a static nonlinearity is sandwiched between two linear parts. For this type of model, it can be proved that the best linear approximation of the system provides a consistent estimates of the two linear parts. This estimate is then used to initialize a Maximum Likelihood Estimation problem in all model parameters

Identification of systems from multirate data

Master'sMASTER OF ENGINEERIN

WAVEFORM AND TRANSCEIVER OPTIMIZATION FOR MULTI-FUNCTIONAL AIRBORNE RADAR THROUGH ADAPTIVE PROCESSING

Pulse compression techniques have been widely used for target detection and remote sensing. The primary concern for pulse compression is the sidelobe interference. Waveform design is an important method to improve the sidelobe performance. As a multi-functional aircraft platform in aviation safety domain, ADS-B system performs functions involving detection, localization and alerting of external traffic. In this work, a binary phase modulation is introduced to convert the original 1090 MHz ADS-B signal waveform into a radar signal. Both the statistical and deterministic models of new waveform are developed and analyzed. The waveform characterization, optimization and its application are studied in details. An alternative way to achieve low sidelobe levels without trading o range resolution and SNR is the adaptive pulse compression - RMMSE (Reiterative Minimum Mean-Square error). Theoretically, RMMSE is able to suppress the sidelobe level down to the receiver noise floor. However, the application of RMMSE to actual radars and the related implementation issues have not been investigated before. In this work, implementation aspects of RMMSE such as waveform sensitivity, noise immunity and computational complexity are addressed. Results generated by applying RMMSE to both simulated and measured radar data are presented and analyzed. Furthermore, a two-dimensional RMMSE algorithm is derived to mitigate the sidelobe effects from both pulse compression processing and antenna radiation pattern. In addition, to achieve even better control of the sidelobe level, a joint transmit and receive optimization scheme (JTRO) is proposed, which reduces the impacts of HPA nonlinearity and receiver distortion. Experiment results obtained with a Ku-band spaceborne radar transceiver testbed are presented

Convex Identifcation of Stable Dynamical Systems

This thesis concerns the scalable application of convex optimization to data-driven modeling of dynamical systems, termed system identi cation in the control community. Two problems commonly arising in system identi cation are model instability (e.g. unreliability of long-term, open-loop predictions), and nonconvexity of quality-of- t criteria, such as simulation error (a.k.a. output error). To address these problems, this thesis presents convex parametrizations of stable dynamical systems, convex quality-of- t criteria, and e cient algorithms to optimize the latter over the former. In particular, this thesis makes extensive use of Lagrangian relaxation, a technique for generating convex approximations to nonconvex optimization problems. Recently, Lagrangian relaxation has been used to approximate simulation error and guarantee nonlinear model stability via semide nite programming (SDP), however, the resulting SDPs have large dimension, limiting their practical utility. The rst contribution of this thesis is a custom interior point algorithm that exploits structure in the problem to signi cantly reduce computational complexity. The new algorithm enables empirical comparisons to established methods including Nonlinear ARX, in which superior generalization to new data is demonstrated. Equipped with this algorithmic machinery, the second contribution of this thesis is the incorporation of model stability constraints into the maximum likelihood framework. Speci - cally, Lagrangian relaxation is combined with the expectation maximization (EM) algorithm to derive tight bounds on the likelihood function, that can be optimized over a convex parametrization of all stable linear dynamical systems. Two di erent formulations are presented, one of which gives higher delity bounds when disturbances (a.k.a. process noise) dominate measurement noise, and vice versa. Finally, identi cation of positive systems is considered. Such systems enjoy substantially simpler stability and performance analysis compared to the general linear time-invariant iv Abstract (LTI) case, and appear frequently in applications where physical constraints imply nonnegativity of the quantities of interest. Lagrangian relaxation is used to derive new convex parametrizations of stable positive systems and quality-of- t criteria, and substantial improvements in accuracy of the identi ed models, compared to existing approaches based on weighted equation error, are demonstrated. Furthermore, the convex parametrizations of stable systems based on linear Lyapunov functions are shown to be amenable to distributed optimization, which is useful for identi cation of large-scale networked dynamical systems

System approach to robust acoustic echo cancellation through semi-blind source separation based on independent component analysis

We live in a dynamic world full of noises and interferences. The conventional acoustic echo cancellation (AEC) framework based on the least mean square (LMS) algorithm by itself lacks the ability to handle many secondary signals that interfere with the adaptive filtering process, e.g., local speech and background noise. In this dissertation, we build a foundation for what we refer to as the system approach to signal enhancement as we focus on the AEC problem. We first propose the residual echo enhancement (REE) technique that utilizes the error recovery nonlinearity (ERN) to "enhances" the filter estimation error prior to the filter adaptation. The single-channel AEC problem can be viewed as a special case of semi-blind source separation (SBSS) where one of the source signals is partially known, i.e., the far-end microphone signal that generates the near-end acoustic echo. SBSS optimized via independent component analysis (ICA) leads to the system combination of the LMS algorithm with the ERN that allows for continuous and stable adaptation even during double talk. Second, we extend the system perspective to the decorrelation problem for AEC, where we show that the REE procedure can be applied effectively in a multi-channel AEC (MCAEC) setting to indirectly assist the recovery of lost AEC performance due to inter-channel correlation, known generally as the "non-uniqueness" problem. We develop a novel, computationally efficient technique of frequency-domain resampling (FDR) that effectively alleviates the non-uniqueness problem directly while introducing minimal distortion to signal quality and statistics. We also apply the system approach to the multi-delay filter (MDF) that suffers from the inter-block correlation problem. Finally, we generalize the MCAEC problem in the SBSS framework and discuss many issues related to the implementation of an SBSS system. We propose a constrained batch-online implementation of SBSS that stabilizes the convergence behavior even in the worst case scenario of a single far-end talker along with the non-uniqueness condition on the far-end mixing system. The proposed techniques are developed from a pragmatic standpoint, motivated by real-world problems in acoustic and audio signal processing. Generalization of the orthogonality principle to the system level of an AEC problem allows us to relate AEC to source separation that seeks to maximize the independence, hence implicitly the orthogonality, not only between the error signal and the far-end signal, but rather, among all signals involved. The system approach, for which the REE paradigm is just one realization, enables the encompassing of many traditional signal enhancement techniques in analytically consistent yet practically effective manner for solving the enhancement problem in a very noisy and disruptive acoustic mixing environment.PhDCommittee Chair: Biing-Hwang Juang; Committee Member: Brani Vidakovic; Committee Member: David V. Anderson; Committee Member: Jeff S. Shamma; Committee Member: Xiaoli M

Deep Learning-Based Machinery Fault Diagnostics

This book offers a compilation for experts, scholars, and researchers to present the most recent advancements, from theoretical methods to the applications of sophisticated fault diagnosis techniques. The deep learning methods for analyzing and testing complex mechanical systems are of particular interest. Special attention is given to the representation and analysis of system information, operating condition monitoring, the establishment of technical standards, and scientific support of machinery fault diagnosis