Integrated Pre-Processing for Bayesian Nonlinear System Identification with Gaussian Processes

We introduce GP-FNARX: a new model for nonlinear system identification based on a nonlinear autoregressive exogenous model (NARX) with filtered regressors (F) where the nonlinear regression problem is tackled using sparse Gaussian processes (GP). We integrate data pre-processing with system identification into a fully automated procedure that goes from raw data to an identified model. Both pre-processing parameters and GP hyper-parameters are tuned by maximizing the marginal likelihood of the probabilistic model. We obtain a Bayesian model of the system's dynamics which is able to report its uncertainty in regions where the data is scarce. The automated approach, the modeling of uncertainty and its relatively low computational cost make of GP-FNARX a good candidate for applications in robotics and adaptive control.Comment: Proceedings of the 52th IEEE International Conference on Decision and Control (CDC), Firenze, Italy, December 201

Advances in model identification using the block-oriented exact solution technique in a predictive modeling framework

Obtaining an accurate model has always been a challenging objective in implementation of Model Predictive Control, especially for nonlinear processes. Part of this work here proposed a model building methodology for a complex block-oriented process, namely a Hammerstein-Wiener system in order to meet such a demand. It is a general system of the more simple structures which are known as Hammerstein and Wiener. This methodology uses sequential step test training data determined from an optimal experimental design and simultaneously estimates all the model coefficients under nonlinear least squares objective function. It is evaluated using four process examples and is compared with a recently proposed method in three of them. Even with less frequent sampling, the proposed method is demonstrated to have advantages in simplicity, the ability to model non-invertible systems, the ability to model multiple input and non-minimum phase processes, and accuracy.;This class of modeling method is also being applied to model normal operation plant data. The common problem seen in this type of dataset including high multi-collinearities of the inputs and low signal to noise ratios for the outputs inhibit modelers to acquire cause and effect relationship. The second part of the work here is to introduced this modeling approach that is capable of developing accurate cause and effect models. It is a special application of the Wiener block-oriented system and the unique and powerful attributes of this approach over existing techniques are demonstrated in a mathematically simulated processes and real processes

Bayes meets Bach: applications of Bayesian statistics to audio restoration

Memoryless nonlinear distortion can be present in audio signals, from recording to reproduction: bad quality or amateurishly operated equipments, physically degraded media and low quality reproducing devices are some examples where nonlinearities can naturally appear. Another quite common defect in old recordings are the long pulses, caused in general by the reproduction of disks with deep scratches or severely degraded magnetic tapes. Such defects are characterized by an initial discontinuity in the waveform, followed by a low-frequency transient of long duration. In both cases audible artifacts can be created, causing an unpleasant experience to the listener. It is then important to develop techniques to mitigate such defects, having at hand only the degraded signal, in a way to recover the original signal. In this thesis, techniques to deal with both problems are presented: the restoration of nonlinearly degraded recordings is tackled in a Bayesian context, considering both autoregressive models and sparsity in the DCT domain for the original signal, as well as through a deterministic solution also based on sparsity; for the suppression of long pulses, a parametric approach is revisited with the addition of an efficient initialization procedure, and a nonparametric modeling via Gaussian process is also presented.Distorções não-lineares podem aparecer em sinais de áudio desde o momento da sua gravação até a posterior reprodução: equipamentos precários ou operados de maneira indevida, mídias fisicamente degradadas e baixa qualidade dos aparelhos de reprodução são somente alguns exemplos onde não-linearidades podem aparecer de modo natural. Outro defeito bastante comum em gravações antigas são os pulsos longos, em geral causados pela reprodução de discos com arranhões muito profundos ou fitas magnéticas severamente degradadas. Tais defeitos são caracterizados por uma descontinuidade inicial na forma de onda, seguida de um transitório de baixa frequência e longa duração. Em ambos os casos, artefatos auditivos podem ser criados, causando assim uma experiência ruim para o ouvinte. E importante então desenvolver técnicas para mitigar tais efeitos, tendo como base somente uma versão do sinal degradado, de modo a recuperar o sinal original não degradado. Nessa tese são apresentadas técnicas para lidar com esses dois problemas: o problema de restaurar gravações corrompidas com distorções não-lineares é abordado em um contexto bayesiano, considerando tanto modelos autorregressivos quanto de esparsidade no domínio da DCT para o sinal original, bem como por uma solução determinística também em usando esparsidade; para a supressão de pulsos longos, uma abordagem paramétrica é revisitada, junto com o acréscimo de um eficiente procedimento de inicialização, sendo também apresentada uma abordagem não-paramétricausando processos gaussianos

Performance Analysis of Fractional Learning Algorithms

Fractional learning algorithms are trending in signal processing and adaptive filtering recently. However, it is unclear whether the proclaimed superiority over conventional algorithms is well-grounded or is a myth as their performance has never been extensively analyzed. In this article, a rigorous analysis of fractional variants of the least mean squares and steepest descent algorithms is performed. Some critical schematic kinks in fractional learning algorithms are identified. Their origins and consequences on the performance of the learning algorithms are discussed and swift ready-witted remedies are proposed. Apposite numerical experiments are conducted to discuss the convergence and efficiency of the fractional learning algorithms in stochastic environments.Comment: 29 pages, 6 figure

Inference techniques for stochastic nonlinear system identification with application to the Wiener-Hammerstein models

Stochastic nonlinear systems are a specific class of nonlinear systems where unknown disturbances affect the system\u27s output through a nonlinear transformation. In general, the identification of parametric models for this kind of systems can be very challenging. A main statistical inference technique for parameter estimation is the Maximum Likelihood estimator. The central object of this technique is the likelihood function, i.e. a mathematical expression describing the probability of obtaining certain observations for given values of the parameter. For many stochastic nonlinear systems, however, the likelihood function is not available in closed-form. Several methods have been developed to obtain approximate solutions to the Maximum Likelihood problem, mainly based on the Monte Carlo method. However, one of the main difficulties of these methods is that they can be computationally expensive, especially when they are combined with numerical optimization techniques for likelihood maximisation.This thesis can be divided in three parts. In the first part, a background on the main statistical techniques for parameter estimation is presented. In particular, two iterative methods for finding the Maximum Likelihood estimator are introduced. They are the gradient-based and the Expectation-Maximisation algorithms.In the second part, the main Monte Carlo methods for approximating the Maximum Likelihood problem are analysed. Their combinations with gradient-based and Expectation-Maximisation algorithms is considered. For ensuring convergence, these algorithms require the use of enormous Monte Carlo effort, i.e. the number of random samples used to build the Monte Carlo estimates. In order to reduce this effort and make the algorithms usable in practice, iterative solutions solutions alternating \emph{local} Monte Carlo approximations and maximisation steps are derived. In particular, a procedure implementing an efficient samples simulation across the steps of a Newton\u27s method is developed. The procedure is based on the sensitivity of the parameter search with respect to the Monte Carlo samples and it results into an accurate and fast algorithm for solving the MLE problem.The considered Maximum Likelihood estimation methods proceed through local explorations of the parameter space. Hence, they have guaranteed convergence only to a local optimizer of the likelihood function. In the third part of the thesis, this issue is addressed by deriving initialization algorithms. The purpose is to generate initial guesses that increase the chances of converging to the global maximum. In particular, initialization algorithms are derived for the Wiener-Hammerstein model, i.e. a nonlinear model where a static nonlinearity is sandwiched between two linear parts. For this type of model, it can be proved that the best linear approximation of the system provides a consistent estimates of the two linear parts. This estimate is then used to initialize a Maximum Likelihood Estimation problem in all model parameters

A Perceptual Comparison of “Black Box” Modeling Algorithms for Nonlinear Audio Systems

Nonlinear systems identification is a widespread topic of interest, particularly within the audio industry, as these techniques are employed to synthesize black box models of nonlinear audio effects. Given the myriad approaches to black box modeling, questions arise as to whether an “optimal” approach exists, or one that achieves valid subjective results as a model with minimal computational expense. This thesis uses ABX listening tests to compare black box models of three hardware audio effects using two popular nonlinear implementations, along with two proposed modified implementations. Models were constructed in the Hammerstein form using sine sweeps and a novel measurement technique for the filters and nonlinearities, respectively. Testing revolved around null hypotheses assuming no change in model identification regardless of the device modeled, implementation used, or program material of the model stimulus. Results provide clear evidence of an effect on all of these accounts, and support a full rejection of the null hypotheses. Outcomes demonstrate a preferable implementation out of the algorithms tested, and suggest the removal of certain implementations as valid approaches altogether