The wavelet-NARMAX representation : a hybrid model structure combining polynomial models with multiresolution wavelet decompositions

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory, and have been extensively used in linear and nonlinear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilise the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model

Ultra-Orthogonal Forward Regression Algorithms for the Identification of Non-Linear Dynamic Systems

A new ultra-least squares (ULS) criterion is introduced for system identification. Unlike the standard least squares criterion which is based on the Euclidean norm of the residuals, the new ULS criterion is derived from the Sobolev space norm. The new criterion measures not only the discrepancy between the observed signals and the model prediction but also the discrepancy between the associated weak derivatives of the observed and the model signals. The new ULS criterion possesses a clear physical interpretation and is easy to implement. Based on this, a new Ultra-Orthogonal Forward Regression (UOFR) algorithm is introduced for nonlinear system identification, which includes converting a least squares regression problem into the associated ultra-least squares problem and solving the ultra-least squares problem using the orthogonal forward regression method. Numerical simulations show that the new UOFR algorithm can significantly improve the performance of the classic OFR algorithm

Time-varying nonlinear causality detection using regularized orthogonal least squares and multi-wavelets with applications to EEG

A new transient Granger causality detection method is proposed based on a time-varying parametric modelling framework, and is applied to real EEG signals to reveal the causal information flow during motor imagery (MI) tasks. The time-varying parametric modelling approach employs a nonlinear autoregressive with external input (NARX) model, whose parameters are approximated by a set of multiwavelet basis functions. A regularized orthogonal least squares (ROLS) algorithm is then used to produce a parsimonious or sparse regression model and estimate the associated model parameters. The time-varying Granger causality between nonstationary signals can be detected accurately by making use of both the good approximation properties of multi-wavelets and the good generalization performance of the ROLS in the presence of high-level noise. Two simulation examples are presented to demonstrate the effectiveness of the proposed method for both linear and nonlinear causal detection respectively. The proposed method is then applied to real EEG signals of MI tasks. It follows that transient causal information flow over the time course between various sensorimotor related channels can be successfully revealed during the whole reaction processes. Experiment results from these case studies confirm the applicability of the proposed scheme and show its utility for the understanding of the associated neural mechanism and the potential significance for developing MI tasks based brain-computer interface (BCI) systems

Adaptive kernel estimation for enhanced filtering and pattern classification of magnetic resonance imaging: novel techniques for evaluating the biomechanics and pathologic conditions of the lumbar spine

This dissertation investigates the contribution the lumbar spine musculature has on etiological and pathogenic characteristics of low back pain and lumbar spondylosis. This endeavor necessarily required a two-step process: 1) design of an accurate post-processing method for extracting relevant information via magnetic resonance images and 2) determine pathological trends by elucidating high-dimensional datasets through multivariate pattern classification. The lumbar musculature was initially evaluated by post-processing and segmentation of magnetic resonance (MR) images of the lumbar spine, which characteristically suffer from nonlinear corruption of the signal intensity. This so called intensity inhomogeneity degrades the efficacy of traditional intensity-based segmentation algorithms. Proposed in this dissertation is a solution for filtering individual MR images by extracting a map of the underlying intensity inhomogeneity to adaptively generate local estimates of the kernel’s optimal bandwidth. The adaptive kernel is implemented and tested within the structure of the non-local means filter, but also generalized and extended to the Gaussian and anisotropic diffusion filters. Testing of the proposed filters showed that the adaptive kernel significantly outperformed their non-adaptive counterparts. A variety of performance metrics were utilized to measure either fine feature preservation or accuracy of post-processed segmentation. Based on these metrics the adaptive filters proposed in this dissertation significantly outperformed the non-adaptive versions. Using the proposed filter, the MR data was semi-automatically segmented to delineate between adipose and lean muscle tissues. Two important findings were reached utilizing this data. First, a clear distinction between the musculature of males and females was established that provided 100% accuracy in being able to predict gender. Second, degenerative lumbar spines were accurately predicted at a rate of up to 92% accuracy. These results solidify prior assumptions made regarding sexual dimorphic anatomy and the pathogenic nature of degenerative spine disease

Regularized System Identification

This open access book provides a comprehensive treatment of recent developments in kernel-based identification that are of interest to anyone engaged in learning dynamic systems from data. The reader is led step by step into understanding of a novel paradigm that leverages the power of machine learning without losing sight of the system-theoretical principles of black-box identification. The authors’ reformulation of the identification problem in the light of regularization theory not only offers new insight on classical questions, but paves the way to new and powerful algorithms for a variety of linear and nonlinear problems. Regression methods such as regularization networks and support vector machines are the basis of techniques that extend the function-estimation problem to the estimation of dynamic models. Many examples, also from real-world applications, illustrate the comparative advantages of the new nonparametric approach with respect to classic parametric prediction error methods. The challenges it addresses lie at the intersection of several disciplines so Regularized System Identification will be of interest to a variety of researchers and practitioners in the areas of control systems, machine learning, statistics, and data science. This is an open access book

Novel methods for biological network inference: an application to circadian Ca2+ signaling network

Biological processes involve complex biochemical interactions among a large number of species like cells, RNA, proteins and metabolites. Learning these interactions is essential to interfering artificially with biological processes in order to, for example, improve crop yield, develop new therapies, and predict new cell or organism behaviors to genetic or environmental perturbations. For a biological process, two pieces of information are of most interest. For a particular species, the first step is to learn which other species are regulating it. This reveals topology and causality. The second step involves learning the precise mechanisms of how this regulation occurs. This step reveals the dynamics of the system. Applying this process to all species leads to the complete dynamical network. Systems biology is making considerable efforts to learn biological networks at low experimental costs. The main goal of this thesis is to develop advanced methods to build models for biological networks, taking the circadian system of Arabidopsis thaliana as a case study. A variety of network inference approaches have been proposed in the literature to study dynamic biological networks. However, many successful methods either require prior knowledge of the system or focus more on topology. This thesis presents novel methods that identify both network topology and dynamics, and do not depend on prior knowledge. Hence, the proposed methods are applicable to general biological networks. These methods are initially developed for linear systems, and, at the cost of higher computational complexity, can also be applied to nonlinear systems. Overall, we propose four methods with increasing computational complexity: one-to-one, combined group and element sparse Bayesian learning (GESBL), the kernel method and reversible jump Markov chain Monte Carlo method (RJMCMC). All methods are tested with challenging dynamical network simulations (including feedback, random networks, different levels of noise and number of samples), and realistic models of circadian system of Arabidopsis thaliana. These simulations show that, while the one-to-one method scales to the whole genome, the kernel method and RJMCMC method are superior for smaller networks. They are robust to tuning variables and able to provide stable performance. The simulations also imply the advantage of GESBL and RJMCMC over the state-of-the-art method. We envision that the estimated models can benefit a wide range of research. For example, they can locate biological compounds responsible for human disease through mathematical analysis and help predict the effectiveness of new treatments

A Data-Driven Toolchain for the Operational Performance Analysis and Optimization of Buildings

Automation has proven indispensable to advancing human endeavors. Within the built environment its evolution and sophistication are on the cusp of moving beyond automatic control into automated prediction and diagnosis. A data-driven toolchain is developed so human efforts can be focused on high-value concerns. The research examines smart buildings as a cyberphysical construct and places the Bayesian perspective as paramount. Prior knowledge is leveraged through common building energy modeling and simulation tools, which are utilized and extended. An iterative, three-step process is developed to 1) classify building energy performance scenarios, 2) forecast dynamics over a planning horizon of interest, and 3) signal human decision-makers concerning deviations from ideal behavior. In the classification step, focus is placed on the discrete wavelet transformation of electrical demand profiles, producing energy and entropy feature extraction from the wavelet levels at definitive time frames, and Bayesian probabilistic hierarchical clustering. The process yields a categorized and manageable set of representative electrical demand profiles for smart grid applications. In the forecasting step, a cyclical two-stage model predictive control process of policy planning followed by execution is evaluated. The results show that even the most complicated nonlinear autoregressive neural network with exogenous input does not appear to warrant the additional efforts in forecasting model development and training in comparison to the simpler models. In the signaling step, a simulation study is considered to assess whole-building energy signaling accuracy in the presence of uncertainty and faults at the submetered level, which may lead to tradeoffs at the whole-building level that are not detectable without submetering. Together, the steps form a data-driven toolchain for the operational performance analysis and optimization of buildings

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level