Training a Linear Neural Network with a Stable LSP Solution for Jamming Cancellation

Two jamming cancellation algorithms are developed based on a stable solution of least squares problem (LSP) provided by regularization. They are based on filtered singular value decomposition (SVD) and modifications of the Greville formula. Both algorithms allow an efficient hardware implementation. Testing results on artificial data modeling difficult real-world situations are also provided

A WEIGHTED LEAST SQUARES B-SPLINE COLLOCATION METHOD FOR ELLIPTIC PARTIAL DIFFERENTIAL EQUATIONS

A new, free-integration approach based upon B-spline for solving boundary value problem is introduced and presented in the paper, called weighted least squares B-spline collocation method. It combines high order B-spline basis functions with high approximation and continuity properties and weighted least squares method which is robust to deal with scattered or randomly knot points distribution. In addition, using appropriate designed B-spline basis function construction, the new approach introduces no difficulties in imposing both Dirichlet and Neumann boundary conditions in the problem domain. As a result, the effectiveness of the new approach is greatly enhanced with the flexibility to cope with both regular and irregular shaped domains. Numerical examples show the applicability and capability of the new approach for solving elliptic partial differential equations in arbitrary domains

Sliding window identification with linear-equality constraints

In this paper, we present a new algorithm of sliding window identification with linear-equality constraints. This algorithm consists in firstly deleting the oldest set of data and in secondly adding the last set of data. The method developed in this paper allows to consider at every step a set of new data by an extension of their result. The proposed algorithm is based on the recursive calculus of the pseudo-inverse matrix from the forms of Albert and Sittler. A simple and easily implementable initialization of the constrained algorithm is proposed. An improvement is obtained by removing the influence of oldest set of data and by satisfying the linear-equality constraints. It is shown that the solutions of the sliding window identification algorithm converge to the true parameter that satisfies the equality constraints. Numerical example is provided to show the effectiveness of the proposed method

Completely Recursive Least Squares and Its Applications

The matrix-inversion-lemma based recursive least squares (RLS) approach is of a recursive form and free of matrix inversion, and has excellent performance regarding computation and memory in solving the classic least-squares (LS) problem. It is important to generalize RLS for generalized LS (GLS) problem. It is also of value to develop an efficient initialization for any RLS algorithm. In Chapter 2, we develop a unified RLS procedure to solve the unconstrained/linear-equality (LE) constrained GLS. We also show that the LE constraint is in essence a set of special error-free observations and further consider the GLS with implicit LE constraint in observations (ILE-constrained GLS). Chapter 3 treats the RLS initialization-related issues, including rank check, a convenient method to compute the involved matrix inverse/pseudoinverse, and resolution of underdetermined systems. Based on auxiliary-observations, the RLS recursion can start from the first real observation and possible LE constraints are also imposed recursively. The rank of the system is checked implicitly. If the rank is deficient, a set of refined non-redundant observations is determined alternatively. In Chapter 4, base on [Li07], we show that the linear minimum mean square error (LMMSE) estimator, as well as the optimal Kalman filter (KF) considering various correlations, can be calculated from solving an equivalent GLS using the unified RLS. In Chapters 5 & 6, an approach of joint state-and-parameter estimation (JSPE) in power system monitored by synchrophasors is adopted, where the original nonlinear parameter problem is reformulated as two loosely-coupled linear subproblems: state tracking and parameter tracking. Chapter 5 deals with the state tracking which determines the voltages in JSPE, where dynamic behavior of voltages under possible abrupt changes is studied. Chapter 6 focuses on the subproblem of parameter tracking in JSPE, where a new prediction model for parameters with moving means is introduced. Adaptive filters are developed for the above two subproblems, respectively, and both filters are based on the optimal KF accounting for various correlations. Simulations indicate that the proposed approach yields accurate parameter estimates and improves the accuracy of the state estimation, compared with existing methods

Sparse representations and harmonic wavelets for stochastic modeling and analysis of diverse structural systems and related excitations

In this thesis, novel analytical and computational approaches are proposed for addressing several topics in the field of random vibration. The first topic pertains to the stochastic response determination of systems with singular parameter matrices. Such systems appear, indicatively, when a redundant coordinate modeling scheme is adopted. This is often associated with computational cost-efficient solution frameworks and modeling flexibility for treating complex systems. Further, structures are subject to environmental excitations, such as ground motions, that typically exhibit non-stationary characteristics. In this regard, aiming at a joint time-frequency analysis of the system response a recently developed generalized harmonic wavelet (GHW)-based solution framework is employed in conjunction with tools originated form the generalized matrix inverse theory. This leads to a generalization of earlier excitation-response relationships of random vibration theory to account for systems with singular matrices. Harmonic wavelet-based statistical linearization techniques are also extended to nonlinear multi-degree-of-freedom (MDOF) systems with singular matrices. The accuracy of the herein proposed framework is further improved by circumventing previous “local stationarity” assumptions about the response. Furthermore, the applicability of the method is extended beyond redundant coordinate modeling applications. This is achieved by a formulation which accounts for generally constrained equations of motion pertaining to diverse engineering applications. These include, indicatively, energy harvesters with coupled electromechanical equations and oscillators subject to non-white excitations modeled via auxiliary filter equations. The second topic relates to the probabilistic modeling of excitation processes in the presence of missing data. In this regard, a compressive sampling methodology is developed for incomplete wind time-histories reconstruction and extrapolation in a single spatial dimension, as well as for related stochastic field statistics estimation. An alternative methodology based on low rank matrices and nuclear norm minimization is also developed for wind field extrapolation in two spatial dimensions. The proposed framework can be employed for monitoring of wind turbine systems utilizing information from a few measured locations as well as in the context of performance-based design optimization of structural systems. Lastly, the problem of with data-driven sparse identification methods of nonlinear dynamics is considered. In particular, utilizing measured responses a Bayesian compressive sampling technique is developed for determining the governing equations of stochastically excited structural systems exhibiting diverse nonlinear behaviors and also endowed with fractional derivative elements. Compared to alternative state-of-the-art schemes that yield deterministic estimates for the identified model, the herein developed methodology exhibits additional sparsity promoting features and is capable of quantifying the uncertainty associated with the model estimates. This provides a quantifiable degree of confidence when employing the proposed framework as a predictive tool

Reducing Communication in the Solution of Linear Systems

There is a growing performance gap between computation and communication on modern computers, making it crucial to develop algorithms with lower latency and bandwidth requirements. Because systems of linear equations are important for numerous scientific and engineering applications, I have studied several approaches for reducing communication in those problems. First, I developed optimizations to dense LU with partial pivoting, which downstream applications can adopt with little to no effort. Second, I consider two techniques to completely replace pivoting in dense LU, which can provide significantly higher speedups, albeit without the same numerical guarantees as partial pivoting. One technique uses randomized preprocessing, while the other is a novel combination of block factorization and additive perturbation. Finally, I investigate using mixed precision in GMRES for solving sparse systems, which reduces the volume of data movement, and thus, the pressure on the memory bandwidth

Concept Graph Learning from Educational Data

This paper addresses an open challenge in educational data mining, i.e., the problem of using observed prerequisite re-lations among courses to learn a directed universal con-cept graph, and using the induced graph to predict un-observed prerequisite relations among a broader range of courses. This is particularly useful to induce prerequisite relations among courses from different providers (universi-ties, MOOCs, etc.). We propose a new framework for in-ference within and across two graphs—at the course level and at the induced concept level—which we call Concept Graph Learning (CGL). In the training phase, our system projects the course-level links onto the concept space to in-duce directed concept links; in the testing phase, the concept links are used to predict (unobserved) prerequisite links for test-set courses within the same institution or across insti-tutions. The dual mappings enable our system to perform an interlingua-style transfer learning, e.g. treating the con-cept graph as the interlingua, and inducing prerequisite links in a transferable manner across different universities. Ex-periments on our newly collected data sets of courses fro

Towards a more efficient and cost-sensitive extreme learning machine: A state-of-the-art review of recent trend

In spite of the prominence of extreme learning machine model, as well as its excellent features such as insignificant intervention for learning and model tuning, the simplicity of implementation, and high learning speed, which makes it a fascinating alternative method for Artificial Intelligence, including Big Data Analytics, it is still limited in certain aspects. These aspects must be treated to achieve an effective and cost-sensitive model. This review discussed the major drawbacks of ELM, which include difficulty in determination of hidden layer structure, prediction instability and Imbalanced data distributions, the poor capability of sample structure preserving (SSP), and difficulty in accommodating lateral inhibition by direct random feature mapping. Other drawbacks include multi-graph complexity, global memory size, one-by-one or chuck-by-chuck (a block of data), global memory size limitation, and challenges with big data. The recent trend proposed by experts for each drawback is discussed in detail towards achieving an effective and cost-sensitive mode