Least squares support vector machine with self-organizing multiple kernel learning and sparsity

© 2018 In recent years, least squares support vector machines (LSSVMs) with various kernel functions have been widely used in the field of machine learning. However, the selection of kernel functions is often ignored in practice. In this paper, an improved LSSVM method based on self-organizing multiple kernel learning is proposed for black-box problems. To strengthen the generalization ability of the LSSVM, some appropriate kernel functions are selected and the corresponding model parameters are optimized using a differential evolution algorithm based on an improved mutation strategy. Due to the large computation cost, a sparse selection strategy is developed to extract useful data and remove redundant data without loss of accuracy. To demonstrate the effectiveness of the proposed method, some benchmark problems from the UCI machine learning repository are tested. The results show that the proposed method performs better than other state-of-the-art methods. In addition, to verify the practicability of the proposed method, it is applied to a real-world converter steelmaking process. The results illustrate that the proposed model can precisely predict the molten steel quality and satisfy the actual production demand

A Sparse Learning Machine for Real-Time SOC Estimation of Li-ion Batteries

The state of charge (SOC) estimation of Li-ion batteries has attracted substantial interests in recent years. Kalman Filter has been widely used in real-time battery SOC estimation, however, to build a suitable dynamic battery state-space model is a key challenge, and most existing methods still use the off-line modelling approach. This paper tackles the challenge by proposing a novel sparse learning machine for real-time SOC estimation. This is achieved first by developing a new learning machine based on the traditional least squares support vector machine (LS-SVM) to capture the process dynamics of Li-ion batteries in real-time. The least squares support vector machine is the least squares version of the conventional support vector machines (SVMs) which suffers from low model sparseness. The proposed learning machine reduces the dimension of the projected high dimensional feature space with no loss of input information, leading to improved model sparsity and accuracy. To accelerate computation, mapping functions in the high feature space are selected using a fast recursive method. To further improve the model accuracy, a weighted regularization scheme and the differential evolution (DE) method are used to optimize the parameters. Then, an unscented Kalman filter (UKF) is used for real-time SOC estimation based on the proposed sparse learning machine model. Experimental results on the Federal Urban Drive Schedule (FUDS) test data reveal that the performance of the proposed algorithm is significantly enhanced, where the maximum absolute error is only one sixth of that obtained by the conventional LS-SVMs and the mean square error of the SOC estimations reaches to 10 −7 , while the proposed method is executed nearly 10 times faster than the conventional LS-SVMs

Direct L2 Support Vector Machine

This dissertation introduces a novel model for solving the L2 support vector machine dubbed Direct L2 Support Vector Machine (DL2 SVM). DL2 SVM represents a new classification model that transforms the SVM\u27s underlying quadratic programming problem into a system of linear equations with nonnegativity constraints. The devised system of linear equations has a symmetric positive definite matrix and a solution vector has to be nonnegative. Furthermore, this dissertation introduces a novel algorithm dubbed Non-Negative Iterative Single Data Algorithm (NN ISDA) which solves the underlying DL2 SVM\u27s constrained system of equations. This solver shows significant speedup compared to several other state-of-the-art algorithms. The training time improvement is achieved at no cost, in other words, the accuracy is kept at the same level. All the experiments that support this claim were conducted on various datasets within the strict double cross-validation scheme. DL2 SVM solved with NN ISDA has faster training time on both medium and large datasets. In addition to a comprehensive DL2 SVM model we introduce and derive its three variants. Three different solvers for the DL2\u27s system of linear equations with nonnegativity constraints were implemented, presented and compared in this dissertation

Hinge-Loss Markov Random Fields and Probabilistic Soft Logic: A Scalable Approach to Structured Prediction

A fundamental challenge in developing impactful artificial intelligence technologies is balancing the ability to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this thesis I introduce two new formalisms for modeling structured data, distinguished from previous approaches by their ability to both capture rich structure and scale to big data. The first, hinge-loss Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. I unite three views of inference from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three views lead to the same inference objective. I then derive HL-MRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HL-MRFs easy to define, refine, and reuse for relational data. PSL uses a syntax based on first-order logic to compactly specify complex models. I next introduce an algorithm for inferring most-probable variable assignments (MAP inference) for HL-MRFs that is extremely scalable, much more so than commercially available software, because it uses message passing to leverage the sparse dependency structures common in inference tasks. I then show how to learn the parameters of HL-MRFs using a number of learning objectives. The learned HL-MRFs are as accurate as traditional, discrete models, but much more scalable. To enable HL-MRFs and PSL to capture even richer dependencies, I then extend learning to support latent variables, i.e., variables without training labels. To overcome the bottleneck of repeated inferences required during learning, I introduce paired-dual learning, which interleaves inference and parameter updates. Paired-dual learning learns accurate models and is also scalable, often completing before traditional methods make even one parameter update. Together, these algorithms enable HL-MRFs and PSL to model rich, structured data at scales not previously possible