Solution Path Algorithm for Twin Multi-class Support Vector Machine

The twin support vector machine and its extensions have made great achievements in dealing with binary classification problems, however, which is faced with some difficulties such as model selection and solving multi-classification problems quickly. This paper is devoted to the fast regularization parameter tuning algorithm for the twin multi-class support vector machine. A new sample dataset division method is adopted and the Lagrangian multipliers are proved to be piecewise linear with respect to the regularization parameters by combining the linear equations and block matrix theory. Eight kinds of events are defined to seek for the starting event and then the solution path algorithm is designed, which greatly reduces the computational cost. In addition, only few points are combined to complete the initialization and Lagrangian multipliers are proved to be 1 as the regularization parameter tends to infinity. Simulation results based on UCI datasets show that the proposed method can achieve good classification performance with reducing the computational cost of grid search method from exponential level to the constant level

Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction:a review

Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported

LIPIcs, Volume 261, ICALP 2023, Complete Volume

LIPIcs, Volume 261, ICALP 2023, Complete Volum

Machine Learning assisted Digital Twin for event identification in electrical power system

The challenges of stable operation in the electrical power system are increasing with the infrastructure shifting of the power grid from the centralized energy supply with fossil fuels towards sustainable energy generation. The predominantly RES plants, due to the non-linear electronic switch, have brought harmonic oscillations into the power grid. These changes lead to difficulties in stable operation, reduction of outages and management of variations in electric power systems. The emergence of the Digital Twin in the power system brings the opportunity to overcome these challenges. Digital Twin is a digital information model that accurately represents the state of every asset in a physical system. It can be used not only to monitor the operation states with actionable insights of physical components to drive optimized operation but also to generate abundant data by simulation according to the guidance on design limits of physical systems. The work addresses the topic of the origin of the Digital Twin concept and how it can be utilized in the optimization of power grid operation.Die Herausforderungen für den zuverfässigen Betrieb des elektrischen Energiesystems werden mit der Umwandlung der Infrastruktur in Stromnetz von der zentralen Energieversorgung mit fossilen Brennstoffen hin zu der regenerativen Energieeinspeisung stetig zugenommen. Der Ausbau der erneuerbaren Energien im Zuge der klimapolitischen Zielsetzung zur CO²-Reduzierung und des Ausstiegs aus der Kernenergie wird in Deutschland zügig vorangetrieben. Aufgrund der nichtlinearen elektronischen Schaltanlagen werden die aus EE-Anlagen hervorgegangenen Oberschwingungen in das Stromnetz eingebracht, was nicht nur die Komplexität des Stromnetzes erhöht, sondern auch die Stabilität des Systems beeinflusst. Diese Entwicklungen erschweren den stabilen Betrieb, die Verringerung der Ausfälle und das Management der Netzschwankungen im elektrischen Energiesystem. Das Auftauchen von Digital Twin bringt die Gelegenheit zur Behebung dieser Herausforderung. Digital Twin ist ein digitales Informationsmodell, das den Zustand des physikalischen genau abbildet. Es kann nicht nur zur Überwachung der Betriebszustände mit nachvollziehbarem Einsichten über physischen Komponenten sondern auch zur Generierung der Daten durch Simulationen unter der Berücksichtigung der Auslegungsgrenze verwendet werden. Diesbezüglich widmet sich die Arbeit zunächste der Fragestellung, woher das Digital Twin Konzept stammt und wie das Digitan Twin für die Optimierung des Stromnetzes eingesetzt wird. Hierfür werden die Perspektiven über die dynamische Zustandsschätzung, die Überwachung des des Betriebszustands, die Erkennung der Anomalien usw. im Stromnetz mit Digital Twin spezifiziert. Dementsprechend wird die Umsetzung dieser Applikationen auf dem Lebenszyklus-Management basiert. Im Rahmen des Lebenszyklusschemas von Digital Twin sind drei wesentliche Verfahren von der Modellierung des Digital Twins zur deren Applizierung erforderlich: Parametrierungsprozess für die Modellierung des Digital Twins, Datengenerierung mit Digital Twin Simulation und Anwendung mit Machine Learning Algorithmus für die Erkennung der Anomalie. Die Validierung der Zuverlässigkeit der Parametrierung für Digital Twin und der Eventserkennung erfolgt mittels numerischer Fallstudien. Dazu werden die Algorithmen für Online und Offline zur Parametrierung des Digital Twins untersucht. Im Rahmen dieser Arbeit wird das auf CIGRÉ basierende Referenznetz zur Abbildung des Digital Twin hinsichtlich der Referenzmessdaten parametriert. So sind neben der Synchronmaschine und Umrichter basierende Einspeisung sowie Erreger und Turbine auch regler von Umrichter für den Parametrierungsprozess berücksichtigt. Nach der Validierung des Digital Twins werden die zahlreichen Simulationen zur Datengenerierung durchgeführt. Jedes Event wird mittels der Daten vo Digital Twin mit einem "Fingerprint" erfasst. Das Training des Machine Learning Algorithmus wird dazu mit den simulierten Daten von Digital Twin abgewickelt. Das Erkennungsergebnis wird durch die Fallstudien validiert und bewertet

LIPIcs, Volume 248, ISAAC 2022, Complete Volume

LIPIcs, Volume 248, ISAAC 2022, Complete Volum

Novel support vector machines for diverse learning paradigms

This dissertation introduces novel support vector machines (SVM) for the following traditional and non-traditional learning paradigms: Online classification, Multi-Target Regression, Multiple-Instance classification, and Data Stream classification. Three multi-target support vector regression (SVR) models are first presented. The first involves building independent, single-target SVR models for each target. The second builds an ensemble of randomly chained models using the first single-target method as a base model. The third calculates the targets\u27 correlations and forms a maximum correlation chain, which is used to build a single chained SVR model, improving the model\u27s prediction performance, while reducing computational complexity. Under the multi-instance paradigm, a novel SVM multiple-instance formulation and an algorithm with a bag-representative selector, named Multi-Instance Representative SVM (MIRSVM), are presented. The contribution trains the SVM based on bag-level information and is able to identify instances that highly impact classification, i.e. bag-representatives, for both positive and negative bags, while finding the optimal class separation hyperplane. Unlike other multi-instance SVM methods, this approach eliminates possible class imbalance issues by allowing both positive and negative bags to have at most one representative, which constitute as the most contributing instances to the model. Due to the shortcomings of current popular SVM solvers, especially in the context of large-scale learning, the third contribution presents a novel stochastic, i.e. online, learning algorithm for solving the L1-SVM problem in the primal domain, dubbed OnLine Learning Algorithm using Worst-Violators (OLLAWV). This algorithm, unlike other stochastic methods, provides a novel stopping criteria and eliminates the need for using a regularization term. It instead uses early stopping. Because of these characteristics, OLLAWV was proven to efficiently produce sparse models, while maintaining a competitive accuracy. OLLAWV\u27s online nature and success for traditional classification inspired its implementation, as well as its predecessor named OnLine Learning Algorithm - List 2 (OLLA-L2), under the batch data stream classification setting. Unlike other existing methods, these two algorithms were chosen because their properties are a natural remedy for the time and memory constraints that arise from the data stream problem. OLLA-L2\u27s low spacial complexity deals with memory constraints imposed by the data stream setting, and OLLAWV\u27s fast run time, early self-stopping capability, as well as the ability to produce sparse models, agrees with both memory and time constraints. The preliminary results for OLLAWV showed a superior performance to its predecessor and was chosen to be used in the final set of experiments against current popular data stream methods. Rigorous experimental studies and statistical analyses over various metrics and datasets were conducted in order to comprehensively compare the proposed solutions against modern, widely-used methods from all paradigms. The experimental studies and analyses confirm that the proposals achieve better performances and more scalable solutions than the methods compared, making them competitive in their respected fields

Non-convex Optimization for Machine Learning

A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. The monograph will lead the reader through several widely used non-convex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.Comment: The official publication is available from now publishers via http://dx.doi.org/10.1561/220000005