Support vector machine based classification in condition monitoring of induction motors

Continuous and trouble-free operation of induction motors is an essential part of modern power and production plants. Faults and failures of electrical machinery may cause remarkable economical losses but also highly dangerous situations. In addition to analytical and knowledge-based models, application of data-based models has established a firm position in the induction motor fault diagnostics during the last decade. For example, pattern recognition with Neural Networks (NN) is widely studied. Support Vector Machine (SVM) is a novel machine learning method introduced in early 90's. It is based on the statistical learning theory presented by V.N. Vapnik, and it has been successfully applied to numerous classification and pattern recognition problems such as text categorization, image recognition and bioinformatics. SVM based classifier is built to minimize the structural misclassification risk, whereas conventional classification techniques often apply minimization of the empirical risk. Therefore, SVM is claimed to lead enhanced generalisation properties. Further, application of SVM results in the global solution for a classification problem. Thirdly, SVM based classification is attractive, because its efficiency does not directly depend on the dimension of classified entities. This property is very useful in fault diagnostics, because the number of fault classification features does not have to be drastically limited. However, SVM has not yet been widely studied in the area of fault diagnostics. Specifically, in the condition monitoring of induction motor, it does not seem to have been considered before this research. In this thesis, a SVM based classification scheme is designed for different tasks in induction motor fault diagnostics and for partial discharge analysis of insulation condition monitoring. Several variables are compared as fault indicators, and forces on rotor are found to be important in fault detection instead of motor current that is currently widely studied. The measurement of forces is difficult, but easily measurable vibrations are directly related to the forces. Hence, vibration monitoring is considered in more detail as the medium for the motor fault diagnostics. SVM classifiers are essentially 2-class classifiers. In addition to the induction motor fault diagnostics, the results of this thesis cover various methods for coupling SVMs for carrying out a multi-class classification problem.reviewe

Examination of the applicability of Support Vector Machines in the context of ischaemia detection

Projecte final de carrera fet en col.laboració amb Technische Universität Dresden. Fakultät Elektrotechnik und InformationstechnikCatalà: El present treball constitueix una proposta per la detecció d'episodis isquemics ST-T basada en la classificació de batecs càrdiacs. Per classificar els batecs de la European ST-T Database (EDB) s'ha aplicat Support Vector Machines (SVMs). Diferents experiments respecte variants de SVM (SVM binària - multi-classe SVM, nucli lineal - nucli RBF, dades d'entrenament balancejades - dades d'entrenament desbalancejades) s'han dut a terme per tal d'extreure la informació adequada sobre els models de SVM. Els resultats obtinguts demostren l'aplicabilitat del SVM en el context donat. No obstant això, la comparació d'aquests resultats amb els resultats prèviament obtinguts en la detecció d'episodis de ST-T demostren la necessitat d'una major investigació sobre el tema. Les observacions que es dedueix d'aquesta tesi són motiu de la suposició que el nombre relativament reduit del patró d'entrenament és la principal raó de les limitacions que s'examinen en els resultats. No obstant això, l'ampliació del conjunt d'entrenament no és una tasca fàcil. Les futures línies de treball haurien d'abordar aquesta qüestió. De tal manera, dues estratègies es poden dur a terme: 1. El subconjunt d'entrenament podria ser ampliat mitjançant la inclusió de més morfologies però mantenint la mateixa mida; això implicaria que el nombre de registres d'entrenament ha de ser major i de cada registre un nombre menor nombre de batecs és seleccionat i mantenint LIBSVM com a mètode aplicat. 2. El subconjunt d'entrenament podria ser ampliat mitjançant la inclusió de més morfologies i l'augment de la mida; això implicaria que el nombre de registres d'entrenament ha de ser major, el total nombre de batecs és major i nous mètodes d?enstrenament han de ser aplicats. L'última estratègia, òbviament, constitueix l'enfocament més integral. Aquesta estratègia s'ha de dur a terme si l'aplicació de SVM està previst fins i tot en altres contextos dins del grup de treball. A més, un possible enfocament podria avaluar la utilitat i eficiència, respectivament, de LaSVM com a paquet base de programari. LaSVM esta especialment destinat a ser utilitzat en el cas de grans conjunts de dades (en el sentit d'un gran nombre de model d'entrenament). Com que LaSVM suporta el mateix format de dades que LIBSVM la majoria de l'experiència i alguns mètodes que s'han ideat en aquest treball podrien ser integrats fàcilment en les funcions de LaSVM.Castellano: El presente trabajo constituye una propuesta para la detección de episodios isquémicos ST-T basada en la clasificación de latidos cardíacos. Para clasificar los latidos de la European ST-T Database (EDB) se ha aplicado Support Vector Machines (SVMs). Diferentes experimentos respecto variantes de SVM (SVM binaria - multi-clase SVM, núcleo lineal - núcleo RBF, datos de entrenamiento balanceadas - datos de entrenamiento desbalanceadas) se han llevado a cabo para extraer la información adecuada sobre los modelos de SVM. Los resultados obtenidos demuestran la aplicabilidad del SVM en el contexto dado. Sin embargo, la comparación de estos resultados con los resultados previamente obtenidos en la detección de episodios de ST-T demuestran la necesidad de una mayor investigación sobre el tema. Las observaciones que se deducen de esta tesis son motivo de la suposición de que el número relativamente reducido del patrón de entrenamiento es la principal razón de las limitaciones que se examinan en los resultados. Sin embargo, la ampliación del conjunto de entrenamiento no es una tarea fácil. Las futuras líneas de trabajo deberían abordar esta cuestión. De tal modo, dos estrategias se pueden llevar a cabo: 1. El subconjunto de entrenamiento podría ser ampliado mediante la inclusión de más morfologías pero manteniendo el mismo tamaño, lo que implicaría que el número de registros de entrenamiento debe ser mayor y de cada registro un número menor de latidos es seleccionado y manteniendo LIBSVM como método aplicado. 2. El subconjunto de entrenamiento podría ser ampliado mediante la inclusión de más morfologías y el aumento del tamaño, lo que implicaría que el número de registros de entrenamiento debe ser mayor, el total número de latidos es mayor y nuevos métodos de enstrenament deben ser aplicados. La última estrategia, obviamente, constituye el enfoque más integral. Esta estrategia debe llevarse a cabo si la aplicación de SVM está previsto incluso en otros contextos dentro del grupo de trabajo. Además, un posible enfoque podría evaluar la utilidad y eficiencia, respectivamente, de LaSVM como paquete base de software. LaSVM está especialmente destinado a ser utilizado en el caso de grandes conjuntos de datos (en el sentido de un gran número de modelo de entrenamiento). Como LaSVM soporta el mismo formato de datos que LIBSVM la mayoría de la experiencia y algunos métodos que se han ideado en este trabajo podrían ser integrados fácilmente en las funciones de LaSVM.English: The presented work constitutes an approach to detect ST-T- ischaemic episodes based on beat classes. To classify the beats of the European ST-T Database (EDB) Support Vector Machines (SVMs) have been applied. Diferent experiments regarding variants of SVMs (binary SVM - multi-class SVM, linear Kernel - RBF Kernel, balanced training data - unbalanced training data) have been carried out in order to extract information on suitable SVM models. The obtained results show the applicability of SVMs in the given context. However, the comparison of these results to previously obtained results on the detection of ST-T-episodes even clarifies the need for further investigation on the topic. The observations which may be deduced from this thesis give cause for the assumption that the relatively small number of training pattern is the main reason of the limitations which are examined within the results. However, expanding the training set is not an easy task. Future works should address this issue. Thereby, two strategies may be pursued: 1. The training subset could be expanded by including more morphologies but maintaining the same size; this would imply that the number of training records should be increased, from each record a smaller number of beats is selected and the applied methods (LibSVM) could be maintained. 2. The training subset could be expanded by including more morphologies and increasing the size; this would imply that the number of training records should be increased, the overall number of beats is increased and new training methods must be applied. The latter strategy obviously constitutes the more comprehensive approach. This strategy should be pursued if the application of SVMs is planned even in other contexts within the working group. Thereto, a possible approach could evaluate the usability and eficiency, respectively, of the software package LaSVM. LaSVM especially is intended to be used in the case of large datasets (in the sense of a big number of training pattern). As LaSVM supports the same data formats as LibSVM most of the experience and some methods which have been devised in this work could be transfered thus allowing an easy integration of LaSVM functions

Multiple voice disorders in the same individual: Investigating handcrafted features, multi-label classification algorithms, and base-learners

Non-invasive acoustic analyses of voice disorders have been at the forefront of current biomedical research. Usual strategies, essentially based on machine learning (ML) algorithms, commonly classify a subject as being either healthy or pathologically-affected. Nevertheless, the latter state is not always a result of a sole laryngeal issue, i.e., multiple disorders might exist, demanding multi-label classification procedures for effective diagnoses. Consequently, the objective of this paper is to investigate the application of five multi-label classification methods based on problem transformation to play the role of base-learners, i.e., Label Powerset, Binary Relevance, Nested Stacking, Classifier Chains, and Dependent Binary Relevance with Random Forest (RF) and Support Vector Machine (SVM), in addition to a Deep Neural Network (DNN) from an algorithm adaptation method, to detect multiple voice disorders, i.e., Dysphonia, Laryngitis, Reinke's Edema, Vox Senilis, and Central Laryngeal Motion Disorder. Receiving as input three handcrafted features, i.e., signal energy (SE), zero-crossing rates (ZCRs), and signal entropy (SH), which allow for interpretable descriptors in terms of speech analysis, production, and perception, we observed that the DNN-based approach powered with SE-based feature vectors presented the best values of F1-score among the tested methods, i.e., 0.943, as the averaged value from all the balancing scenarios, under Saarbrücken Voice Database (SVD) and considering 20% of balancing rate with Synthetic Minority Over-sampling Technique (SMOTE). Finally, our findings of most false negatives for laryngitis may explain the reason why its detection is a serious issue in speech technology. The results we report provide an original contribution, allowing for the consistent detection of multiple speech pathologies and advancing the state-of-the-art in the field of handcrafted acoustic-based non-invasive diagnosis of voice disorders

Interpretable Machine Learning을 활용한 구간단속시스템 설치에 따른 인명피해사고 감소 효과 연구

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 건설환경공학부, 2020. 8. 김동규.In this study, a prediction model for casualty crash occurrence was developed considering whether to install SSES and the effect of SSES installation was quantified by dividing it into direct and indirect effects through the analysis of mediation effect. Also, it was recommended what needs to be considered in selecting the candidate sites for SSES installation. For this, crash prediction model was developed by using the machine learning for binary classification based on whether or not casualty crash occurred and the effects of SSES installation were analyzed based on crashes and speed-related variables. Especially, the IML methodology was applied that considered the predictive performance as well as the interpretability of the forecast results as important. When developing the IML which consisted of black-box and interpretable model, KNN, RF, and SVM were reviewed as black-box model, and DT and BLR were reviewed as interpretable model. In the model development, the hyper-parameters that could be set in each methodology were optimized through k-fold cross validation. The SVM with a polynomial kernel trick was selected as black-box model and the BLR was selected as interpretable model to predict the probability of casualty crash occurrence. For the developed IML model, the evaluation was conducted through comparison with the typical BLR from the perspective of the PDR framework. The evaluation confirmed that the results of the IML were more excellent than the typical BLR in terms of predictive accuracy, descriptive accuracy, and relevancy from a human in the loop. Using the result of IML's model development, the effect on SSES installation were quantified based on the probability equation of casualty crash occurrence. The equation is the logistic function that consists of SSES, SOR, SV, TVL, HVR, and CR. The result of analysis confirmed that the SSES installation reduced the probability of casualty crash occurrence by about 28%. In addition, the analysis of mediation effects on the variables affected by installing SSES was conducted to quantify the direct and indirect effects on the probability of reducing the casualty crashes caused by the SSES installation. The proportion of indirect effects through reducing the ratio of exceeding the speed limit (SOR) was about 30% and the proportion of indirect effects through reduction of speed variance (SV) was not statistically significant at the 95% confidence level. Finally, the probability equation of casualty crash occurrence developed in this study was applied to the sections of Yeongdong Expressway to compare the crash risk section with the actual crash data to examine the applicability of the development model. The analysis result verified that the equation was reasonable. Therefore, it may be considered to select dangerous sites based on casualty crash and speeding firstly, and then to install SSES at the section where traffic volume (TVL), heavy vehicle ratio (HVR), and curve ratio (CR) are higher than the other sections.본 연구에서는 구간단속시스템(Section Speed Enforcement System, SSES) 설치 효과를 정량화하기 위해 인명피해사고 예측모형을 개발하고, 매개효과 분석을 통해 SSES 설치에 대한 직접효과와 간접효과를 구분하여 정량화하였다. 또한, 개발한 예측모형에 대한 고속도로에서의 적용 가능성을 검토하고, SSES 설치 대상지 선정 시 고려해야할 사항을 제안하였다. 모형 개발에는 인명피해사고 발생 여부를 종속변수로 하는 이진분류형 기계학습을 활용하였으며, 기계학습 중에서는 모형의 예측 성능과 더불어 예측 결과에 대한 해석력을 중요하게 고려하는 인터프리터블 머신 러닝(Interpretable Machine Learning, IML) 방법론을 적용하였다. IML은 블랙박스 모델과 인터프리터블 모델로 구성되며, 본 연구에서는 블랙박스 모델로 KNN, RF 및 SVM을, 인터프리터블 모델로 DT와 BLR을 검토하였다. 모형 개발 시에는 각 기법에서 튜닝이 가능한 하이퍼 파라미터에 대하여 교차검증 과정을 거쳐 최적화하였다. 블랙박스 모델은 폴리노미얼 커널 트릭을 활용한 SVM을, 인터프리터블 모델은 BLR을 적용하여 인명피해사고 발생 확률을 예측하는 모형을 개발하였다. 개발된 IML 모델에 대해서는 PDR(Predictive accuracy, Descriptive accuracy and Relevancy) 프레임워크 관점에서 (typical) BLR 모델과 비교 평가를 진행하였다. 평가 결과 예측 정확도, 해석 정확도 및 인간의 이해관점에서의 적합성 등에서 모두 IML 모델이 우수함을 확인하였다. 또한, 본 연구에서 개발된 IML 모델 기반의 인명피해사고 발생 확률식은 SSES, SOR, SV, TVL, HVR 및 CR의 독립변수로 구성되었으며, 이 확률식을 기반으로 SSES 설치에 대한 효과를 정량화하였다. 정량화 분석 결과, SSES 설치로 인해 약 28% 정도의 인명피해사고 발생 확률이 감소함을 확인할 수 있었다. 또한, 모형 개발에 활용된 변수 중 SSES 설치로 인해 영향을 받는 변수들(SOR 및 SV)에 대한 매개효과 분석을 통해 SSES 설치로 인한 인명피해사고 감소 확률을 직접효과와 간접효과를 구분하여 제시하였다. 분석 결과, SSES와 제한속도 초과비율(SOR)의 관계에서 있어서는 약 30%가 간접효과이고, SSES와 속도분산(SV)의 관계에 있어서는 매개효과가 통계적으로 유의하지 않음을 확인할 수 있었다. 마지막으로 영동고속도로를 대상으로 인명피해사고 발생 확률식 기반의 예측 위험구간과 실제 인명사고 다발 구간에 대한 비교 분석을 통해 연구 결과의 활용 가능성을 확인하였다. 또한, SSES 설치 대상지 선정 시에는 사고 및 속도 분석을 통한 위험구간을 선별한 후 교통량(TVL)이 많은 곳, 통과차량 중 중차량 비율(HVR)이 높은 곳 및 구간 내 곡선비율(CR)이 높은 곳을 우선적으로 검토하는 것을 제안하였다.1. Introduction 1 1.1. Background of research 1 1.2. Objective of research 4 1.3. Research Flow 6 2. Literature Review 11 2.1. Research related to SSES 11 2.1.1. Effectiveness of SSES 11 2.1.2. Installation criteria of SSES 15 2.2. Machine learning about transportation 17 2.2.1. Machine learning algorithm 17 2.2.2. Machine learning algorithm about transportation 19 2.3. Crash prediction model 23 2.3.1. Frequency of crashes 23 2.3.2. Severity of crash 26 2.4. Interpretable Machine Learning (IML) 31 2.4.1. Introduction 31 2.4.2. Application of IML 33 3. Model Specification 37 3.1. Analysis of SSES effectiveness 37 3.1.1. Crashes analysis 37 3.1.2. Speed analysis 39 3.2. Data collection & pre-analysis 40 3.2.1. Data collection 40 3.2.2. Basic statistics of variables 42 3.3. Response variable selection 50 3.4. Model selection 52 3.4.1. Binary classification 52 3.4.2. Accuracy vs. Interpretability 53 3.4.3. Overview of IML 54 3.4.4. Process of model specification 57 4. Model development 59 4.1. Black-box and interpretable model 59 4.1.1. Consists of IML 59 4.1.2. Black-box model 60 4.1.3. Interpretable model 68 4.2. Model development 72 4.2.1. Procedure 72 4.2.2. Measures of effectiveness 74 4.2.3. K-fold cross validation 76 4.3. Result of model development 78 4.3.1. Result of black-box model 78 4.3.2. Result of interpretable model 85 5. Evaluation & Application 91 5.1. Evaluation 91 5.1.1. The PDR framework for IML 91 5.1.2. Predictive accuracy 93 5.1.3. Descriptive accuracy 94 5.1.4. Relevancy 99 5.2. Impact of Casualty Crash Reduction 102 5.2.1. Quantification of the effectiveness 102 5.2.2. Mediation effect analysis 106 5.3. Application for the Korean expressway 118 6. Conclusion 121 6.1. Summary and Findings 121 6.2. Further Research 125Docto

Knowledge-Based Analysis of Genomic Expression Data by Using Different Machine Learning Algorithms for the Purpose of Diagnostic, Prognostic or Therapeutic Application

With more and more biological information generated, the most pressing task of bioinformatics has become to analyze and interpret various types of data, including nucleotide and amino acid sequences, protein structures, gene expression profiling and so on. In this dissertation, we apply the data mining techniques of feature generation, feature selection, and feature integration with learning algorithms to tackle the problems of disease phenotype classification, clinical outcome and patient survival prediction from gene expression profiles. We analyzed the effect of batch noise in microarray data on the performance of classification. Batchmatch, a batch adjusting algorithm based on double scaling method is advantageous over Combat, another batch correcting algorithm based on the empirical bayes frame work. In order to identify genes associated with disease phenotype classification or patient survival prediction from gene expression data, we compared and analyzed the performance of five feature selection algorithms. Our observations from these studies indicated that Gainratio algorithm performs better and more consistently over the other algorithms studied. When it comes to performance metric to choose the best classifiers, MCC gives unbiased performance results over accuracy in some endpoints, where class imbalance is more. In the aspect of classification algorithms, no single algorithm is absolutely superior to all others, though SVM achieved fairly good results in most endpoints. Naive bayes algorithm also performed well in some endpoints. Overall, from the total 65 models we reported (5 top models for 13 end points) SVM and SMO (a variant of SVM) dominate mostly, also the linear kernel performed well over RBF in our binary classifications

IDENTIFICATION OF HEAT RELEASE SHAPES AND COMBUSTION CONTROL OF AN LTC ENGINE

Low Temperature Combustion (LTC) regimes have gained attention in internal combustion engines since they deliver low nitrogen oxides (NOx) and soot emissions with higher thermal efficiency and better combustion efficiency, compared to conventional combustion regimes. However, the operating region of these high-efficiency combustion regimes is limited as it is prone to knocking and high in-cylinder pressure rise rate outside the engine safe zone. By allowing multi-regime operation, high-efficiency region of the engine is extended. To control these complex engines, understanding and identification of heat release rate shapes is essential. Experimental data collected from a 2 liter 4 cylinder LTC engine with in-cylinder pressure measurements, is used in this study to calculate Heat Release Rate (HRR). Fractions of early and late heat release are calculated from HRR as a ratio of cumulative heat release in the early or late window to the total energy of the fuel injected into the cylinder. Three specific HRR patterns and two transition zones are identified. A rule based algorithm is developed to classify these patterns as a function of fraction of early and late heat release percentages. Combustion parameters evaluated also showed evidence on characteristics of classification. Supervised and unsupervised machine learning approaches are also evaluated to classify the HRR shapes. Supervised learning method ( Decision Tree)is studied to develop an automatic classifier based on the control inputs to the engine. In addition, supervised learning method (Convolutional Neural Network (CNN)) and unsupervised learning method (k-means clustering) are studied to develop an automatic classifier based on HRR trace obtained from the engine. The unsupervised learning approach wasn\u27t successful in classification as the arrived k-means centroids didn\u27t clearly represent a particular combustion regime. Supervised learning techniques, CNN method is found with a classifier accuracy of 70% for identifying heat release shapes and Decision Tree with the accuracy of 74.5% as a function of control inputs. On rule based classified traces with the use of principle component analysis (PCA) and linear regression, heat release rate classifiers are built as a function of engine input parameters including, Engine speed, Start of injection (SOI), Fuel quantity (FQ) and Premixed ratio (PR). The results are then used to build a linear parameter varying (LPV) model as a function of the modelled combustion classifiers by using the least square support vector machine (LS-SVM) approach. LPV model could predict CA50(Combustion phasing), IMEP (indicated mean effective pressure) and MPRR (maximum pressure rise rate) with a RMSE of 0.4 CAD, 16.6 kPa and 0.4 bar/CAD respectively. The designed LPV model is then incorporated in a model predictive control (MPC) platform to adjust CA50, IMEP and MPRR. The results show the designed LTC engine controller could track CA50 and IMEP with average error of 1.2 CAD and 6.2 kPa while limiting MPRR to 6 bar/CAD. The controller uses three engine inputs including, SOI, PR and FQ as manipulated variables, that are optimally changed to control the LTC engine

Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data

Similarity-based approaches represent a promising direction for time series analysis. However, many such methods rely on parameter tuning, and some have shortcomings if the time series are multivariate (MTS), due to dependencies between attributes, or the time series contain missing data. In this paper, we address these challenges within the powerful context of kernel methods by proposing the robust \emph{time series cluster kernel} (TCK). The approach taken leverages the missing data handling properties of Gaussian mixture models (GMM) augmented with informative prior distributions. An ensemble learning approach is exploited to ensure robustness to parameters by combining the clustering results of many GMM to form the final kernel. We evaluate the TCK on synthetic and real data and compare to other state-of-the-art techniques. The experimental results demonstrate that the TCK is robust to parameter choices, provides competitive results for MTS without missing data and outstanding results for missing data.Comment: 23 pages, 6 figure

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

Multivariate Statistical Machine Learning Methods for Genomic Prediction

This book is open access under a CC BY 4.0 license This open access book brings together the latest genome base prediction models currently being used by statisticians, breeders and data scientists. It provides an accessible way to understand the theory behind each statistical learning tool, the required pre-processing, the basics of model building, how to train statistical learning methods, the basic R scripts needed to implement each statistical learning tool, and the output of each tool. To do so, for each tool the book provides background theory, some elements of the R statistical software for its implementation, the conceptual underpinnings, and at least two illustrative examples with data from real-world genomic selection experiments. Lastly, worked-out examples help readers check their own comprehension. The book will greatly appeal to readers in plant (and animal) breeding, geneticists and statisticians, as it provides in a very accessible way the necessary theory, the appropriate R code, and illustrative examples for a complete understanding of each statistical learning tool. In addition, it weighs the advantages and disadvantages of each tool