FSL-BM: Fuzzy Supervised Learning with Binary Meta-Feature for Classification

This paper introduces a novel real-time Fuzzy Supervised Learning with Binary Meta-Feature (FSL-BM) for big data classification task. The study of real-time algorithms addresses several major concerns, which are namely: accuracy, memory consumption, and ability to stretch assumptions and time complexity. Attaining a fast computational model providing fuzzy logic and supervised learning is one of the main challenges in the machine learning. In this research paper, we present FSL-BM algorithm as an efficient solution of supervised learning with fuzzy logic processing using binary meta-feature representation using Hamming Distance and Hash function to relax assumptions. While many studies focused on reducing time complexity and increasing accuracy during the last decade, the novel contribution of this proposed solution comes through integration of Hamming Distance, Hash function, binary meta-features, binary classification to provide real time supervised method. Hash Tables (HT) component gives a fast access to existing indices; and therefore, the generation of new indices in a constant time complexity, which supersedes existing fuzzy supervised algorithms with better or comparable results. To summarize, the main contribution of this technique for real-time Fuzzy Supervised Learning is to represent hypothesis through binary input as meta-feature space and creating the Fuzzy Supervised Hash table to train and validate model.Comment: FICC201

A Comprehensive Survey on the Cyber-Security of Smart Grids: Cyber-Attacks, Detection, Countermeasure Techniques, and Future Directions

One of the significant challenges that smart grid networks face is cyber-security. Several studies have been conducted to highlight those security challenges. However, the majority of these surveys classify attacks based on the security requirements, confidentiality, integrity, and availability, without taking into consideration the accountability requirement. In addition, some of these surveys focused on the Transmission Control Protocol/Internet Protocol (TCP/IP) model, which does not differentiate between the application, session, and presentation and the data link and physical layers of the Open System Interconnection (OSI) model. In this survey paper, we provide a classification of attacks based on the OSI model and discuss in more detail the cyber-attacks that can target the different layers of smart grid networks communication. We also propose new classifications for the detection and countermeasure techniques and describe existing techniques under each category. Finally, we discuss challenges and future research directions

Vehicle make and model recognition for intelligent transportation monitoring and surveillance.

Vehicle Make and Model Recognition (VMMR) has evolved into a significant subject of study due to its importance in numerous Intelligent Transportation Systems (ITS), such as autonomous navigation, traffic analysis, traffic surveillance and security systems. A highly accurate and real-time VMMR system significantly reduces the overhead cost of resources otherwise required. The VMMR problem is a multi-class classification task with a peculiar set of issues and challenges like multiplicity, inter- and intra-make ambiguity among various vehicles makes and models, which need to be solved in an efficient and reliable manner to achieve a highly robust VMMR system. In this dissertation, facing the growing importance of make and model recognition of vehicles, we present a VMMR system that provides very high accuracy rates and is robust to several challenges. We demonstrate that the VMMR problem can be addressed by locating discriminative parts where the most significant appearance variations occur in each category, and learning expressive appearance descriptors. Given these insights, we consider two data driven frameworks: a Multiple-Instance Learning-based (MIL) system using hand-crafted features and an extended application of deep neural networks using MIL. Our approach requires only image level class labels, and the discriminative parts of each target class are selected in a fully unsupervised manner without any use of part annotations or segmentation masks, which may be costly to obtain. This advantage makes our system more intelligent, scalable, and applicable to other fine-grained recognition tasks. We constructed a dataset with 291,752 images representing 9,170 different vehicles to validate and evaluate our approach. Experimental results demonstrate that the localization of parts and distinguishing their discriminative powers for categorization improve the performance of fine-grained categorization. Extensive experiments conducted using our approaches yield superior results for images that were occluded, under low illumination, partial camera views, or even non-frontal views, available in our real-world VMMR dataset. The approaches presented herewith provide a highly accurate VMMR system for rea-ltime applications in realistic environments.\\ We also validate our system with a significant application of VMMR to ITS that involves automated vehicular surveillance. We show that our application can provide law inforcement agencies with efficient tools to search for a specific vehicle type, make, or model, and to track the path of a given vehicle using the position of multiple cameras

Interpretable Machine Learning을 활용한 구간단속시스템 설치에 따른 인명피해사고 감소 효과 연구

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 건설환경공학부, 2020. 8. 김동규.In this study, a prediction model for casualty crash occurrence was developed considering whether to install SSES and the effect of SSES installation was quantified by dividing it into direct and indirect effects through the analysis of mediation effect. Also, it was recommended what needs to be considered in selecting the candidate sites for SSES installation. For this, crash prediction model was developed by using the machine learning for binary classification based on whether or not casualty crash occurred and the effects of SSES installation were analyzed based on crashes and speed-related variables. Especially, the IML methodology was applied that considered the predictive performance as well as the interpretability of the forecast results as important. When developing the IML which consisted of black-box and interpretable model, KNN, RF, and SVM were reviewed as black-box model, and DT and BLR were reviewed as interpretable model. In the model development, the hyper-parameters that could be set in each methodology were optimized through k-fold cross validation. The SVM with a polynomial kernel trick was selected as black-box model and the BLR was selected as interpretable model to predict the probability of casualty crash occurrence. For the developed IML model, the evaluation was conducted through comparison with the typical BLR from the perspective of the PDR framework. The evaluation confirmed that the results of the IML were more excellent than the typical BLR in terms of predictive accuracy, descriptive accuracy, and relevancy from a human in the loop. Using the result of IML's model development, the effect on SSES installation were quantified based on the probability equation of casualty crash occurrence. The equation is the logistic function that consists of SSES, SOR, SV, TVL, HVR, and CR. The result of analysis confirmed that the SSES installation reduced the probability of casualty crash occurrence by about 28%. In addition, the analysis of mediation effects on the variables affected by installing SSES was conducted to quantify the direct and indirect effects on the probability of reducing the casualty crashes caused by the SSES installation. The proportion of indirect effects through reducing the ratio of exceeding the speed limit (SOR) was about 30% and the proportion of indirect effects through reduction of speed variance (SV) was not statistically significant at the 95% confidence level. Finally, the probability equation of casualty crash occurrence developed in this study was applied to the sections of Yeongdong Expressway to compare the crash risk section with the actual crash data to examine the applicability of the development model. The analysis result verified that the equation was reasonable. Therefore, it may be considered to select dangerous sites based on casualty crash and speeding firstly, and then to install SSES at the section where traffic volume (TVL), heavy vehicle ratio (HVR), and curve ratio (CR) are higher than the other sections.본 연구에서는 구간단속시스템(Section Speed Enforcement System, SSES) 설치 효과를 정량화하기 위해 인명피해사고 예측모형을 개발하고, 매개효과 분석을 통해 SSES 설치에 대한 직접효과와 간접효과를 구분하여 정량화하였다. 또한, 개발한 예측모형에 대한 고속도로에서의 적용 가능성을 검토하고, SSES 설치 대상지 선정 시 고려해야할 사항을 제안하였다. 모형 개발에는 인명피해사고 발생 여부를 종속변수로 하는 이진분류형 기계학습을 활용하였으며, 기계학습 중에서는 모형의 예측 성능과 더불어 예측 결과에 대한 해석력을 중요하게 고려하는 인터프리터블 머신 러닝(Interpretable Machine Learning, IML) 방법론을 적용하였다. IML은 블랙박스 모델과 인터프리터블 모델로 구성되며, 본 연구에서는 블랙박스 모델로 KNN, RF 및 SVM을, 인터프리터블 모델로 DT와 BLR을 검토하였다. 모형 개발 시에는 각 기법에서 튜닝이 가능한 하이퍼 파라미터에 대하여 교차검증 과정을 거쳐 최적화하였다. 블랙박스 모델은 폴리노미얼 커널 트릭을 활용한 SVM을, 인터프리터블 모델은 BLR을 적용하여 인명피해사고 발생 확률을 예측하는 모형을 개발하였다. 개발된 IML 모델에 대해서는 PDR(Predictive accuracy, Descriptive accuracy and Relevancy) 프레임워크 관점에서 (typical) BLR 모델과 비교 평가를 진행하였다. 평가 결과 예측 정확도, 해석 정확도 및 인간의 이해관점에서의 적합성 등에서 모두 IML 모델이 우수함을 확인하였다. 또한, 본 연구에서 개발된 IML 모델 기반의 인명피해사고 발생 확률식은 SSES, SOR, SV, TVL, HVR 및 CR의 독립변수로 구성되었으며, 이 확률식을 기반으로 SSES 설치에 대한 효과를 정량화하였다. 정량화 분석 결과, SSES 설치로 인해 약 28% 정도의 인명피해사고 발생 확률이 감소함을 확인할 수 있었다. 또한, 모형 개발에 활용된 변수 중 SSES 설치로 인해 영향을 받는 변수들(SOR 및 SV)에 대한 매개효과 분석을 통해 SSES 설치로 인한 인명피해사고 감소 확률을 직접효과와 간접효과를 구분하여 제시하였다. 분석 결과, SSES와 제한속도 초과비율(SOR)의 관계에서 있어서는 약 30%가 간접효과이고, SSES와 속도분산(SV)의 관계에 있어서는 매개효과가 통계적으로 유의하지 않음을 확인할 수 있었다. 마지막으로 영동고속도로를 대상으로 인명피해사고 발생 확률식 기반의 예측 위험구간과 실제 인명사고 다발 구간에 대한 비교 분석을 통해 연구 결과의 활용 가능성을 확인하였다. 또한, SSES 설치 대상지 선정 시에는 사고 및 속도 분석을 통한 위험구간을 선별한 후 교통량(TVL)이 많은 곳, 통과차량 중 중차량 비율(HVR)이 높은 곳 및 구간 내 곡선비율(CR)이 높은 곳을 우선적으로 검토하는 것을 제안하였다.1. Introduction 1 1.1. Background of research 1 1.2. Objective of research 4 1.3. Research Flow 6 2. Literature Review 11 2.1. Research related to SSES 11 2.1.1. Effectiveness of SSES 11 2.1.2. Installation criteria of SSES 15 2.2. Machine learning about transportation 17 2.2.1. Machine learning algorithm 17 2.2.2. Machine learning algorithm about transportation 19 2.3. Crash prediction model 23 2.3.1. Frequency of crashes 23 2.3.2. Severity of crash 26 2.4. Interpretable Machine Learning (IML) 31 2.4.1. Introduction 31 2.4.2. Application of IML 33 3. Model Specification 37 3.1. Analysis of SSES effectiveness 37 3.1.1. Crashes analysis 37 3.1.2. Speed analysis 39 3.2. Data collection & pre-analysis 40 3.2.1. Data collection 40 3.2.2. Basic statistics of variables 42 3.3. Response variable selection 50 3.4. Model selection 52 3.4.1. Binary classification 52 3.4.2. Accuracy vs. Interpretability 53 3.4.3. Overview of IML 54 3.4.4. Process of model specification 57 4. Model development 59 4.1. Black-box and interpretable model 59 4.1.1. Consists of IML 59 4.1.2. Black-box model 60 4.1.3. Interpretable model 68 4.2. Model development 72 4.2.1. Procedure 72 4.2.2. Measures of effectiveness 74 4.2.3. K-fold cross validation 76 4.3. Result of model development 78 4.3.1. Result of black-box model 78 4.3.2. Result of interpretable model 85 5. Evaluation & Application 91 5.1. Evaluation 91 5.1.1. The PDR framework for IML 91 5.1.2. Predictive accuracy 93 5.1.3. Descriptive accuracy 94 5.1.4. Relevancy 99 5.2. Impact of Casualty Crash Reduction 102 5.2.1. Quantification of the effectiveness 102 5.2.2. Mediation effect analysis 106 5.3. Application for the Korean expressway 118 6. Conclusion 121 6.1. Summary and Findings 121 6.2. Further Research 125Docto

Modélisation formelle des systèmes de détection d'intrusions

L’écosystème de la cybersécurité évolue en permanence en termes du nombre, de la diversité, et de la complexité des attaques. De ce fait, les outils de détection deviennent inefficaces face à certaines attaques. On distingue généralement trois types de systèmes de détection d’intrusions : détection par anomalies, détection par signatures et détection hybride. La détection par anomalies est fondée sur la caractérisation du comportement habituel du système, typiquement de manière statistique. Elle permet de détecter des attaques connues ou inconnues, mais génère aussi un très grand nombre de faux positifs. La détection par signatures permet de détecter des attaques connues en définissant des règles qui décrivent le comportement connu d’un attaquant. Cela demande une bonne connaissance du comportement de l’attaquant. La détection hybride repose sur plusieurs méthodes de détection incluant celles sus-citées. Elle présente l’avantage d’être plus précise pendant la détection. Des outils tels que Snort et Zeek offrent des langages de bas niveau pour l’expression de règles de reconnaissance d’attaques. Le nombre d’attaques potentielles étant très grand, ces bases de règles deviennent rapidement difficiles à gérer et à maintenir. De plus, l’expression de règles avec état dit stateful est particulièrement ardue pour reconnaître une séquence d’événements. Dans cette thèse, nous proposons une approche stateful basée sur les diagrammes d’état-transition algébriques (ASTDs) afin d’identifier des attaques complexes. Les ASTDs permettent de représenter de façon graphique et modulaire une spécification, ce qui facilite la maintenance et la compréhension des règles. Nous étendons la notation ASTD avec de nouvelles fonctionnalités pour représenter des attaques complexes. Ensuite, nous spécifions plusieurs attaques avec la notation étendue et exécutons les spécifications obtenues sur des flots d’événements à l’aide d’un interpréteur pour identifier des attaques. Nous évaluons aussi les performances de l’interpréteur avec des outils industriels tels que Snort et Zeek. Puis, nous réalisons un compilateur afin de générer du code exécutable à partir d’une spécification ASTD, capable d’identifier de façon efficiente les séquences d’événements.Abstract : The cybersecurity ecosystem continuously evolves with the number, the diversity, and the complexity of cyber attacks. Generally, we have three types of Intrusion Detection System (IDS) : anomaly-based detection, signature-based detection, and hybrid detection. Anomaly detection is based on the usual behavior description of the system, typically in a static manner. It enables detecting known or unknown attacks but also generating a large number of false positives. Signature based detection enables detecting known attacks by defining rules that describe known attacker’s behavior. It needs a good knowledge of attacker behavior. Hybrid detection relies on several detection methods including the previous ones. It has the advantage of being more precise during detection. Tools like Snort and Zeek offer low level languages to represent rules for detecting attacks. The number of potential attacks being large, these rule bases become quickly hard to manage and maintain. Moreover, the representation of stateful rules to recognize a sequence of events is particularly arduous. In this thesis, we propose a stateful approach based on algebraic state-transition diagrams (ASTDs) to identify complex attacks. ASTDs allow a graphical and modular representation of a specification, that facilitates maintenance and understanding of rules. We extend the ASTD notation with new features to represent complex attacks. Next, we specify several attacks with the extended notation and run the resulting specifications on event streams using an interpreter to identify attacks. We also evaluate the performance of the interpreter with industrial tools such as Snort and Zeek. Then, we build a compiler in order to generate executable code from an ASTD specification, able to efficiently identify sequences of events

Advanced Data Analytics Methodologies for Anomaly Detection in Multivariate Time Series Vehicle Operating Data

Early detection of faults in the vehicle operating systems is a research domain of high significance to sustain full control of the systems since anomalous behaviors usually result in performance loss for a long time before detecting them as critical failures. In other words, operating systems exhibit degradation when failure begins to occur. Indeed, multiple presences of the failures in the system performance are not only anomalous behavior signals but also show that taking maintenance actions to keep the system performance is vital. Maintaining the systems in the nominal performance for the lifetime with the lowest maintenance cost is extremely challenging and it is important to be aware of imminent failure before it arises and implement the best countermeasures to avoid extra losses. In this context, the timely anomaly detection of the performance of the operating system is worthy of investigation. Early detection of imminent anomalous behaviors of the operating system is difficult without appropriate modeling, prediction, and analysis of the time series records of the system. Data based technologies have prepared a great foundation to develop advanced methods for modeling and prediction of time series data streams. In this research, we propose novel methodologies to predict the patterns of multivariate time series operational data of the vehicle and recognize the second-wise unhealthy states. These approaches help with the early detection of abnormalities in the behavior of the vehicle based on multiple data channels whose second-wise records for different functional working groups in the operating systems of the vehicle. Furthermore, a real case study data set is used to validate the accuracy of the proposed prediction and anomaly detection methodologies

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

Intelligent video surveillance

In the focus of this thesis are the new and modified algorithms for object detection, recognition and tracking within the context of video analytics. The manual video surveillance has been proven to have low effectiveness and, at the same time, high expense because of the need in manual labour of operators, which are additionally prone to erroneous decisions. Along with increase of the number of surveillance cameras, there is a strong need to push for automatisation of the video analytics. The benefits of this approach can be found both in military and civilian applications. For military applications, it can help in localisation and tracking of objects of interest. For civilian applications, the similar object localisation procedures can make the criminal investigations more effective, extracting the meaningful data from the massive video footage. Recently, the wide accessibility of consumer unmanned aerial vehicles has become a new threat as even the simplest and cheapest airborne vessels can carry some cargo that means they can be upgraded to a serious weapon. Additionally they can be used for spying that imposes a threat to a private life. The autonomous car driving systems are now impossible without applying machine vision methods. The industrial applications require automatic quality control, including non-destructive methods and particularly methods based on the video analysis. All these applications give a strong evidence in a practical need in machine vision algorithms for object detection, tracking and classification and gave a reason for writing this thesis. The contributions to knowledge of the thesis consist of two main parts: video tracking and object detection and recognition, unified by the common idea of its applicability to video analytics problems. The novel algorithms for object detection and tracking, described in this thesis, are unsupervised and have only a small number of parameters. The approach is based on rigid motion segmentation by Bayesian filtering. The Bayesian filter, which was proposed specially for this method and contributes to its novelty, is formulated as a generic approach, and then applied to the video analytics problems. The method is augmented with optional object coordinate estimation using plain two-dimensional terrain assumption which gives a basis for the algorithm usage inside larger sensor data fusion models. The proposed approach for object detection and classification is based on the evolving systems concept and the new Typicality-Eccentricity Data Analytics (TEDA) framework. The methods are capable of solving classical problems of data mining: clustering, classification, and regression. The methods are proposed in a domain-independent way and are capable of addressing shift and drift of the data streams. Examples are given for the clustering and classification of the imagery data. For all the developed algorithms, the experiments have shown sustainable results on the testing data. The practical applications of the proposed algorithms are carefully examined and tested

On-line Condition Monitoring, Fault Detection and Diagnosis in Electrical Machines and Power Electronic Converters

The objective of this PhD research is to develop robust, and non-intrusive condition monitoring methods for induction motors fed by closed-loop inverters. The flexible energy forms synthesized by these connected power electronic converters greatly enhance the performance and expand the operating region of induction motors. They also significantly alter the fault behavior of these electric machines and complicate the fault detection and protection. The current state of the art in condition monitoring of power-converter-fed electric machines is underdeveloped as compared to the maturing condition monitoring techniques for grid-connected electric machines. This dissertation first investigates the stator turn-to-turn fault modelling for induction motors (IM) fed by a grid directly. A novel and more meaningful model of the motor itself was developed and a comprehensive study of the closed-loop inverter drives was conducted. A direct torque control (DTC) method was selected for controlling IM’s electromagnetic torque and stator flux-linkage amplitude in industrial applications. Additionally, a new driver based on DTC rules, predictive control theory and fuzzy logic inference system for the IM was developed. This novel controller improves the performance of the torque control on the IM as it reduces most of the disadvantages of the classical and predictive DTC drivers. An analytical investigation of the impacts of the stator inter-turn short-circuit of the machine in the controller and its reaction was performed. This research sets a based knowledge and clear foundations of the events happening inside the IM and internally in the DTC when the machine is damaged by a turn fault in the stator. This dissertation also develops a technique for the health monitoring of the induction machine under stator turn failure. The developed technique was based on the monitoring of the off-diagonal term of the sequence component impedance matrix. Its advantages are that it is independent of the IM parameters, it is immune to the sensors’ errors, it requires a small learning stage, compared with NN, and it is not intrusive, robust and online. The research developed in this dissertation represents a significant advance that can be utilized in fault detection and condition monitoring in industrial applications, transportation electrification as well as the utilization of renewable energy microgrids. To conclude, this PhD research focuses on the development of condition monitoring techniques, modelling, and insightful analyses of a specific type of electric machine system. The fundamental ideas behind the proposed condition monitoring technique, model and analysis are quite universal and appeals to a much wider variety of electric machines connected to power electronic converters or drivers. To sum up, this PhD research has a broad beneficial impact on a wide spectrum of power-converter-fed electric machines and is thus of practical importance

Clustering and Classification for Time Series Data in Visual Analytics: A Survey

Visual analytics for time series data has received a considerable amount of attention. Different approaches have been developed to understand the characteristics of the data and obtain meaningful statistics in order to explore the underlying processes, identify and estimate trends, make decisions and predict the future. The machine learning and visualization areas share a focus on extracting information from data. In this paper, we consider not only automatic methods but also interactive exploration. The ability to embed efficient machine learning techniques (clustering and classification) in interactive visualization systems is highly desirable in order to gain the most from both humans and computers. We present a literature review of some of the most important publications in the field and classify over 60 published papers from six different perspectives. This review intends to clarify the major concepts with which clustering or classification algorithms are used in visual analytics for time series data and provide a valuable guide for both new researchers and experts in the emerging field of integrating machine learning techniques into visual analytics