Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization

Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. They traditionally assume equal misclassification costs for each class; however, real-world problems may have an uneven class distribution. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs. EBCS-SVM handles imbalanced classification problems by simultaneously learning the support vectors and optimizing the SVM hyperparameters, which comprise the kernel parameter and misclassification costs. The resulting optimization problem is a bilevel problem, where the lower level determines the support vectors and the upper level the hyperparameters. This optimization problem is solved using an evolutionary algorithm (EA) at the upper level and sequential minimal optimization (SMO) at the lower level. These two methods work in a nested fashion, that is, the optimal support vectors help guide the search of the hyperparameters, and the lower level is initialized based on previous successful solutions. The proposed method is assessed using 70 datasets of imbalanced classification and compared with several state-of-the-art methods. The experimental results, supported by a Bayesian test, provided evidence of the effectiveness of EBCS-SVM when working with highly imbalanced datasets.Comment: Copyright 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other work

Design Preference Elicitation, Identification and Estimation.

Understanding user preference has long been a challenging topic in the design research community. Econometric methods have been adopted to link design and market, achieving design solutions sound from both engineering and business perspectives. This approach, however, only refines existing designs from revealed or stated preference data. What is needed for generating new designs is an environment for concept exploration and a channel to collect and analyze preferences on newly-explored concepts. This dissertation focuses on the development of querying techniques that learn and extract individual preferences efficiently. Throughout the dissertation, we work in the context of a human-computer interaction where in each iteration the subject is asked to choose preferred designs out of a set. The computer learns from the subject and creates the next query set so that the responses from the subject will yield the most information on the subject's preferences. The challenges of this research are: (1) To learn subject preferences within short interactions with enormous candidate designs; (2) To facilitate real-time interactions with efficient computation. Three problems are discussed surrounding how information-rich queries can be made. The major effort is devoted to preference elicitation, where we discuss how to locate the most preferred design of a subject. Using efficient global optimization, we develop search algorithms that combine exploration of new concepts and exploitation of existing knowledge, achieving near-optimal solutions with a small number of queries. For design demonstration, the elicitation algorithm is incorporated with an online 3D car modeler. The effectiveness of the algorithm is confirmed by real user tests on finding car models close to the users' targets. In preference identification, we consider designs as binary labeled, and the objective is to classify preferred designs from not-preferred ones. We show that this classification problem can be formulated and solved by the same active learning technique used for preference estimation, where the objective is to estimate a preference function. Conceptually, this dissertation discusses how to extract preference information effectively by asking relevant but not redundant questions during an interaction.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/91578/1/yiren_1.pd

Using rule extraction to improve the comprehensibility of predictive models.

Whereas newer machine learning techniques, like artifficial neural net-works and support vector machines, have shown superior performance in various benchmarking studies, the application of these techniques remains largely restricted to research environments. A more widespread adoption of these techniques is foiled by their lack of explanation capability which is required in some application areas, like medical diagnosis or credit scoring. To overcome this restriction, various algorithms have been proposed to extract a meaningful description of the underlying `blackbox' models. These algorithms' dual goal is to mimic the behavior of the black box as closely as possible while at the same time they have to ensure that the extracted description is maximally comprehensible. In this research report, we first develop a formal definition of`rule extraction and comment on the inherent trade-off between accuracy and comprehensibility. Afterwards, we develop a taxonomy by which rule extraction algorithms can be classiffied and discuss some criteria by which these algorithms can be evaluated. Finally, an in-depth review of the most important algorithms is given.This report is concluded by pointing out some general shortcomings of existing techniques and opportunities for future research.Models; Model; Algorithms; Criteria; Opportunities; Research; Learning; Neural networks; Networks; Performance; Benchmarking; Studies; Area; Credit; Credit scoring; Behavior; Time;

Ensemble of texture descriptors and classifiers for face recognition

Abstract Presented in this paper is a novel system for face recognition that works well in the wild and that is based on ensembles of descriptors that utilize different preprocessing techniques. The power of our proposed approach is demonstrated on two datasets: the FERET dataset and the Labeled Faces in the Wild (LFW) dataset. In the FERET datasets, where the aim is identification, we use the angle distance. In the LFW dataset, where the aim is to verify a given match, we use the Support Vector Machine and Similarity Metric Learning. Our proposed system performs well on both datasets, obtaining, to the best of our knowledge, one of the highest performance rates published in the literature on the FERET datasets. Particularly noteworthy is the fact that these good results on both datasets are obtained without using additional training patterns. The MATLAB source of our best ensemble approach will be freely available at https://www.dei.unipd.it/node/2357

Study on multi-SVM systems and their applications to pattern recognition

制度:新 ; 報告番号:甲3136号 ; 学位の種類:博士(工学) ; 授与年月日:2010/7/12 ; 早大学位記番号:新541

Reliability problems in eartquake engineering

This monograph deals with the problem of reliability analysis in the field of Earthquake Engineering. Chapter 1 is devoted to a summary of the most widely used reliability methods, with emphasis on Monte Carlo and solver surrogate techniques used in the subsequent chapters. Chapter 2 presents a discussion of the Monte Carlo from the viewpoint of Information Theory. Then, a discussion is made in Chapter 3 on the selection of random variables in Earthquake Engineering. Next, some practical methods for computing failure probabilities under seismic loads are reported in Chapter 4. Finally, a method for reliability-based design optimization under seismic loads is presented in Chapter 5

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

A Survey of Fingerprint Classification Part I: Taxonomies on Feature Extraction Methods and Learning Models

This paper reviews the fingerprint classification literature looking at the problem from a double perspective. We first deal with feature extraction methods, including the different models considered for singular point detection and for orientation map extraction. Then, we focus on the different learning models considered to build the classifiers used to label new fingerprints. Taxonomies and classifications for the feature extraction, singular point detection, orientation extraction and learning methods are presented. A critical view of the existing literature have led us to present a discussion on the existing methods and their drawbacks such as difficulty in their reimplementation, lack of details or major differences in their evaluations procedures. On this account, an experimental analysis of the most relevant methods is carried out in the second part of this paper, and a new method based on their combination is presented.Research Projects CAB(CDTI) TIN2011-28488 TIN2013-40765Spanish Government FPU12/0490