Heuristic creation of deep rule ensemble through iterative expansion of feature space

Rule learning approaches, which essentially aim to gerenate a decision tree or a set of “if-then” rules, have been popularly used in practice for automatically building rule-based models for prediction tasks, e.g., classification and regression. The key strength of rule-based models is their ability to interpret how an output is obtained given an input, in comparison with models trained by other machine learning approaches, e.g., neural networks. Moreover, ensemble learning approaches have been adopted as a popular way for advancing the performance of rule-based prediction through producing multiple rule-based models with diversity. Traditional approaches of ensemble learning are typically designed to train a single ensemble. In recent years, there have been some studies on creation of multiple ensembles towards increasing the diversity among rule-based models and the depth of ensemble learning. In this paper, we propose a feature expansion driven approach for automatic creation of deep rule ensembles, i.e., the dimensionality of the feature space is increased at each iteration by adding features newly created at the previous iteration. The proposed approach is compared with more recent approaches of rule learning and ensemble creation. The experimental results show that the proposed approach achieves improved performance on various data sets

Multi-perspective creation of diversity for image classification in ensemble learning context

Image classification is a special type of classification tasks in the setting of supervised machine learning. In general, in order to achieve good performance of image classification, it is important to select high quality features for training classifiers. However, different instances of images would usually present very diverse features even if the instances belong to the same class. In other words, one types of features may better describe some instances, whereas other instances present more other types of features. The above description would indicate that the adoption of feature selection is likely to result in the case that redundant features are removed from some instances leading to more effective recognition but some important information may get lost from other instances leading to more difficulty in image classification. On the other hand, image features are typically in the form of continuous attributes which can be handled by decision tree learning algorithms in various ways, leading to diverse classifiers being trained. In this paper, we investigate diversified adoption of the C4.5 and KNN algorithms from different perspectives, such as diversified use of features and various ways of handling continuous attributes. In particular, we propose a multi-perspective approach of diversity creation for image classification in the setting of ensemble learning. We compare the proposed approach with those popular algorithms that are used to train classifiers on either a full set of original features or a subset of selected features for image classification. The experimental results show that the performance of image classification is improved through the adoption of our proposed approach of ensemble creation

Multi-task ensemble creation for advancing performance of image segmentation

Image classification is a special type of applied machine learning tasks, where each image can be treated as an instance if there is only one target object that belongs to a specific class and needs to be recognized from an image. In the case of recognizing multiple target objects from an image, the image classification task can be formulated as image segmentation, leading to multiple instances being extracted from an image. In the setting of machine learning, each instance newly extracted from an image belongs to a specific class (a special type of target objects to be recognized) and presents specific features. In this context, in order to achieve effective recognition of each target object, it is crucial to undertake effective selection of features relevant to each specific class and appropriate setting of the training of classifiers on the selected features. In this paper, a multi-task approach of ensemble creation is proposed. The proposed approach is designed to first adopt multiple methods of multi-task feature selection for obtaining multiple groups of feature subsets (i.e., multiple subsets of features selected for each class), then to employ the KNN algorithm to create an ensemble of classifiers using each group of feature subsets resulting from a specific one of the multi-task feature selection methods, and finally all the ensembles are fused to classify each instance. We compare the performance obtained using our proposed way of ensemble creation with the one obtained using a single classifier trained on either a full set of original features or a reduced set of features selected using a single method of feature selection. The experimental results show some advances achieved in the image segmentation performance through using our proposed ensemble creation approaches, in comparison with the use of existing methods

Multi-stage mixed rule learning approach for advancing performance of rule-based classification

Rule learning is a special type of machine learning approaches, and its key advantage is the generation of interpretable models, which provides a transparent process of showing how an input is mapped to an output. Traditional rule learning algorithms are typically based on Boolean logic for inducing rule antecedents, which are very effective for training models on data sets that involve discrete attributes only. When continuous attributes are present in a data set, traditional rule learning approaches need to employ crisp intervals. However, in reality, problems usually show shades of grey, which motivated the development of fuzzy rule learning approaches by employing fuzzy intervals for handling continuous attributes. While a data set contains a large portion of discrete attributes or even no continuous attributes, fuzzy approaches cannot be used to learn rules effectively, leading to a drop in the performance. In this paper, a multi-stage approach of mixed rule learning is proposed, which involves strategic combination of both traditional and fuzzy approaches to handle effectively various types of attributes. We compare our proposed approach with existing algorithms of rule learning. Our experimental results show that our proposed approach leads to significant advances in the performance compared with the existing algorithms

Multi-level fusion of classifiers through fuzzy ensemble learning

Classification is a popular task of supervised machine learning, which can be achieved by training a single classifier or a group of classifiers. In general, the performance of each traditional learning algorithm which leads to the production of a single classifier is varied on different data sets, i.e., each learning algorithm may produce good classifiers on some data sets, but may produce poor classifiers on the other data sets. In order to achieve a more stable performance of machine learning, ensemble learning has been undertaken more popularly to produce a group of classifiers that can be complementary to each other. In this paper, we focus on advancing fuzzy classification through multi-level fusion of fuzzy classifiers in the setting of ensemble learning. In particular, we propose an ensemble learning framework that leads to creating a group of fuzzy classifiers that are complementary to each other. The experimental results show that the proposed ensemble learning framework leads to considerable advances in the performance of fuzzy classification, in comparison with using each single fuzzy classifier

Heuristic target class selection for advancing performance of coverage-based rule learning

Rule learning is a popular branch of machine learning, which can provide accurate and interpretable classification results. In general, two main strategies of rule learning are referred to as 'divide and conquer' and 'separate and con-quer'. Decision tree generation that follows the former strategy has a serious drawback, which is known as the replicated sub-tree problem, resulting from the constraint that all branches of a decision tree must have one or more common attributes. The above problem is likely to result in high computational complexity and the risk of overfitting, which leads to the necessity to develop rule learning algorithms (e.g., Prism) that follow the separate and conquer strategy. The replicated sub-tree problem can be effectively solved using the Prism algorithm , but the trained models are still complex due to the need of training an independent rule set for each selected target class. In order to reduce the risk of overfitting and the model complexity, we propose in this paper a variant of the Prism algorithm referred to as PrismCTC. The experimental results show that the PrismCTC algorithm leads to advances in classification performance and reduction of model complexity, in comparison with the C4.5 and Prism algorithms

Subclass-based semi-random data partitioning for improving sample representativeness

In machine learning tasks, it is essential for a data set to be partitioned into a training set and a test set in a specific ratio. In this context, the training set is used for learning a model for making predictions on new instances, whereas the test set is used for evaluating the prediction accuracy of a model on new instances. In the context of human learning, a training set can be viewed as learning material that covers knowledge, whereas a test set can be viewed as an exam paper that provides questions for students to answer. In practice, data partitioning has typically been done by randomly selecting 70% instances for training and the rest for testing. In this paper, we argue that random data partitioning is likely to result in the sample representativeness issue, i.e., training and test instances show very dissimilar characteristics leading to the case similar to testing students on material that was not taught. To address the above issue, we propose a subclass-based semi-random data partitioning approach. The experimental results show that the proposed data partitioning approach leads to significant advances in learning performance due to the improvement of sample representativeness

Effective Radiotherapy Cured Cauda Equina Syndrome Caused by Remitted Intracranial Germinoma Depositing

Cauda equina syndrome (CES) in children is very rare and can permanently disable. A remitted intracranial germinoma depositing on the spinal cord, leading to CES, has never been reported. We discuss the case of a 10-year-old girl who presented with sudden ataxia, low back pain, sensory deficits of the left lower extremity, and difficulty urinating and defecating 7 months after totally remitted intracranial germinoma postintracranial surgery and cranial irradiation. Magnetic resonance imaging (MRI) of the brain and spine showed multiple intradural extramedullary homogeneous masses from the cervical to lumbar levels, compressing the conus medullaris and cauda equina. After emergent craniospinal irradiation, the patient's neurologic symptoms dramatically subsided. A remitted intracranial germinoma depositing on her spinal cord could be the cause of CES. Early identification and a proper craniospinal irradiation may halt the progression of symptoms

Determining rotation cycle and distribution frequency for a vendor-buyer integrated multi-item system considering an external provider and rework

Transnational corporations, which operate in competitive global marketplaces, have to build the best possible intra-supply chain model for meeting, on time, clients’ need for multiple products with the requisite quality. Since fabrication capacity is always limited, the introduction of the external provider option can assist in leveling utilization, smoothing manufacturing schedules, eliminating overtime usage, and shortening the length of the fabrication cycle. Seeking to support intra-supply chain planning, this research aims to provide a concurrent decision on rotation cycle length and delivery frequency for a multi-item vendor-buyer incorporated type of intra-supply chain system with an external provider and rework. First, a model is built to represent this hybrid inventory replenishing problem. Then, renewal reward theory, mathematical derivation, and Hessian matrix equations are utilized to arrive at the expected total cost of the model, as well as the best policies for both cycle time and distribution. Last, the applicability and sensitivity analyses of our results are exhibited by a numerical demonstration. The insights obtained from this study about critical system-related information, such as the individual and joint impacts of the variation in outsourcing and reworking-related features on the system’s optimal operating policy and various performance parameters, will offer crucial help to the managerial functions of planning and decision making in firms using this realistic multi-item hybrid intra-supply chain system