PASS: a simple classifier system for data analysis

Let x be a vector of predictors and y a scalar response associated with it. Consider the regression problem of inferring the relantionship between predictors and response on the basis of a sample of observed pairs (x,y). This is a familiar problem for which a variety of methods are available. This paper describes a new method based on the classifier system approach to problem solving. Classifier systems provide a rich framework for learning and induction, and they have been suc:cessfully applied in the artificial intelligence literature for some time. The present method emiches the simplest classifier system architecture with some new heuristic and explores its potential in a purely inferential context. A prototype called PASS (Predictive Adaptative Sequential System) has been built to test these ideas empirically. Preliminary Monte Carlo experiments indicate that PASS is able to discover the structure imposed on the data in a wide array of cases

A Cluster-Based Opposition Differential Evolution Algorithm Boosted by a Local Search for ECG Signal Classification

Electrocardiogram (ECG) signals, which capture the heart's electrical activity, are used to diagnose and monitor cardiac problems. The accurate classification of ECG signals, particularly for distinguishing among various types of arrhythmias and myocardial infarctions, is crucial for the early detection and treatment of heart-related diseases. This paper proposes a novel approach based on an improved differential evolution (DE) algorithm for ECG signal classification for enhancing the performance. In the initial stages of our approach, the preprocessing step is followed by the extraction of several significant features from the ECG signals. These extracted features are then provided as inputs to an enhanced multi-layer perceptron (MLP). While MLPs are still widely used for ECG signal classification, using gradient-based training methods, the most widely used algorithm for the training process, has significant disadvantages, such as the possibility of being stuck in local optimums. This paper employs an enhanced differential evolution (DE) algorithm for the training process as one of the most effective population-based algorithms. To this end, we improved DE based on a clustering-based strategy, opposition-based learning, and a local search. Clustering-based strategies can act as crossover operators, while the goal of the opposition operator is to improve the exploration of the DE algorithm. The weights and biases found by the improved DE algorithm are then fed into six gradient-based local search algorithms. In other words, the weights found by the DE are employed as an initialization point. Therefore, we introduced six different algorithms for the training process (in terms of different local search algorithms). In an extensive set of experiments, we showed that our proposed training algorithm could provide better results than the conventional training algorithms.Comment: 44 pages, 9 figure

Spectral feature classification of oceanographic processes using an autonomous underwater vehicle

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2000The thesis develops and demonstrates methods of classifying ocean processes using an underwater moving platform such as an Autonomous Underwater Vehicle (AUV). The "mingled spectrum principle" is established which concisely relates observations from a moving platform to the frequency-wavenumber spectrum of the ocean process. It clearly reveals the role of the AUV speed in mingling temporal and spatial information. For classifying different processes, an AUV is not only able to jointly utilize the time-space information, but also at a tunable proportion by adjusting its cruise speed. In this respect, AUVs are advantageous compared with traditional oceanographic platforms. Based on the mingled spectrum principle, a parametric tool for designing an AUVbased spectral classifier is developed. An AUV's controllable speed tunes the separability between the mingled spectra of different processes. This property is the key to optimizing the classifier's performance. As a case study, AUV-based classification is applied to distinguish ocean convection from internal waves. The mingled spectrum templates are derived from the MIT Ocean Convection Model and the Garrett-Munk internal wave spectrum model. To allow for mismatch between modeled templates and real measurements, the AUVbased classifier is designed to be robust to parameter uncertainties. By simulation tests on the classifier, it is demonstrated that at a higher AUV speed, convection's distinct spatial feature is highlighted to the advantage of classification. Experimental data are used to test the AUV-based classifier. An AUV-borne flow measurement system is designed and built, using an Acoustic Doppler Velocimeter (ADV). The system is calibrated in a high-precision tow tank. In February 1998, the AUV acquired field data of flow velocity in the Labrador Sea Convection Experiment. The Earth-referenced vertical flow velocity is extracted from the raw measurements. The classification test result detects convection's occurrence, a finding supported by more traditional oceanographic analyses and observations. The thesis work provides an important foundation for future work in autonomous detection and sampling of oceanographic processes.This thesis research has been funded by the Office of Naval Research (ONR) under Grants NOOOl4-95-1-1316, NOO0l4-97-1-0470, and by the MIT Sea Grant College Program under Grant NA46RG0434

An ensemble-based computational approach for incremental learning in non-stationary environments related to schema- and scaffolding-based human learning

The principal dilemma in a learning process, whether human or computer, is adapting to new information, especially in cases where this new information conflicts with what was previously learned. The design of computer models for incremental learning is an emerging topic for classification and prediction of large-scale data streams undergoing change in underlying class distributions (definitions) over time; yet currently, they often ignore significant foundational learning theory that has been developed in the domain of human learning. This shortfall leads to many deficiencies in the ability to organize existing knowledge and to retain relevant knowledge for long periods of time. In this work, we introduce a unique computer-learning algorithm for incremental knowledge acquisition using an ensemble of classifiers, Learn++.NSE (Non-Stationary Environments), specifically for the case where the nature of knowledge to be learned is evolving. Learn++.NSE is a novel approach to evaluating and organizing existing knowledge (classifiers) according to the most recent data environment. Under this architecture, we address the learning problem at both the learner and supervisor end, discussing and implementing three main approaches: knowledge weighting/organization, forgetting prior knowledge, and change/drift detection. The framework is evaluated on a variety of canonical and real-world data streams (weather prediction, electricity price prediction, and spam detection). This study reveals the catastrophic effect of forgetting prior knowledge, supporting the organization technique proposed by Learn++.NSE as the most consistent performer during various drift scenarios, while also addressing the sheer difficulty in designing a system that strikes a balance between maintaining all knowledge and making decisions based only on relevant knowledge, especially in severe, unpredictable environments which are often encountered in the real-world

Spectral Pattern Recognition and Fuzzy Artmap Classification: Design Features, System Dynamics and Real World Simulation

Classification of terrain cover from satellite radar imagery represents an area of considerable current interest and research. Most satellite sensors used for land applications are of the imaging type. They record data in a variety of spectral channels and at a variety of ground resolutions. Spectral pattern recognition refers to classification procedures utilizing pixel-by-pixel spectral information as the basis for automated land cover classification. A number of methods have been developed in the past to classify pixels [resolution cells] from multispectral imagery to a priori given land cover categories. Their ability to provide land cover information with high classification accuracies is significant for work where accurate and reliable thematic information is needed. The current trend towards the use of more spectral bands on satellite instruments, such as visible and infrared imaging spectrometers, and finer pixel and grey level resolutions will offer more precise possibilities for accurate identification. But as the complexity of the data grows, so too does the need for more powerful tools to analyse them. It is the major objective of this study to analyse the capabilities and applicability of the neural pattern recognition system, called fuzzy ARTMAP, to generate high quality classifications of urban land cover using remotely sensed images. Fuzzy ARTMAP synthesizes fuzzy logic and Adaptive Resonance Theory (ART) by exploiting the formal similarity between the computations of fuzzy subsethood and the dynamics of category choice, search and learning. The paper describes design features, system dynamics and simulation algorithms of this learning system, which is trained and tested for classification (8 a priori given classes) of a multispectral image of a Landsat-5 Thematic Mapper scene (270 x 360 pixels) from the City of Vienna on a pixel-by-pixel basis. Fuzzy ARTMAP performance is compared with that of an error-based learning system based upon the multi-layer perceptron, and the Gaussian maximum likelihood classifier as conventional statistical benchmark on the same database. Both neural classifiers outperform the conventional classifier in terms of classification accuracy. Fuzzy ARTMAP leads to out-of-sample classification accuracies, very close to maximum performance, while the multi-layer perceptron--like the conventional classifier--shows difficulties to distinguish between some land use categories

Feature Selection for Text and Image Data Using Differential Evolution with SVM and Naïve Bayes Classifiers

Classification problems are increasing in various important applications such as text categorization, images, medical imaging diagnosis and bimolecular analysis etc. due to large amount of attribute set. Feature extraction methods in case of large dataset play an important role to reduce the irrelevant feature and thereby increases the performance of classifier algorithm. There exist various methods based on machine learning for text and image classification. These approaches are utilized for dimensionality reduction which aims to filter less informative and outlier data. Therefore, these approaches provide compact representation and computationally better tractable accuracy. At the same time, these methods can be challenging if the search space is doubled multiple time. To optimize such challenges, a hybrid approach is suggested in this paper. The proposed approach uses differential evolution (DE) for feature selection with naïve bayes (NB) and support vector machine (SVM) classifiers to enhance the performance of selected classifier. The results are verified using text and image data which reflects improved accuracy compared with other conventional techniques. A 25 benchmark datasets (UCI) from different domains are considered to test the proposed algorithms.  A comparative study between proposed hybrid classification algorithms are presented in this work. Finally, the experimental result shows that the differential evolution with NB classifier outperforms and produces better estimation of probability terms. The proposed technique in terms of computational time is also feasible

Machine Learning-Assisted Pattern Recognition Algorithms for Estimating Ultimate Tensile Strength in Fused Deposition Modeled Polylactic Acid Specimens

In this study, we investigate the application of supervised machine learning algorithms for estimating the Ultimate Tensile Strength (UTS) of Polylactic Acid (PLA) specimens fabricated using the Fused Deposition Modeling (FDM) process. A total of 31 PLA specimens were prepared, with Infill Percentage, Layer Height, Print Speed, and Extrusion Temperature serving as input parameters. The primary objective was to assess the accuracy and effectiveness of four distinct supervised classification algorithms, namely Logistic Classification, Gradient Boosting Classification, Decision Tree, and K-Nearest Neighbor, in predicting the UTS of the specimens. The results revealed that while the Decision Tree and K-Nearest Neighbor algorithms both achieved an F1 score of 0.71, the KNN algorithm exhibited a higher Area Under the Curve (AUC) score of 0.79, outperforming the other algorithms. This demonstrates the superior ability of the KNN algorithm in differentiating between the two classes of ultimate tensile strength within the dataset, rendering it the most favorable choice for classification in the context of this research. This study represents the first attempt to estimate the UTS of PLA specimens using machine learning-based classification algorithms, and the findings offer valuable insights into the potential of these techniques in improving the performance and accuracy of predictive models in the domain of additive manufacturing