A Rough Set Approach to Dimensionality Reduction for Performance Enhancement in Machine Learning

Machine learning uses complex mathematical algorithms to turn data set into a model for a problem domain. Analysing high dimensional data in their raw form usually causes computational overhead because the higher the size of the data, the higher the time it takes to process it. Therefore, there is a need for a more robust dimensionality reduction approach, among other existing methods, for feature projection (extraction) and selection from data set, which can be passed to a machine learning algorithm for optimal performance. This paper presents a generic mathematical approach for transforming data from a high dimensional space to low dimensional space in such a manner that the intrinsic dimension of the original data is preserved using the concept of indiscernibility, reducts, and the core of the rough set theory. The flue detection dataset available on the Kaggle website was used in this research for demonstration purposes. The original and reduced datasets were tested using a logistic regression machine learning algorithm yielding the same accuracy of 97% with a training time of 25 min and 11 min respectively

Enhancing the success rates by performing pooling decisions adjacent to the output layer

Learning classification tasks of (2^nx2^n) inputs typically consist of \le n (2x2) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using the CIFAR-10 database, that pooling decisions adjacent to the last convolutional layer significantly enhance accuracy success rates (SRs). In particular, average SRs of the advanced VGG with m layers (A-VGGm) architectures are 0.936, 0.940, 0.954, 0.955, and 0.955 for m=6, 8, 14, 13, and 16, respectively. The results indicate A-VGG8s' SR is superior to VGG16s', and that the SRs of A-VGG13 and A-VGG16 are equal, and comparable to that of Wide-ResNet16. In addition, replacing the three fully connected (FC) layers with one FC layer, A-VGG6 and A-VGG14, or with several linear activation FC layers, yielded similar SRs. These significantly enhanced SRs stem from training the most influential input-output routes, in comparison to the inferior routes selected following multiple MP decisions along the deep architecture. In addition, SRs are sensitive to the order of the non-commutative MP and average pooling operators adjacent to the output layer, varying the number and location of training routes. The results call for the reexamination of previously proposed deep architectures and their SRs by utilizing the proposed pooling strategy adjacent to the output layer.Comment: 27 pages, 3 figures, 1 table and Supplementary Informatio

ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detection

Effective feature fusion of multispectral images plays a crucial role in multi-spectral object detection. Previous studies have demonstrated the effectiveness of feature fusion using convolutional neural networks, but these methods are sensitive to image misalignment due to the inherent deffciency in local-range feature interaction resulting in the performance degradation. To address this issue, a novel feature fusion framework of dual cross-attention transformers is proposed to model global feature interaction and capture complementary information across modalities simultaneously. This framework enhances the discriminability of object features through the query-guided cross-attention mechanism, leading to improved performance. However, stacking multiple transformer blocks for feature enhancement incurs a large number of parameters and high spatial complexity. To handle this, inspired by the human process of reviewing knowledge, an iterative interaction mechanism is proposed to share parameters among block-wise multimodal transformers, reducing model complexity and computation cost. The proposed method is general and effective to be integrated into different detection frameworks and used with different backbones. Experimental results on KAIST, FLIR, and VEDAI datasets show that the proposed method achieves superior performance and faster inference, making it suitable for various practical scenarios. Code will be available at https://github.com/chanchanchan97/ICAFusion.Comment: submitted to Pattern Recognition Journal, minor revisio

A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU

Deep learning (DL) has emerged as a powerful subset of machine learning (ML) and artificial intelligence (AI), outperforming traditional ML methods, especially in handling unstructured and large datasets. Its impact spans across various domains, including speech recognition, healthcare, autonomous vehicles, cybersecurity, predictive analytics, and more. However, the complexity and dynamic nature of real-world problems present challenges in designing effective deep learning models. Consequently, several deep learning models have been developed to address different problems and applications. In this article, we conduct a comprehensive survey of various deep learning models, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Models, Deep Reinforcement Learning (DRL), and Deep Transfer Learning. We examine the structure, applications, benefits, and limitations of each model. Furthermore, we perform an analysis using three publicly available datasets: IMDB, ARAS, and Fruit-360. We compare the performance of six renowned deep learning models: CNN, Simple RNN, Long Short-Term Memory (LSTM), Bidirectional LSTM, Gated Recurrent Unit (GRU), and Bidirectional GRU.Comment: 16 pages, 29 figure

Assistive diagnostic technology for congenital heart disease based on fusion features and deep learning

Introduction: Congenital heart disease (CHD) is a cardiovascular disorder caused by structural defects in the heart. Early screening holds significant importance for the effective treatment of this condition. Heart sound analysis is commonly employed to assist in the diagnosis of CHD. However, there is currently a lack of an efficient automated model for heart sound classification, which could potentially replace the manual process of auscultation.Methods: This study introduces an innovative and efficient screening and classification model, combining a locally concatenated fusion approach with a convolutional neural network based on coordinate attention (LCACNN). In this model, Mel-frequency spectral coefficients (MFSC) and envelope features are locally fused and employed as input to the LCACNN network. This model automatically analyzes feature map energy information, eliminating the need for denoising processes.Discussion: The proposed classification model in this study demonstrates a robust capability for identifying congenital heart disease, potentially substituting manual auscultation to facilitate the detection of patients in remote areas.Results: This study introduces an innovative and efficient screening and classification model, combining a locally concatenated fusion approach with a convolutional neural network based on coordinate attention (LCACNN). In this model, Mel-frequency spectral coefficients (MFSC) and envelope features are locally fused and employed as input to the LCACNN network. This model automatically analyzes feature map energy information, eliminating the need for denoising processes. To assess the performance of the classification model, comparative ablation experiments were conducted, achieving classification accuracies of 91.78% and 94.79% on the PhysioNet and HS databases, respectively. These results significantly outperformed alternative classification models

A General Framework for Robust G-Invariance in G-Equivariant Networks

We introduce a general method for achieving robust group-invariance in group-equivariant convolutional neural networks ($G$-CNNs), which we call the $G$-triple-correlation ($G$-TC) layer. The approach leverages the theory of the triple-correlation on groups, which is the unique, lowest-degree polynomial invariant map that is also complete. Many commonly used invariant maps - such as the max - are incomplete: they remove both group and signal structure. A complete invariant, by contrast, removes only the variation due to the actions of the group, while preserving all information about the structure of the signal. The completeness of the triple correlation endows the $G$-TC layer with strong robustness, which can be observed in its resistance to invariance-based adversarial attacks. In addition, we observe that it yields measurable improvements in classification accuracy over standard Max $G$-Pooling in $G$-CNN architectures. We provide a general and efficient implementation of the method for any discretized group, which requires only a table defining the group's product structure. We demonstrate the benefits of this method for $G$-CNNs defined on both commutative and non-commutative groups - $SO(2)$, $O(2)$, $SO(3)$, and $O(3)$ (discretized as the cyclic $C8$, dihedral $D16$, chiral octahedral $O$ and full octahedral $O_h$ groups) - acting on $\mathbb{R}^2$ and $\mathbb{R}^3$ on both $G$-MNIST and $G$-ModelNet10 datasets

Kaba küme tabanlı çok kriterli karar verme yöntemi ve uygulaması

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Çok kriterli karar verme problemi, çağımız yöneticilerinin sıklıkla başvurmuş olduğu yöntemlerden birisidir. Verilerin belirsiz ya da eksik olması durumunda, mevcut olan çok kriterli karar verme yöntemleri yetersiz kalırken, önermiş olduğumuz kaba küme tabanlı çok kriterli karar verme algoritması, bu eksikliği gidermede en büyük yardımcı olarak karşımıza çıkmaktadır. Bununla birlikte, hızla artan veri trafiğinde, mevcut verilerin verimli bir şekilde kullanılması da beraberinde önemli bir durumu ortaya çıkartmaktadır. 1982 yılında ilk olarak Pawlak[1] tarafından önerilen kaba küme kavramı, büyük veri tabanlarını kullanarak gerekli olan bilginin keşfini sağlayan önemli bir araç olarak kullanılmaktadır. Kaba küme kavramı, çok kriterli karar verme problemlerinde kullanılmak üzere, kesin olmayan yapıların analizi için bulanık mantık yaklaşımından türetilmiştir. Kaba küme teorisi, kural indirgeme ve sınıflandırma yaklaşım özellikleri ile büyük verilerin analiz işleminin yanı sıra çok kriterli karar verme problemlerinde de kullanılabilmektedir. Kaba küme teorisi bulanık küme teorisinin bir alt kolu olarak geliştirilmiştir. Eksik, belirsiz verilerin değerlendirilmesi sürecinde, alt ve üst yaklaşımlar kullanılarak, veriler analiz edilmektedir. Bulanık kümeler gibi kesin sınırlamaları içermeyen bir yapıya sahiptir. Eksik bilgi analizi, bilgi tabanı indirgemesi yöntemleri kullanılarak, verilerdeki belirsizlik en aza indirgenmeye çalışılmaktadır. Tutarsız, eksik bilgi içeren veri yapılarından kural çıkarımı ve sınıflandırma konusunda kaba küme teorisi ilerleyen zamanlarda daha fazla tercih edilecek bir yöntem olarak çıkabilecektir. Bu çalışmada kaba kümeleme teorisine ait temel kavramlar kaba küme tabanlı bilgi keşfi ve kaba küme kavramı dikkate alınarak geliştirilen algoritma ile birlikte, çok kriterli karar verme probleminin çözümüne yönelik algoritma geliştirilmiştir ve diğer ÇKKV algoritmaları ile karşılaştırılmıştır. Anahtar kelimeler:Kaba Küme Teorisi, Çok Kriterli Karar Verme EntropiThe multi-criteria decision-making problem is one of the methods that preffered and applied by the managers. Multi criteria decision making data set may include the uncertain or incomplete data, in this situation, decision is getting difficult and impossible, the suggested rough set based multi criteria decision making algorithm can able to solve this manner problem. However, in the rapidly increasing data traffic, the efficient use of existing data also brings about an important situation. The rough set concept firstly proposed by Pawlak in 1982[1] that is used as an important tool for the discovery of the necessary information by using large databases. In the case of multi-criteria decision-making problems, the concept of rough set theory is derived from the fuzzy logic approach to perform the analysis of uncertain structures. The rough set theory also has the property of being able to be used in multi-criteria decision-making problems with the rules of rule reduction and classification during the analysis of large data. Rough set theory has a structure that does not contain definite limitations, such as fuzzy sets. Therefore, the rough set approach can able to analysis of the incomplete, inadequate and ambiguous information suitable for data analysis, uses incomplete information analysis, knowledge base reduction methods during this process. Rough set theory can be used as a natural method that deals with inconsistent and incomplete information, which is the basic problem of rule extraction and classification. In this study, the basic concepts of rough set theory is given. The algorithm for solving multi-criteria decision making has been developed by considering the rough set based knowledge discovery and rough set concept. Keywords: Rough Set Theory, Multi Criteria Decision Making Entrop

Granular computing approach for intelligent classifier design

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.Granular computing facilitates dealing with information by providing a theoretical framework to deal with information as granules at different levels of granularity (different levels of specificity/abstraction). It aims to provide an abstract explainable description of the data by forming granules that represent the features or the underlying structure of corresponding subsets of the data. In this thesis, a granular computing approach to the design of intelligent classification systems is proposed. The proposed approach is employed for different classification systems to investigate its efficiency. Fuzzy inference systems, neural networks, neuro-fuzzy systems and classifier ensembles are considered to evaluate the efficiency of the proposed approach. Each of the considered systems is designed using the proposed approach and classification performance is evaluated and compared to that of the standard system. The proposed approach is based on constructing information granules from data at multiple levels of granularity. The granulation process is performed using a modified fuzzy c-means algorithm that takes classification problem into account. Clustering is followed by a coarsening process that involves merging small clusters into large ones to form a lower granularity level. The resulted granules are used to build each of the considered binary classifiers in different settings and approaches. Granules produced by the proposed granulation method are used to build a fuzzy classifier for each granulation level or set of levels. The performance of the classifiers is evaluated using real life data sets and measured by two classification performance measures: accuracy and area under receiver operating characteristic curve. Experimental results show that fuzzy systems constructed using the proposed method achieved better classification performance. In addition, the proposed approach is used for the design of neural network classifiers. Resulted granules from one or more granulation levels are used to train the classifiers at different levels of specificity/abstraction. Using this approach, the classification problem is broken down into the modelling of classification rules represented by the information granules resulting in more interpretable system. Experimental results show that neural network classifiers trained using the proposed approach have better classification performance for most of the data sets. In a similar manner, the proposed approach is used for the training of neuro-fuzzy systems resulting in similar improvement in classification performance. Lastly, neural networks built using the proposed approach are used to construct a classifier ensemble. Information granules are used to generate and train the base classifiers. The final ensemble output is produced by a weighted sum combiner. Based on the experimental results, the proposed approach has improved the classification performance of the base classifiers for most of the data sets. Furthermore, a genetic algorithm is used to determine the combiner weights automatically.Higher Committee for Education Development in Iraq (HCED

Intelligent Sensors for Human Motion Analysis

The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems