A Bonferroni Mean Based Fuzzy K Nearest Centroid Neighbor Classifier

K-nearest neighbor (KNN) is an effective nonparametric classifier that determines the neighbors of a point based only on distance proximity. The classification performance of KNN is disadvantaged by the presence of outliers in small sample size datasets and its performance deteriorates on datasets with class imbalance. We propose a local Bonferroni Mean based Fuzzy K-Nearest Centroid Neighbor (BM-FKNCN) classifier that assigns class label of a query sample dependent on the nearest local centroid mean vector to better represent the underlying statistic of the dataset. The proposed classifier is robust towards outliers because the Nearest Centroid Neighborhood (NCN) concept also considers spatial distribution and symmetrical placement of the neighbors. Also, the proposed classifier can overcome class domination of its neighbors in datasets with class imbalance because it averages all the centroid vectors from each class to adequately interpret the distribution of the classes. The BM-FKNCN classifier is tested on datasets from the Knowledge Extraction based on Evolutionary Learning (KEEL) repository and benchmarked with classification results from the KNN, Fuzzy-KNN (FKNN), BM-FKNN and FKNCN classifiers. The experimental results show that the BM-FKNCN achieves the highest overall average classification accuracy of 89.86% compared to the other four classifiers

Finger Vein Recognition Using Principle Component Analysis and Adaptive k-Nearest Centroid Neighbor Classifier

The k-nearest centroid neighbor kNCN classifier is one of the non-parametric classifiers which provide a powerful decision based on the geometrical surrounding neighborhood. Essentially, the main challenge in the kNCN is due to slow classification time that utilizing all training samples to find each nearest centroid neighbor. In this work, an adaptive k-nearest centroid neighbor (akNCN) is proposed as an improvement to the kNCN classifier. Two new rules are introduced to adaptively select the neighborhood size of the test sample. The neighborhood size for the test sample is changed through the following ways: 1) The neighborhood size, k will be adapted to j if the centroid distance of j-th nearest centroid neighbor is greater than the predefined boundary. 2) There is no need to look for further nearest centroid neighbors if the maximum number of samples of the same class is found among jth nearest centroid neighbor. Thus, the size of neighborhood is adaptively changed to j. Experimental results on theFinger Vein USM (FV-USM) image database demonstrate the promising results in which the classification time of the akNCN classifier is significantly reduced to 51.56% in comparison to the closest competitors, kNCN and limited-kNCN. It also outperforms its competitors by achieving the best reduction ratio of 12.92% whilemaintaining the classification accuracy

Analysis and improvement proposal on self-supervised deep learning

Self-supervised learning is an emerging deep learning paradigm that aims at removing the label-dependency problems suffered by most supervised learning algorithms. Instance discrimination algorithms have proved to be very successful as they have reduced the gap between supervised and self-supervised ones to less than 5%. While most instance discrimination approaches focus on contrasting two augmentations of the same image, Neighbour Contrastive Learning approaches aim to increase the generalization of deep networks by pulling together representations from different images (neighbours) that belong to the same semantical class. However, they are limited mainly by their low accuracy regarding the neighbour selection. They also suffer from reduced efficiency while using multiple neighbours. Instance discrimination algorithms have their own particularities in solving the learning problem, and combining different approaches, bringing in the best of algorithms, is very interesting. In this thesis, we propose a neighbour contrast learning method called Musketeer. This method introduces Self-attention operations to create single representations, defined as centroids, from the extracted neighbours. Directly contrasting these centroids increases the neighbour retrieval accuracy while avoiding any efficiency loss. Moreover, Musketeer combines its neighbour contrast objective with a feature redundancy reduction objective, forming a symbiosis that proves to be beneficial in the overall performance of the framework. Our proposed symbiotic approach consistently outperforms SoTA instance discrimination frameworks on popular image classification benchmarking datasets, namely, CIFAR-10, CIFAR-100 and ImageNet-100. Additionally, we build an analysis pipeline that further explores the quantitative and qualitative results, providing numerous insights into the explainability of instance discrimination approaches

Freeman chain code as representation in offline signature verification system

Over recent years, there has been an explosive growth of interest in the pattern recognition. For example, handwritten signature is one of human biometric that can be used in many areas in terms of access control and security. However, handwritten signature is not a uniform characteristic such as fingerprint, iris or vein. It may change to several factors; mood, environment and age. Signature Verification System (SVS) is a part of pattern recognition that can be a solution for such situation. The system can be decomposed into three stages: data acquisition and preprocessing, feature extraction and verification. This paper presents techniques for SVS that uses Freeman chain code (FCC) as data representation. In the first part of feature extraction stage, the FCC was extracted by using boundary-based style on the largest contiguous part of the signature images. The extracted FCC was divided into four, eight or sixteen equal parts. In the second part of feature extraction, six global features were calculated. Finally, verification utilized k-Nearest Neighbour (k-NN) to test the performance. MCYT bimodal database was used in every stage in the system. Based on our systems, the best result achieved was False Rejection Rate (FRR) 14.67%, False Acceptance Rate (FAR) 15.83% and Equal Error Rate (EER) 0.43% with shortest computation, 7.53 seconds and 47 numbers of features

ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning

Confidence-based pseudo-labeling is among the dominant approaches in semi-supervised learning (SSL). It relies on including high-confidence predictions made on unlabeled data as additional targets to train the model. We propose ProtoCon, a novel SSL method aimed at the less-explored label-scarce SSL where such methods usually underperform. ProtoCon refines the pseudo-labels by leveraging their nearest neighbours' information. The neighbours are identified as the training proceeds using an online clustering approach operating in an embedding space trained via a prototypical loss to encourage well-formed clusters. The online nature of ProtoCon allows it to utilise the label history of the entire dataset in one training cycle to refine labels in the following cycle without the need to store image embeddings. Hence, it can seamlessly scale to larger datasets at a low cost. Finally, ProtoCon addresses the poor training signal in the initial phase of training (due to fewer confident predictions) by introducing an auxiliary self-supervised loss. It delivers significant gains and faster convergence over state-of-the-art across 5 datasets, including CIFARs, ImageNet and DomainNet.Comment: Accepted in CVPR2023 (highlight

One-class classifiers based on entropic spanning graphs

One-class classifiers offer valuable tools to assess the presence of outliers in data. In this paper, we propose a design methodology for one-class classifiers based on entropic spanning graphs. Our approach takes into account the possibility to process also non-numeric data by means of an embedding procedure. The spanning graph is learned on the embedded input data and the outcoming partition of vertices defines the classifier. The final partition is derived by exploiting a criterion based on mutual information minimization. Here, we compute the mutual information by using a convenient formulation provided in terms of the $\alpha$-Jensen difference. Once training is completed, in order to associate a confidence level with the classifier decision, a graph-based fuzzy model is constructed. The fuzzification process is based only on topological information of the vertices of the entropic spanning graph. As such, the proposed one-class classifier is suitable also for data characterized by complex geometric structures. We provide experiments on well-known benchmarks containing both feature vectors and labeled graphs. In addition, we apply the method to the protein solubility recognition problem by considering several representations for the input samples. Experimental results demonstrate the effectiveness and versatility of the proposed method with respect to other state-of-the-art approaches.Comment: Extended and revised version of the paper "One-Class Classification Through Mutual Information Minimization" presented at the 2016 IEEE IJCNN, Vancouver, Canad