Deep Metric Learning with Soft Orthogonal Proxies

Deep Metric Learning (DML) models rely on strong representations and similarity-based measures with specific loss functions. Proxy-based losses have shown great performance compared to pair-based losses in terms of convergence speed. However, proxies that are assigned to different classes may end up being closely located in the embedding space and hence having a hard time to distinguish between positive and negative items. Alternatively, they may become highly correlated and hence provide redundant information with the model. To address these issues, we propose a novel approach that introduces Soft Orthogonality (SO) constraint on proxies. The constraint ensures the proxies to be as orthogonal as possible and hence control their positions in the embedding space. Our approach leverages Data-Efficient Image Transformer (DeiT) as an encoder to extract contextual features from images along with a DML objective. The objective is made of the Proxy Anchor loss along with the SO regularization. We evaluate our method on four public benchmarks for category-level image retrieval and demonstrate its effectiveness with comprehensive experimental results and ablation studies. Our evaluations demonstrate the superiority of our proposed approach over state-of-the-art methods by a significant margin

Decoding Clinical Biomarker Space of COVID-19: Exploring Matrix Factorization-based Feature Selection Methods

One of the most critical challenges in managing complex diseases like COVID-19 is to establish an intelligent triage system that can optimize the clinical decision-making at the time of a global pandemic. The clinical presentation and patients’ characteristics are usually utilized to identify those patients who need more critical care. However, the clinical evidence shows an unmet need to determine more accurate and optimal clinical biomarkers to triage patients under a condition like the COVID-19 crisis. Here we have presented a machine learning approach to find a group of clinical indicators from the blood tests of a set of COVID-19 patients that are predictive of poor prognosis and morbidity. Our approach consists of two interconnected schemes: Feature Selection and Prognosis Classification. The former is based on different Matrix Factorization (MF)-based methods, and the latter is performed using Random Forest algorithm. Our model reveals that Arterial Blood Gas (ABG) O2 Saturation and C-Reactive Protein (CRP) are the most important clinical biomarkers determining the poor prognosis in these patients. Our approach paves the path of building quantitative and optimized clinical management systems for COVID-19 and similar diseases

High dimensionality reduction by matrix factorization for systems pharmacology

The extraction of predictive features from the complex high-dimensional multi-omic data is necessary for decoding and overcoming the therapeutic responses in systems pharmacology. Developing computational methods to reduce high-dimensional space of features in in vitro, in vivo and clinical data is essential to discover the evolution and mechanisms of the drug responses and drug resistance. In this paper, we have utilized the matrix factorization (MF) as a modality for high dimensionality reduction in systems pharmacology. In this respect, we have proposed three novel feature selection methods using the mathematical conception of a basis for features. We have applied these techniques as well as three other MF methods to analyze eight different gene expression datasets to investigate and compare their performance for feature selection. Our results show that these methods are capable of reducing the feature spaces and find predictive features in terms of phenotype determination. The three proposed techniques outperform the other methods used and can extract a 2-gene signature predictive of a tyrosine kinase inhibitor treatment response in the Cancer Cell Line Encyclopedia.</p

Unsupervised feature selection based on variance–covariance subspace distance

Funding Information: We acknowledge the support of following EU projects: CALLISTO ( 101004152 ), E-Vita ( 101016453 ), and the ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) Dresden/Leipzig ( IS18026A-F ). This work was also supported by the Research Council of Finland (Flagship programme: Finnish Center for Artificial Intelligence FCAI , and grants 336033 , 352986 , 358246 ) and EU (H2020 grant 101016775 and NextGenerationEU). | openaire: EC/H2020/101016775/EU//INTERVENESubspace distance is an invaluable tool exploited in a wide range of feature selection methods. The power of subspace distance is that it can identify a representative subspace, including a group of features that can efficiently approximate the space of original features. On the other hand, employing intrinsic statistical information of data can play a significant role in a feature selection process. Nevertheless, most of the existing feature selection methods founded on the subspace distance are limited in properly fulfilling this objective. To pursue this void, we propose a framework that takes a subspace distance into account which is called “Variance–Covariance subspace distance”. The approach gains advantages from the correlation of information included in the features of data, thus determines all the feature subsets whose corresponding Variance–Covariance matrix has the minimum norm property. Consequently, a novel, yet efficient unsupervised feature selection framework is introduced based on the Variance–Covariance distance to handle both the dimensionality reduction and subspace learning tasks. The proposed framework has the ability to exclude those features that have the least variance from the original feature set. Moreover, an efficient update algorithm is provided along with its associated convergence analysis to solve the optimization side of the proposed approach. An extensive number of experiments on nine benchmark datasets are also conducted to assess the performance of our method from which the results demonstrate its superiority over a variety of state-of-the-art unsupervised feature selection methods. The source code is available at https://github.com/SaeedKarami/VCSDFS.Peer reviewe

Dual Regularized Unsupervised Feature Selection Based on Matrix Factorization and Minimum Redundancy with application in gene selection

Gene expression data have become increasingly important in machine learning and computational biology over the past few years. In the field of gene expression analysis, several matrix factorization-based dimensionality reduction methods have been developed. However, such methods can still be improved in terms of efficiency and reliability. In this paper, an innovative approach to feature selection, called Dual Regularized Unsupervised Feature Selection Based on Matrix Factorization and Minimum Redundancy (DR-FS-MFMR), is introduced. The major focus of DR-FS-MFMR is to discard redundant features from the set of original features. In order to reach this target, the primary feature selection problem is defined in terms of two aspects: (1) the matrix factorization of data matrix in terms of the feature weight matrix and the representation matrix, and (2) the correlation information related to the selected features set. Then, the objective function is enriched by employing two data representation characteristics along with an inner product regularization criterion to perform both the redundancy minimization process and the sparsity task more precisely. To demonstrate the proficiency of the DR-FS-MFMR method, a large number of experimental studies are conducted on nine gene expression datasets. The obtained computational results indicate the efficiency and productivity of DR-FS-MFMR for the gene selection task. © 2022 The Author(s

Dual Regularized Unsupervised Feature Selection Based on Matrix Factorization and Minimum Redundancy with application in gene selection

Abstract Gene expression data have become increasingly important in machine learning and computational biology over the past few years. In the field of gene expression analysis, several matrix factorization-based dimensionality reduction methods have been developed. However, such methods can still be improved in terms of efficiency and reliability. In this paper, an innovative approach to feature selection, called Dual Regularized Unsupervised Feature Selection Based on Matrix Factorization and Minimum Redundancy (DR-FS-MFMR), is introduced. The major focus of DR-FS-MFMR is to discard redundant features from the set of original features. In order to reach this target, the primary feature selection problem is defined in terms of two aspects: (1) the matrix factorization of data matrix in terms of the feature weight matrix and the representation matrix, and (2) the correlation information related to the selected features set. Then, the objective function is enriched by employing two data representation characteristics along with an inner product regularization criterion to perform both the redundancy minimization process and the sparsity task more precisely. To demonstrate the proficiency of the DR-FS-MFMR method, a large number of experimental studies are conducted on nine gene expression datasets. The obtained computational results indicate the efficiency and productivity of DR-FS-MFMR for the gene selection task