Learning Binary Decision Trees by Argmin Differentiation

We address the problem of learning binary decision trees that partition data for some downstream task. We propose to learn discrete parameters (i.e., for tree traversals and node pruning) and continuous parameters (i.e., for tree split functions and prediction functions) simultaneously using argmin differentiation. We do so by sparsely relaxing a mixed-integer program for the discrete parameters, to allow gradients to pass through the program to continuous parameters. We derive customized algorithms to efficiently compute the forward and backward passes. This means that our tree learning procedure can be used as an (implicit) layer in arbitrary deep networks, and can be optimized with arbitrary loss functions. We demonstrate that our approach produces binary trees that are competitive with existing single tree and ensemble approaches, in both supervised and unsupervised settings. Further, apart from greedy approaches (which do not have competitive accuracies), our method is faster to train than all other tree-learning baselines we compare with. The code for reproducing the results is available at https://github.com/vzantedeschi/LatentTrees

Classification of Alzheimer's disease using robust TabNet neural networks on genetic data

Alzheimer's disease (AD) is one of the most common neurodegenerative diseases and its onset is significantly associated with genetic factors. Being the capabilities of high specificity and accuracy, genetic testing has been considered as an important technique for AD diagnosis. In this paper, we presented an improved deep learning (DL) algorithm, namely differential genes screening TabNet (DGS-TabNet) for AD binary and multi-class classifications. For performance evaluation, our proposed approach was compared with three novel DLs of multi-layer perceptron (MLP), neural oblivious decision ensembles (NODE), TabNet as well as five classical machine learnings (MLs) including decision tree (DT), random forests (RF), gradient boosting decision tree (GBDT), light gradient boosting machine (LGBM) and support vector machine (SVM) on the public data set of gene expression omnibus (GEO). Moreover, the biological interpretability of global important genetic features implemented for AD classification was revealed by the Kyoto encyclopedia of genes and genomes (KEGG) and gene ontology (GO). The results demonstrated that our proposed DGS-TabNet achieved the best performance with an accuracy of 93.80% for binary classification, and with an accuracy of 88.27% for multi-class classification. Meanwhile, the gene pathway analyses demonstrated that there existed two most important global genetic features of AVIL and NDUFS4 and those obtained 22 feature genes were partially correlated with AD pathogenesis. It was concluded that the proposed DGS-TabNet could be used to detect AD-susceptible genes and the biological interpretability of susceptible genes also revealed the potential possibility of being AD biomarkers