Discriminant feature extraction by generalized difference subspace

This paper reveals the discriminant ability of the orthogonal projection of data onto a generalized difference subspace (GDS) both theoretically and experimentally. In our previous work, we have demonstrated that GDS projection works as the quasi-orthogonalization of class subspaces. Interestingly, GDS projection also works as a discriminant feature extraction through a similar mechanism to the Fisher discriminant analysis (FDA). A direct proof of the connection between GDS projection and FDA is difficult due to the significant difference in their formulations. To avoid the difficulty, we first introduce geometrical Fisher discriminant analysis (gFDA) based on a simplified Fisher criterion. gFDA can work stably even under few samples, bypassing the small sample size (SSS) problem of FDA. Next, we prove that gFDA is equivalent to GDS projection with a small correction term. This equivalence ensures GDS projection to inherit the discriminant ability from FDA via gFDA. Furthermore, we discuss two useful extensions of these methods, 1) nonlinear extension by kernel trick, 2) the combination of convolutional neural network (CNN) features. The equivalence and the effectiveness of the extensions have been verified through extensive experiments on the extended Yale B+, CMU face database, ALOI, ETH80, MNIST and CIFAR10, focusing on the SSS problem

Sparse multinomial kernel discriminant analysis (sMKDA)

Dimensionality reduction via canonical variate analysis (CVA) is important for pattern recognition and has been extended variously to permit more flexibility, e.g. by "kernelizing" the formulation. This can lead to over-fitting, usually ameliorated by regularization. Here, a method for sparse, multinomial kernel discriminant analysis (sMKDA) is proposed, using a sparse basis to control complexity. It is based on the connection between CVA and least-squares, and uses forward selection via orthogonal least-squares to approximate a basis, generalizing a similar approach for binomial problems. Classification can be performed directly via minimum Mahalanobis distance in the canonical variates. sMKDA achieves state-of-the-art performance in terms of accuracy and sparseness on 11 benchmark datasets

Fatty Acid Metabolites Combine with Reduced β Oxidation to Activate Th17 Inflammation in Human Type 2 Diabetes

Mechanisms that regulate metabolites and downstream energy generation are key determinants of T cell cytokine production, but the processes underlying the Th17 profile that predicts the metabolic status of people with obesity are untested. Th17 function requires fatty acid uptake, and our new data show that blockade of CPT1A inhibits Th17-associated cytokine production by cells from people with type 2 diabetes (T2D). A low CACT:CPT1A ratio in immune cells from T2D subjects indicates altered mitochondrial function and coincides with the preference of these cells to generate ATP through glycolysis rather than fatty acid oxidation. However, glycolysis was not critical for Th17 cytokines. Instead, β oxidation blockade or CACT knockdown in T cells from lean subjects to mimic characteristics of T2D causes cells to utilize 16C-fatty acylcarnitine to support Th17 cytokines. These data show long-chain acylcarnitine combines with compromised β oxidation to promote disease-predictive inflammation in human T2D. Although glycolysis generally fuels inflammation, Nicholas, Proctor, and Agrawal et al. report that PBMCs from subjects with type 2 diabetes use a different mechanism to support chronic inflammation largely independent of fuel utilization. Loss- and gain-of-function experiments in cells from healthy subjects show mitochondrial alterations combine with increases in fatty acid metabolites to drive chronic T2D-like inflammation

Circulating Mitochondrial-Derived Vesicles, Inflammatory Biomarkers and Amino Acids in Older Adults With Physical Frailty and Sarcopenia: A Preliminary BIOSPHERE Multi-Marker Study Using Sequential and Orthogonalized Covariance Selection \u2013 Linear Discriminant Analysis

Physical frailty and sarcopenia (PF&S) is a prototypical geriatric condition characterized by reduced physical function and low muscle mass. The multifaceted pathophysiology of this condition recapitulates all hallmarks of aging making the identification of specific biomarkers challenging. In the present study, we explored the relationship among three processes that are thought to be involved in PF&S (i.e., systemic inflammation, amino acid dysmetabolism, and mitochondrial dysfunction). We took advantage of the well-characterized cohort of older adults recruited in the \u201cBIOmarkers associated with Sarcopenia and Physical frailty in EldeRly pErsons\u201d (BIOSPHERE) study to preliminarily combine in a multi-platform analytical approach inflammatory biomolecules, amino acids and derivatives, and mitochondrial-derived vesicle (MDV) cargo molecules to evaluate their performance as possible biomarkers for PF&S. Eleven older adults aged 70 years and older with PF&S and 10 non-sarcopenic non-frail controls were included in the analysis based on the availability of the three categories of biomolecules. A sequential and orthogonalized covariance selection\u2014linear discriminant analysis (SO-CovSel\u2013LDA) approach was used for biomarkers selection. Of the 75 analytes assayed, 16 had concentrations below the detection limit. Within the remaining 59 biomolecules, So-CovSel\u2013LDA selected a set comprising two amino acids (phosphoethanolamine and tryptophan), two cytokines (interleukin 1 receptor antagonist and macrophage inflammatory protein 1\u3b2), and MDV-derived nicotinamide adenine dinucleotide reduced form:ubiquinone oxidoreductase subunit S3 as the best predictors for discriminating older people with and without PF&S. The evaluation of these biomarkers in larger cohorts and their changes over time or in response to interventions may unveil specific pathogenetic pathways of PF&S and identify new biological targets for drug development

Discriminant analysis based feature extraction for pattern recognition

Fisher's linear discriminant analysis (FLDA) has been widely used in pattern recognition applications. However, this method cannot be applied for solving the pattern recognition problems if the within-class scatter matrix is singular, a condition that occurs when the number of the samples is small relative to the dimension of the samples. This problem is commonly known as the small sample size (SSS) problem and many of the FLDA variants proposed in the past to deal with this problem suffer from excessive computational load because of the high dimensionality of patterns or lose some useful discriminant information. This study is concerned with developing efficient techniques for discriminant analysis of patterns while at the same time overcoming the small sample size problem. With this objective in mind, the work of this research is divided into two parts. In part 1, a technique by solving the problem of generalized singular value decomposition (GSVD) through eigen-decomposition is developed for linear discriminant analysis (LDA). The resulting algorithm referred to as modified GSVD-LDA (MGSVD-LDA) algorithm is thus devoid of the singularity problem of the scatter matrices of the traditional LDA methods. A theorem enunciating certain properties of the discriminant subspace derived by the proposed GSVD-based algorithms is established. It is shown that if the samples of a dataset are linearly independent, then the samples belonging to different classes are linearly separable in the derived discriminant subspace; and thus, the proposed MGSVD-LDA algorithm effectively captures the class structure of datasets with linearly independent samples. Inspired by the results of this theorem that essentially establishes a class separability of linearly independent samples in a specific discriminant subspace, in part 2, a new systematic framework for the pattern recognition of linearly independent samples is developed. Within this framework, a discriminant model, in which the samples of the individual classes of the dataset lie on parallel hyperplanes and project to single distinct points of a discriminant subspace of the underlying input space, is shown to exist. Based on this model, a number of algorithms that are devoid of the SSS problem are developed to obtain this discriminant subspace for datasets with linearly independent samples. For the discriminant analysis of datasets for which the samples are not linearly independent, some of the linear algorithms developed in this thesis are also kernelized. Extensive experiments are conducted throughout this investigation in order to demonstrate the validity and effectiveness of the ideas developed in this study. It is shown through simulation results that the linear and nonlinear algorithms for discriminant analysis developed in this thesis provide superior performance in terms of the recognition accuracy and computational complexit

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.Comment: 232 page

Representation of foreseeable choice outcomes in orbitofrontal cortex triplet-wise interactions.

Shared neuronal variability has been shown to modulate cognitive processing. However, the relationship between shared variability and behavioral performance is heterogeneous and complex in frontal areas such as the orbitofrontal cortex (OFC). Mounting evidence shows that single-units in OFC encode a detailed cognitive map of task-space events, but the existence of a robust neuronal ensemble coding for the predictability of choice outcome is less established. Here, we hypothesize that the coding of foreseeable outcomes is potentially unclear from the analysis of units activity and their pairwise correlations. However, this code might be established more conclusively when higher-order neuronal interactions are mapped to the choice outcome. As a case study, we investigated the trial-to-trial shared variability of neuronal ensemble activity during a two-choice interval-discrimination task in rodent OFC, specifically designed such that a lose-switch strategy is optimal by repeating the rewarded stimulus in the upcoming trial. Results show that correlations among triplets are higher during correct choices with respect to incorrect ones, and that this is sustained during the entire trial. This effect is not observed for pairwise nor for higher than third-order correlations. This scenario is compatible with constellations of up to three interacting units assembled during trials in which the task is performed correctly. More interestingly, a state-space spanned by such constellations shows that only correct outcome states that can be successfully predicted are robust over 100 trials of the task, and thus they can be accurately decoded. However, both incorrect and unpredictable outcome representations were unstable and thus non-decodeable, due to spurious negative correlations. Our results suggest that predictability of successful outcomes, and hence the optimal behavioral strategy, can be mapped out in OFC ensemble states reliable over trials of the task, and revealed by sufficiency complex neuronal interactions

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.Comment: 232 page