Tensor Regression

Regression analysis is a key area of interest in the field of data analysis and machine learning which is devoted to exploring the dependencies between variables, often using vectors. The emergence of high dimensional data in technologies such as neuroimaging, computer vision, climatology and social networks, has brought challenges to traditional data representation methods. Tensors, as high dimensional extensions of vectors, are considered as natural representations of high dimensional data. In this book, the authors provide a systematic study and analysis of tensor-based regression models and their applications in recent years. It groups and illustrates the existing tensor-based regression methods and covers the basics, core ideas, and theoretical characteristics of most tensor-based regression methods. In addition, readers can learn how to use existing tensor-based regression methods to solve specific regression tasks with multiway data, what datasets can be selected, and what software packages are available to start related work as soon as possible. Tensor Regression is the first thorough overview of the fundamentals, motivations, popular algorithms, strategies for efficient implementation, related applications, available datasets, and software resources for tensor-based regression analysis. It is essential reading for all students, researchers and practitioners of working on high dimensional data.Comment: 187 pages, 32 figures, 10 table

Enhanced performance by a hybrid NIRS–EEG brain computer interface

Noninvasive Brain Computer Interfaces (BCI) have been promoted to be used for neuroprosthetics. However, reports on applications with electroencephalography (EEG) show a demand for a better accuracy and stability. Here we investigate whether near-infrared spectroscopy (NIRS) can be used to enhance the EEG approach. In our study both methods were applied simultaneously in a real-time Sensory Motor Rhythm (SMR)-based BCI paradigm, involving executed movements as well as motor imagery. We tested how the classification of NIRS data can complement ongoing real-time EEG classification. Our results show that simultaneous measurements of NIRS and EEG can significantly improve the classification accuracy of motor imagery in over 90% of considered subjects and increases performance by 5% on average (p < 0:01). However, the long time delay of the hemodynamic response may hinder an overall increase of bit-rates. Furthermore we find that EEG and NIRS complement each other in terms of information content and are thus a viable multimodal imaging technique, suitable for BCI

Deep sparse multi-task learning for feature selection in Alzheimer’s disease diagnosis

Recently, neuroimaging-based Alzheimer’s disease (AD) or mild cognitive impairment (MCI) diagnosis has attracted researchers in the field, due to the increasing prevalence of the diseases. Unfortunately, the unfavorable high-dimensional nature of neuroimaging data, but a limited small number of samples available, makes it challenging to build a robust computer-aided diagnosis system. Machine learning techniques have been considered as a useful tool in this respect and, among various methods, sparse regression has shown its validity in the literature. However, to our best knowledge, the existing sparse regression methods mostly try to select features based on the optimal regression coefficients in one step. We argue that since the training feature vectors are composed of both informative and uninformative or less informative features, the resulting optimal regression coefficients are inevidently affected by the uninformative or less informative features. To this end, we first propose a novel deep architecture to recursively discard uninformative features by performing sparse multi-task learning in a hierarchical fashion. We further hypothesize that the optimal regression coefficients reflect the relative importance of features in representing the target response variables. In this regard, we use the optimal regression co-efficients learned in one hierarchy as feature weighting factors in the following hierarchy, and formulate a weighted sparse multi-task learning method. Lastly, we also take into account the distributional characteristics of samples per class and use clustering-induced subclass label vectors as target response values in our sparse regression model. In our experiments on the ADNI cohort, we performed both binary and multi-class classification tasks in AD/MCI diagnosis and showed the superiority of the proposed method by comparing with the state-of-the-art methods

Contributions to Robust Graph Clustering: Spectral Analysis and Algorithms

This dissertation details the design of fast, and parameter free, graph clustering methods to robustly determine set cluster assignments. It provides spectral analysis as well as algorithms that adapt the obtained theoretical results to the implementation of robust graph clustering techniques. Sparsity is of importance in graph clustering and a first contribution of the thesis is the definition of a sparse graph model consistent with the graph clustering objectives. This model is based on an advantageous property, arising from a block diagonal representation, of a matrix that promotes the density of connections within clusters and sparsity between them. Spectral analysis of the sparse graph model including the eigen-decomposition of the Laplacian matrix is conducted. The analysis of the Laplacian matrix is simplified by defining a vector that carries all the relevant information that is contained in the Laplacian matrix. The obtained spectral properties of sparse graphs are adapted to sparsity-aware clustering based on two methods that formulate the determination of the sparsity level as approximations to spectral properties of the sparse graph models. A second contribution of this thesis is to analyze the effects of outliers on graph clustering and to propose algorithms that address robustness and the level of sparsity jointly. The basis for this contribution is to specify fundamental outlier types that occur in the cases of extreme sparsity and the mathematical analysis of their effects on sparse graphs to develop graph clustering algorithms that are robust against the investigated outlier effects. Based on the obtained results, two different robust and sparsity-aware affinity matrix construction methods are proposed. Motivated by the outliers’ effects on eigenvectors, a robust Fiedler vector estimation and a robust spectral clustering methods are proposed. Finally, an outlier detection algorithm that is built upon the vertex degree is proposed and applied to gait analysis. The results of this thesis demonstrate the importance of jointly addressing robustness and the level of sparsity for graph clustering algorithms. Additionally, simplified Laplacian matrix analysis provides promising results to design graph construction methods that may be computed efficiently through the optimization in a vector space instead of the usually used matrix space

Data-Driven Transducer Design and Identification for Internally-Paced Motor Brain Computer Interfaces: A Review

Brain-Computer Interfaces (BCIs) are systems that establish a direct communication pathway between the users' brain activity and external effectors. They offer the potential to improve the quality of life of motor-impaired patients. Motor BCIs aim to permit severely motor-impaired users to regain limb mobility by controlling orthoses or prostheses. In particular, motor BCI systems benefit patients if the decoded actions reflect the users' intentions with an accuracy that enables them to efficiently interact with their environment. One of the main challenges of BCI systems is to adapt the BCI's signal translation blocks to the user to reach a high decoding accuracy. This paper will review the literature of data-driven and user-specific transducer design and identification approaches and it focuses on internally-paced motor BCIs. In particular, continuous kinematic biomimetic and mental-task decoders are reviewed. Furthermore, static and dynamic decoding approaches, linear and non-linear decoding, offline and real-time identification algorithms are considered. The current progress and challenges related to the design of clinical-compatible motor BCI transducers are additionally discussed

A Comparison of Movement-Related Cortical Potentials and Their Application in Brain-Computer Interfaces for Autism Spectrum Disorder

Brain-computer interfaces have the potential to improve the lives of many populations who benefit from neurofeedback. Autism Spectrum Disorder is a condition experienced by many and its deficits are potentially improved for some using brain-computer interface technology. Various techniques have already been used to illustrate improvements in ASD across different brain signals and interactive interfaces. In particular, movement-related cortical potentials are related to executive functioning of movement and have been shown to be successful in other systems. This thesis investigates the effect of Autism Spectrum Disorder in adults on how movement-related cortical potentials are elicited in the brain compared to neurotypical populations to determine whether the motor systems that elicit such signals are abnormally functioning, and as a result whether they may be improved with neurofeedback. In addition to understanding the EEG response for people with ASD to brain-computer interfaces, it is important to gain insights into their perception of such technologies. This thesis also examines how people with ASD perceive different potential brain-computer interfaces. Quantitative and qualitative data was collected and analysed across three different interfaces (auditory, visual, and haptic) and two different tasks (real movement and imagined movement execution). The EEG results show statistically significant differences in the elicitation of movement-related cortical potentials (MRCPs) between the autistic and neurotypical group, thus indicating possible underlying abnormalities in the motor systems being activated. The features of MRCP were much smaller in amplitude in the ASD group, suggesting that fewer neurons are being recruited for movement-based actions. Since other studies have demonstrated success when improving MRCPs in populations suffering from Parkinson’s and stroke, it is thus inferred that such neurofeedback may also benefit those with Autism Spectrum Disorder. While there were no statistical differences regarding EEG-related performance for different modalities, qualitative results suggest common themes regarding people with ASD’s subjective perceptions, including the need for feedback on performance and strong preferences for different types of modalities. These results emphasize the importance of considering both quantitative and qualitative data when designing brain-computer interfaces for these populations. This research demonstrates an opportunity to use MRCP-based neurofeedback to help populations with ASD, as well as emphasizes the importance and insights of capturing qualitative data in the process

VIVA: An Online Algorithm for Piecewise Curve Estimation Using &ell;\u3csup\u3e0\u3c/sup\u3e Norm Regularization

Many processes deal with piecewise input functions, which occur naturally as a result of digital commands, user interfaces requiring a confirmation action, or discrete-time sampling. Examples include the assembly of protein polymers and hourly adjustments to the infusion rate of IV fluids during treatment of burn victims. Estimation of the input is straightforward regression when the observer has access to the timing information. More work is needed if the input can change at unknown times. Successful recovery of the change timing is largely dependent on the choice of cost function minimized during parameter estimation. Optimal estimation of a piecewise input will often proceed by minimization of a cost function which includes an estimation error term (most commonly mean square error) and the number (cardinality) of input changes (number of commands). Because the cardinality (ℓ0 norm) is not convex, the ℓ2 norm (quadratic smoothing) and ℓ1 norm (total variation minimization) are often substituted because they permit the use of convex optimization algorithms. However, these penalize the magnitude of input changes and therefore bias the piecewise estimates. Another disadvantage is that global optimization methods must be run after the end of data collection. One approach to unbiasing the piecewise parameter fits would include application of total variation minimization to recover timing, followed by piecewise parameter fitting. Another method is presented herein: a dynamic programming approach which iteratively develops populations of candidate estimates of increasing length, pruning those proven to be dominated. Because the usage of input data is entirely causal, the algorithm recovers timing and parameter values online. A functional definition of the algorithm, which is an extension of Viterbi decoding and integrates the pruning concept from branch-and-bound, is presented. Modifications are introduced to improve handling of non-uniform sampling, non-uniform confidence, and burst errors. Performance tests using synthesized data sets as well as volume data from a research system recording fluid infusions show five-fold (piecewise-constant data) and 20-fold (piecewise-linear data) reduction in error compared to total variation minimization, along with improved sparsity and reduced sensitivity to the regularization parameter. Algorithmic complexity and delay are also considered

Reconstruction de l'activité corticale à partir de données MEG à l'aide de réseaux cérébraux et de délais de transmission estimés à partir d'IRMd

White matter fibers transfer information between brain regions with delays that are observable with magnetoencephalography and electroencephalography (M/EEG) due to their millisecond temporal resolution. We can represent the brain as a graph where nodes are the cortical sources or areas and edges are the physical connections between them: either local (between adjacent vertices on the cortical mesh) or non-local (long-range white matter fibers). Long-range anatomical connections can be obtained with diffusion MRI (dMRI) tractography which yields a set of streamlines representing white matter fiber bundles. Given the streamlines’ lengths and the information conduction speed, transmission delays can be estimated for each connection. dMRI can thus give an insight into interaction delays of the macroscopicbrain network.Localizing and recovering electrical activity of the brain from M/EEG measurements is known as the M/EEG inverse problem. Generally, there are more unknowns (brain sources) than the number of sensors, so the solution is non-unique and the problem ill-posed. To obtain a unique solution, prior constraints on the characteristics of source distributions are needed. Traditional linear inverse methods deploy different constraints which can favour solutions with minimum norm, impose smoothness constraints in space and/or time along the cortical surface, etc. Yet, structural connectivity is rarely considered and transmission delays almost always neglected.The first contribution of this thesis consists of a multimodal preprocessing pipeline used to integrate structural MRI, dMRI and MEG data into a same framework, and of a simulation procedure of source-level brain activity that was used as a synthetic dataset to validate the proposed reconstruction approaches.In the second contribution, we proposed a new framework to solve the M/EEG inverse problem called Connectivity-Informed M/EEG Inverse Problem (CIMIP), where prior transmission delays supported by dMRI were included to enforce temporal smoothness between time courses of connected sources. This was done by incorporating a Laplacian operator into the regularization, that operates on a time-dependent connectivity graph. Nonetheless, some limitations of the CIMIP approach arised, mainly due to the nature of the Laplacian, which acts on the whole graph, favours smooth solutions across all connections, for all delays, and it is agnostic to directionality.In this thesis, we aimed to investigate patterns of brain activity during visuomotor tasks, during which only a few regions typically get significantly activated, as shown by previous studies. This led us to our third contribution, an extension of the CIMIP approach that addresses the aforementioned limitations, named CIMIP_OML (“Optimal Masked Laplacian”). We restricted the full source space network (the whole cortical mesh) to a network of regions of interest and tried to find how the information is transferred between its nodes. To describe the interactions between nodes in a directed graph, we used the concept of network motifs. We proposed an algorithm that (1) searches for an optimal network motif – an optimal pattern of interaction between different regions and (2) reconstructs source activity given the found motif. Promising results are shown for both simulated and real MEG data for a visuomotor task and compared with 3 different state-of-the-art reconstruction methods.To conclude, we tackled a difficult problem of exploiting delays supported by dMRI for the reconstruction of brain activity, while also considering the directionality in the information transfer, and provided new insights into the complex patterns of brain activity.Les fibres de la matière blanche permettent le transfert d’information dans le cerveau avec des délais observables en Magnétoencéphalographie et Électroencéphalographie (M/EEG) grâce à leur haute résolution temporelle. Le cerveau peut être représenté comme un graphe où les nœuds sont les régions corticales et les liens sont les connexions physiques entre celles-ci: soit locales (entre sommets adjacents sur le maillage cortical), soit non locales (fibres de la matière blanche). Les connexions non-locales peuvent être reconstruites avec la tractographie de l’IRM de diffusion (IRMd) qui génère un ensemble de courbes («streamlines») représentant des fibres de la matière blanche. Sachant les longueurs des fibres et la vitesse de conduction de l’information, les délais de transmission peuvent être estimés. L’IRMd peut donc donner un aperçu des délais d’interaction du réseau cérébral macroscopique.La localisation et la reconstruction de l’activité électrique cérébrale à partir des mesures M/EEG est un problème inverse. En général, il y a plus d’inconnues (sources cérébrales) que de capteurs. La solution n’est donc pas unique et le problème est dit mal posé. Pour obtenir une solution unique, des hypothèses sur les caractéristiques des distributions de sources sont requises. Les méthodes inverses linéaires traditionnelles utilisent différentes hypothèses qui peuvent favoriser des solutions de norme minimale, imposer des contraintes de lissage dans l’espace et/ou dans le temps, etc. Pourtant, la connectivité structurelle est rarement prise en compte et les délais de transmission sont presque toujours négligés.La première contribution de cette thèse est un pipeline de prétraitement multimodal utilisé pour l’intégration des données d’IRM, IRMd et MEG dans un même cadre, et d’une méthode de simulation de l’activité corticale qui a été utilisée comme jeu de données synthétiques pour valider les approches de reconstruction proposées. Nous proposons également une nouvelle approche pour résoudre le problème inverse M/EEG appelée «Problème Inverse M/EEG Informé par la Connectivité» (CIMIP pour Connectivity-Informed M/EEG Inverse Problem), où des délais de transmission provenant de l’IRMd sont inclus pour renforcer le lissage temporel entre les décours des sources connectées. Pour cela, un opérateur Laplacien, basé sur un graphe de connectivité en fonction du temps, a été intégré dans la régularisation. Cependant, certaines limites de l’approche CIMIP sont apparues en raison de la nature du Laplacien qui agit sur le graphe entier et favorise les solutions lisses sur toutes les connexions, pour tous les délais, et indépendamment de la directionnalité.Lors de tâches visuo-motrices, seules quelques régions sont généralement activées significativement. Notre troisième contribution est une extension de CIMIP pour ce type de tâches qui répond aux limitations susmentionnées, nommée CIMIP_OML («Optimal Masked Laplacian») ou Laplacien Masqué Optimal. Nous essayons de trouver comment l’information est transférée entre les nœuds d’un sous-réseau de régions d’intérêt du réseau complet de l’espace des sources. Pour décrire les interactions entre nœuds dans un graphe orienté, nous utilisons le concept de motifs de réseau. Nous proposons un algorithme qui 1) cherche un motif de réseau optimal- un modèle optimal d’interaction entre régions et 2) reconstruit l’activité corticale avec le motif trouvé. Des résultats prometteurs sont présentés pour des données MEG simulées et réelles (tâche visuo-motrice) et comparés avec 3 méthodes de l’état de l’art. Pour conclure, nous avons abordé un problème difficile d’exploitation des délais de l’IRMd lors l’estimation de l’activité corticale en tenant compte de la directionalité du transfert d’information, fournissant ainsi de nouvelles perspectives sur les patterns complexes de l’activité cérébrale