Feature map size selection for fMRI classification on end-to-end deep convolutional neural networks

The emergence of convolutional neural networks (CNN) in various fields has also paved numerous ways for advancement in the field of medical imaging. This paper focuses on functional magnetic resonance imaging (fMRI) in the field of neuroimaging. It has high temporal resolution and robust to control or non-control subjects. CNN analysis on structural magnetic resonance imaging (MRI) and fMRI datasets is compared to rule out one of the grey areas in building CNNs for medical imaging analysis. This study focuses on the feature map size selection on fMRI datasets with CNNs where the selected sizes are evaluated for their performances. Although few outstanding studies on fMRI have been published, the availability of diverse previous studies on MRI previous works impulses us to study to learn the pattern of feature map sizes for CNN configuration. Six configurations are analyzed with prominent public fMRI dataset, names as Human Connectome Project (HCP). This dataset is widely used for any type of fMRI classification. With three set of data divisions, the accuracy values for validation set of fMRI classification are assessed and discussed. Despite the fact that only one slice of every 118 subjects' temporal brain images is used in the study, the validation of classification for three training-excluded subjects known as validation set, has proven the need for feature map size selection. This paper emphasizes the indispensable step of selecting the feature map sizes when designing CNN for fMRI classification. In addition, we provide proofs that validation set should consist of distinct subjects for definite evaluation of any model performance

A data-driven robotic Chinese calligraphy system using convolutional auto-encoder and differential evolution

The Chinese stroke evaluation and generation systems required in an autonomous calligraphy robot play a crucial role in producing high-quality writing results with good diversity. These systems often suffer from inefficiency and non-optima despite of intensive research effort investment by the robotic community. This paper proposes a new learning system to allow a robot to automatically learn to write Chinese calligraphy effectively. In the proposed system, the writing quality evaluation subsystem assesses written strokes using a convolutional auto-encoder network (CAE), which enables the generation of aesthetic strokes with various writing styles. The trained CAE network effectively excludes poorly written strokes through stroke reconstruction, but guarantees the inheritance of information from well-written ones. With the support of the evaluation subsystem, the writing trajectory model generation subsystem is realized by multivariate normal distributions optimized by differential evolution (DE), a type of heuristic optimization search algorithm. The proposed approach was validated and evaluated using a dataset of nine stroke categories; high-quality written strokes have been resulted with good diversity which shows the robustness and efficacy of the proposed approach and its potential in autonomous action-state space exploration for other real-world applications

Weighted ensemble-model and network analysis: a method to predict fluid intelligence via naturalistic functional connectivity

Objectives: Functional connectivity triggered by naturalistic stimulus (e.g., movies) and machine learning techniques provide a great insight in exploring the brain functions such as fluid intelligence. However, functional connectivity are considered to be multi-layered, while traditional machine learning based on individual models not only are limited in performance, but also fail to extract multi-dimensional and multi-layered information from brain network. Methods: In this study, inspired by multi-layer brain network structure, we propose a new method namely Weighted Ensemble-model and Network Analysis, which combines the machine learning and graph theory for improved fluid intelligence prediction. Firstly, functional connectivity analysis and graphical theory were jointly employed. The network and graphical indices computed using the preprocessed fMRI data were then fed into auto-encoder parallelly for feature extraction to predict the fluid intelligence. In order to improve the performance, different models were automatically stacked and fused with weighted values. Finally, layers of auto-encoder were visualized to better illustrate the impacts, followed by the evaluation of the performance to justify the mechanism of brain functions. Results: Our proposed methods achieved best performance with 3.85 mean absolute deviation, 0.66 correlation coefficient and 0.42 R-squared coefficient, outperformed other state-of-the-art methods. It is also worth noting that, the optimization of the biological pattern extraction was automated though the auto-encoder algorithm. Conclusion: The proposed method not only outperforming the state-of-the-art reports, but also able to effectively capturing the common and biological pattern from functional connectivity during naturalistic movies state for potential clinical explorations

Self-supervised pretraining improves the performance of classification of task functional magnetic resonance imaging

IntroductionDecoding brain activities is one of the most popular topics in neuroscience in recent years. And deep learning has shown high performance in fMRI data classification and regression, but its requirement for large amounts of data conflicts with the high cost of acquiring fMRI data.MethodsIn this study, we propose an end-to-end temporal contrastive self-supervised learning algorithm, which learns internal spatiotemporal patterns within fMRI and allows the model to transfer learning to datasets of small size. For a given fMRI signal, we segmented it into three sections: the beginning, middle, and end. We then utilized contrastive learning by taking the end-middle (i.e., neighboring) pair as the positive pair, and the beginning-end (i.e., distant) pair as the negative pair.ResultsWe pretrained the model on 5 out of 7 tasks from the Human Connectome Project (HCP) and applied it in a downstream classification of the remaining two tasks. The pretrained model converged on data from 12 subjects, while a randomly initialized model required 100 subjects. We then transferred the pretrained model to a dataset containing unpreprocessed whole-brain fMRI from 30 participants, achieving an accuracy of 80.2 ± 4.7%, while the randomly initialized model failed to converge. We further validated the model’s performance on the Multiple Domain Task Dataset (MDTB), which contains fMRI data of 26 tasks from 24 participants. Thirteen tasks of fMRI were selected as inputs, and the results showed that the pre-trained model succeeded in classifying 11 of the 13 tasks. When using the 7 brain networks as input, variations of the performance were observed, with the visual network performed as well as whole brain inputs, while the limbic network almost failed in all 13 tasks.DiscussionOur results demonstrated the potential of self-supervised learning for fMRI analysis with small datasets and unpreprocessed data, and for analysis of the correlation between regional fMRI activity and cognitive tasks

BolT: Fused Window Transformers for fMRI Time Series Analysis

Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time scales. Here, we present BolT, a blood-oxygen-level-dependent transformer model, for analyzing multi-variate fMRI time series. BolT leverages a cascade of transformer encoders equipped with a novel fused window attention mechanism. Encoding is performed on temporally-overlapped windows within the time series to capture local representations. To integrate information temporally, cross-window attention is computed between base tokens in each window and fringe tokens from neighboring windows. To gradually transition from local to global representations, the extent of window overlap and thereby number of fringe tokens are progressively increased across the cascade. Finally, a novel cross-window regularization is employed to align high-level classification features across the time series. Comprehensive experiments on large-scale public datasets demonstrate the superior performance of BolT against state-of-the-art methods. Furthermore, explanatory analyses to identify landmark time points and regions that contribute most significantly to model decisions corroborate prominent neuroscientific findings in the literature