Hyperalignment of motor cortical areas based on motor imagery during action observation

Multivariate Pattern Analysis (MVPA) has grown in importance due to its capacity to use both coarse and fine scale patterns of brain activity. However, a major limitation of multivariate analysis is the difficulty of aligning features across brains, which makes MVPA a subject specific analysis. Recent work by Haxby et al. (2011) introduced a method called Hyperalignment that explored neural activity in ventral temporal cortex during object recognition and demonstrated the ability to align individual patterns of brain activity into a common high dimensional space to facilitate Between Subject Classification (BSC). Here we examined BSC based on Hyperalignment of motor cortex during a task of motor imagery of three natural actions (lift, knock and throw). To achieve this we collected brain activity during the combined tasks of action observation and motor imagery to a parametric action space containing 25 stick-figure blends of the three natural actions. From these responses we derived Hyperalignment transformation parameters that were used to map subjects’ representational spaces of the motor imagery task in the motor cortex into a common model representational space. Results showed that BSC of the neural response patterns based on Hyperalignment exceeded both BSC based on anatomical alignment as well as a standard Within Subject Classification (WSC) approach. We also found that results were sensitive to the order in which participants entered the Hyperalignment algorithm. These results demonstrate the effectiveness of Hyperalignment to align neural responses across subject in motor cortex to enable BSC of motor imagery

Decoding the consumer’s brain: Neural representations of consumer experience

Understanding consumer experience – what consumers think about brands, how they feel about services, whether they like certain products – is crucial to marketing practitioners. ‘Neuromarketing’, as the application of neuroscience in marketing research is called, has generated excitement with the promise of understanding consumers’ minds by probing their brains directly. Recent advances in neuroimaging analysis leverage machine learning and pattern classification techniques to uncover patterns from neuroimaging data that can be associated with thoughts and feelings. In this dissertation, I measure brain responses of consumers by functional magnetic resonance imaging (fMRI) in order to ‘decode’ their mind. In three different studies, I have demonstrated how different aspects of consumer experience can be studied with fMRI recordings. First, I study how consumers think about brand image by comparing their brain responses during passive viewing of visual templates (photos depicting various social scenarios) to those during active visualizing of a brand’s image. Second, I use brain responses during viewing of affective pictures to decode emotional responses during watching of movie-trailers. Lastly, I examine whether marketing videos that evoke s

Modality-specific brain representations during automatic processing of face, voice and body expressions

A central question in affective science and one that is relevant for its clinical applications is how emotions provided by different stimuli are experienced and represented in the brain. Following the traditional view emotional signals are recognized with the help of emotion concepts that are typically used in descriptions of mental states and emotional experiences, irrespective of the sensory modality. This perspective motivated the search for abstract representations of emotions in the brain, shared across variations in stimulus type (face, body, voice) and sensory origin (visual, auditory). On the other hand, emotion signals like for example an aggressive gesture, trigger rapid automatic behavioral responses and this may take place before or independently of full abstract representation of the emotion. This pleads in favor specific emotion signals that may trigger rapid adaptative behavior only by mobilizing modality and stimulus specific brain representations without relying on higher order abstract emotion categories. To test this hypothesis, we presented participants with naturalistic dynamic emotion expressions of the face, the whole body, or the voice in a functional magnetic resonance (fMRI) study. To focus on automatic emotion processing and sidestep explicit concept-based emotion recognition, participants performed an unrelated target detection task presented in a different sensory modality than the stimulus. By using multivariate analyses to assess neural activity patterns in response to the different stimulus types, we reveal a stimulus category and modality specific brain organization of affective signals. Our findings are consistent with the notion that under ecological conditions emotion expressions of the face, body and voice may have different functional roles in triggering rapid adaptive behavior, even if when viewed from an abstract conceptual vantage point, they may all exemplify the same emotion. This has implications for a neuroethologically grounded emotion research program that should start from detailed behavioral observations of how face, body, and voice expressions function in naturalistic contexts

3D Convolution Neural Networks for Medical Imaging; Classification and Segmentation : A Doctor’s Third Eye

Master's thesis in Information- and communication technology (IKT591)In this thesis, we studied and developed 3D classification and segmentation models for medical imaging. The classification is done for Alzheimer’s Disease and segmentation is for brain tumor sub-regions. For the medical imaging classification task we worked towards developing a novel deep architecture which can accomplish the complex task of classifying Alzheimer’s Disease volumetrically from the MRI scans without the need of any transfer learning. The experiments were performed for both binary classification of Alzheimer’s Disease (AD) from Normal Cognitive (NC), as well as multi class classification between the three stages of Alzheimer’s called NC, AD and Mild cognitive impairment (MCI). We tested our model on the ADNI dataset and achieved mean accuracy of 94.17% and 89.14% for binary classification and multiclass classification respectively. In the second part of this thesis which is segmentation of tumors sub-regions in brain MRI images we studied some popular architecture for segmentation of medical imaging and inspired from them, proposed our architecture of end-to-end trainable fully convolutional neural net-work which uses attention block to learn the localization of different features of the multiple sub-regions of tumor. Also experiments were done to see the effect of weighted cross-entropy loss function and dice loss function on the performance of the model and the quality of the output segmented labels. The results of evaluation of our model are received through BraTS’19 dataset challenge. The model is able to achieve a dice score of 0.80 for the segmentation of whole tumor, and a dice scores of 0.639 and 0.536 for other two sub-regions within the tumor on validation data. In this thesis we successfully applied computer vision techniques for medical imaging analysis. We show the huge potential and numerous benefits of deep learning to combat and detect diseases opens up more avenues for research and application for automating medical imaging analysis

The neurobiology of cortical music representations

Music is undeniable one of humanity’s defining traits, as it has been documented since the earliest days of mankind, is present in all knowcultures and perceivable by all humans nearly alike. Intrigued by its omnipresence, researchers of all disciplines started the investigation of music’s mystical relationship and tremendous significance to humankind already several hundred years ago. Since comparably recently, the immense advancement of neuroscientific methods also enabled the examination of cognitive processes related to the processing of music. Within this neuroscience ofmusic, the vast majority of research work focused on how music, as an auditory stimulus, reaches the brain and howit is initially processed, aswell as on the tremendous effects it has on and can evoke through the human brain. However, intermediate steps, that is how the human brain achieves a transformation of incoming signals to a seemingly specialized and abstract representation of music have received less attention. Aiming to address this gap, the here presented thesis targeted these transformations, their possibly underlying processes and how both could potentially be explained through computational models. To this end, four projects were conducted. The first two comprised the creation and implementation of two open source toolboxes to first, tackle problems inherent to auditory neuroscience, thus also affecting neuroscientific music research and second, provide the basis for further advancements through standardization and automation. More precisely, this entailed deteriorated hearing thresholds and abilities in MRI settings and the aggravated localization and parcellation of the human auditory cortex as the core structure involved in auditory processing. The third project focused on the human’s brain apparent tuning to music by investigating functional and organizational principles of the auditory cortex and network with regard to the processing of different auditory categories of comparable social importance, more precisely if the perception of music evokes a is distinct and specialized pattern. In order to provide an in depth characterization of the respective patterns, both the segregation and integration of auditory cortex regions was examined. In the fourth and final project, a highly multimodal approach that included fMRI, EEG, behavior and models of varying complexity was utilized to evaluate how the aforementioned music representations are generated along the cortical hierarchy of auditory processing and how they are influenced by bottom-up and top-down processes. The results of project 1 and 2 demonstrated the necessity for the further advancement of MRI settings and definition of working models of the auditory cortex, as hearing thresholds and abilities seem to vary as a function of the used data acquisition protocol and the localization and parcellation of the human auditory cortex diverges drastically based on the approach it is based one. Project 3 revealed that the human brain apparently is indeed tuned for music by means of a specialized representation, as it evoked a bilateral network with a right hemispheric weight that was not observed for the other included categories. The result of this specialized and hierarchical recruitment of anterior and posterior auditory cortex regions was an abstract music component ix x SUMMARY that is situated in anterior regions of the superior temporal gyrus and preferably encodes music, regardless of sung or instrumental. The outcomes of project 4 indicated that even though the entire auditory cortex, again with a right hemispheric weight, is involved in the complex processing of music in particular, anterior regions yielded an abstract representation that varied excessively over time and could not sufficiently explained by any of the tested models. The specialized and abstract properties of this representation was furthermore underlined by the predictive ability of the tested models, as models that were either based on high level features such as behavioral representations and concepts or complex acoustic features always outperformed models based on single or simpler acoustic features. Additionally, factors know to influence auditory and thus music processing, like musical training apparently did not alter the observed representations. Together, the results of the projects suggest that the specialized and stable cortical representation of music is the outcome of sophisticated transformations of incoming sound signals along the cortical hierarchy of auditory processing that generate a music component in anterior regions of the superior temporal gyrus by means of top-down processes that interact with acoustic features, guiding their processing.Musik ist unbestreitbarer Weise eine der definierenden Eigenschaften des Menschen. Dokumentiert seit den frühesten Tagen der Menschheit und in allen bekannten Kulturen vorhanden, ist sie von allenMenschen nahezu gleichwahrnehmbar. Fasziniert von ihrerOmnipräsenz haben Wissenschaftler aller Disziplinen vor einigen hundert Jahren begonnen die mystische Beziehung zwischen Musik und Mensch, sowie ihre enorme Bedeutung für selbigen zu untersuchen. Seit einem vergleichsweise kurzem Zeitraum ist es durch den immensen Fortschritt neurowissenschafticher Methoden auch möglich die kognitiven Prozesse, welche an der Verarbeitung von Musik beteiligt, sind zu untersuchen. Innerhalb dieser Neurowissenschaft der Musik hat sich ein Großteil der Forschungsarbeit darauf konzentriert wie Musik, als auditorischer Stimulus, das menschliche Gehirn erreicht und wie sie initial verarbeitet wird, als auch welche kolossallen Effekte sie auf selbiges hat und auch dadurch bewirken kann. Jedoch haben die Zwischenschritte, also wie das menschliche Gehirn eintreffende Signale in eine scheinbar spezialisierte und abstrakte Repräsentation vonMusik umwandelt, vergleichsweise wenig Aufmerksamkeit erhalten. Um die dadurch entstandene Lücke zu adressieren, hat die hier vorliegende Dissertation diese Prozesse und wie selbige durch Modelle erklärt werden können in vier Projekten untersucht. Die ersten beiden Projekte beinhalteten die Herstellung und Implementierung von zwei Toolboxen um erstens, inhärente Probleme der auditorischen Neurowissenschaft, daher auch neurowissenschaftlicher Untersuchungen von Musik, zu verbessern und zweitens, eine Basis für weitere Fortschritte durch Standardisierung und Automatisierung zu schaffen. Im genaueren umfasste dies die stark beeinträchtigten Hörschwellen und –fähigkeiten in MRT-Untersuchungen und die erschwerte Lokalisation und Parzellierung des menschlichen auditorischen Kortex als Kernstruktur auditiver Verarbeitung. Das dritte Projekt befasste sich mit der augenscheinlichen Spezialisierung von Musik im menschlichen Gehirn durch die Untersuchung funktionaler und organisatorischer Prinzipien des auditorischen Kortex und Netzwerks bezüglich der Verarbeitung verschiedener auditorischer Kategorien vergleichbarer sozialer Bedeutung, im genaueren ob die Wahrnehmung von Musik ein distinktes und spezialisiertes neuronalenMuster hervorruft. Umeine ausführliche Charakterisierung der entsprechenden neuronalen Muster zu ermöglichen wurde die Segregation und Integration der Regionen des auditorischen Kortex untersucht. Im vierten und letzten Projekt wurde ein hochmultimodaler Ansatz,welcher fMRT, EEG, Verhalten undModelle verschiedener Komplexität beinhaltete, genutzt, umzu evaluieren, wie die zuvor genannten Repräsentationen von Musik entlang der kortikalen Hierarchie der auditorischen Verarbeitung generiert und wie sie möglicherweise durch Bottom-up- und Top-down-Ansätze beeinflusst werden. Die Ergebnisse von Projekt 1 und 2 demonstrierten die Notwendigkeit für weitere Verbesserungen von MRTUntersuchungen und die Definition eines Funktionsmodells des auditorischen Kortex, daHörxi xii ZUSAMMENFASSUNG schwellen und –fähigkeiten stark in Abhängigkeit der verwendeten Datenerwerbsprotokolle variierten und die Lokalisation, sowie Parzellierung des menschlichen auditorischen Kortex basierend auf den zugrundeliegenden Ansätzen drastisch divergiert. Projekt 3 zeigte, dass das menschliche Gehirn tatsächlich eine spezialisierte Repräsentation vonMusik enthält, da selbige als einzige auditorische Kategorie ein bilaterales Netzwerk mit rechtshemisphärischer Gewichtung evozierte. Aus diesemNetzwerk, welches die Rekrutierung anteriorer und posteriorer Teile des auditorischen Kortex beinhaltete, resultierte eine scheinbar abstrakte Repräsentation von Musik in anterioren Regionen des Gyrus temporalis superior, welche präferiert Musik enkodiert, ungeachtet ob gesungen oder instrumental. Die Resultate von Projekt 4 deuten darauf hin, dass der gesamte auditorische Kortex, erneut mit rechtshemisphärischer Gewichtung, an der komplexen Verarbeitung vonMusik beteiligt ist, besonders aber anteriore Regionen, die bereits genannten abstrakte Repräsentation hervorrufen, welche sich exzessiv über die Zeitdauer derWahrnehmung verändert und nicht hinreichend durch eines der getestetenModelle erklärt werden kann. Die spezialisierten und abstrakten Eigenschaften dieser Repräsentationen wurden weiterhin durch die prädiktiven Fähigkeiten der getestetenModelle unterstrichen, daModelle, welche entweder auf höheren Eigenschaften wie Verhaltensrepräsentationen und mentalen Konzepten oder komplexen akustischen Eigenschaften basierten, stets Modelle, welche auf niederen Attributen wie simplen akustischen Eigenschaften basierten, übertrafen. Zusätzlich konnte kein Effekt von Faktoren, wie z.B. musikalisches Training, welche bekanntermaßen auditorische und daherMusikverarbeitung beeinflussen, nachgewiesen werden. Zusammengefasst deuten die Ergebnisse der Projekte darauf, hin dass die spezialisierte und stabile kortikale Repräsentation vonMusik ein Resultat komplexer Prozesse ist, welche eintreffende Signale entlang der kortikalen Hierarchie auditorischer Verarbeitung in eine abstrakte Repräsentation vonMusik innerhalb anteriorer Regionen des Gyrus temporalis superior durch Top-Down-Prozesse, welche mit akustischen Eigenschaften interagieren und deren Verarbeitung steuern, umwandeln

Algorithms for Neural Prosthetic Applications

abstract: In the last 15 years, there has been a significant increase in the number of motor neural prostheses used for restoring limb function lost due to neurological disorders or accidents. The aim of this technology is to enable patients to control a motor prosthesis using their residual neural pathways (central or peripheral). Recent studies in non-human primates and humans have shown the possibility of controlling a prosthesis for accomplishing varied tasks such as self-feeding, typing, reaching, grasping, and performing fine dexterous movements. A neural decoding system comprises mainly of three components: (i) sensors to record neural signals, (ii) an algorithm to map neural recordings to upper limb kinematics and (iii) a prosthetic arm actuated by control signals generated by the algorithm. Machine learning algorithms that map input neural activity to the output kinematics (like finger trajectory) form the core of the neural decoding system. The choice of the algorithm is thus, mainly imposed by the neural signal of interest and the output parameter being decoded. The various parts of a neural decoding system are neural data, feature extraction, feature selection, and machine learning algorithm. There have been significant advances in the field of neural prosthetic applications. But there are challenges for translating a neural prosthesis from a laboratory setting to a clinical environment. To achieve a fully functional prosthetic device with maximum user compliance and acceptance, these factors need to be addressed and taken into consideration. Three challenges in developing robust neural decoding systems were addressed by exploring neural variability in the peripheral nervous system for dexterous finger movements, feature selection methods based on clinically relevant metrics and a novel method for decoding dexterous finger movements based on ensemble methods.Dissertation/ThesisDoctoral Dissertation Bioengineering 201

Application of fMRI for action representation: decoding, aligning and modulating

Functional magnetic resonance imaging (fMRI) is an important tool for understanding neural mechanisms underlying human brain function. Understanding how the human brain responds to stimuli and how different cortical regions represent the information, and if these representational spaces are shared across brains and critical for our understanding of how the brain works. Recently, multivariate pattern analysis (MVPA) has a growing importance to predict mental states from fMRI data and to detect the coarse and fine scale neural responses. However, a major limitation of MVPA is the difficulty of aligning features across brains due to high variability in subjects’ responses and hence MVPA has been generally used as a subject specific analysis. Hyperalignment, solved this problem of feature alignment across brains by mapping neural responses into a common model to facilitate between subject classifications. Another technique of growing importance in understanding brain function is real-time fMRI Neurofeedback, which can be used to enable individuals to alter their brain activity. It facilitates people’s ability to learn control of their cognitive processes like motor control and pain by learning to modulate their brain activation in targeted regions. The aim of this PhD research is to decode and to align the motor representations of multi-joint arm actions based on different modalities of motor simulation, for instance Motor Imagery (MI) and Action Observation (AO) using functional Magnetic Resonance Imaging (fMRI) and to explore the feasibility of using a real-time fMRI neurofeedback to alter these action representations. The first experimental study of this thesis was performed on able-bodied participants to align the neural representation of multi-joint arm actions (lift, knock and throw) during MI tasks in the motor cortex using hyperalignment. Results showed that hyperalignment affords a statistically higher between-subject classification (BSC) performance compared to anatomical alignment. Also, hyperalignment is sensitive to the order in which subjects entered the hyperalignment algorithm to create the common model space. These results demonstrate the effectiveness of hyperalignment to align neural responses in motor cortex across subjects to enable BSC of motor imagery. The second study extended the use of hyperalignment to align fronto-parietal motor regions by addressing the problems of localization and cortical parcellation using cortex based alignment. Also, representational similarity analysis (RSA) was applied to investigate the shared neural code between AO+MI and MI of different actions. Results of MVPA revealed that these actions as well as their modalities can be decoded using the subject’s native or the hyperaligned neural responses. Furthermore, the RSA showed that AO+MI and MI representations formed separate clusters but that the representational organization of action types within these clusters was identical. These findings suggest that the neural representations of AO+MI and MI are neither the same nor totally distinct but exhibit a similar structural geometry with respect to different types of action. Results also showed that MI dominates in the AO+MI condition. The third study was performed on phantom limb pain (PLP) patients to explore the feasibility of using real-time fMRI neurofeedback to down-regulate the activity of premotor (PM) and anterior cingulate (ACC) cortices and whether the successful modulation will reduce the pain intensity. Results demonstrated that PLP patients were able to gain control and decrease the ACC and PM activation. Those patients reported decrease in the ongoing level of pain after training, but it was not statistically significant. The fourth study was conducted on healthy participants to study the effectiveness of fMRI neurofeedback on improving motor function by targeting Supplementary Motor Cortex (SMA). Results showed that participants learnt to up-regulate their SMA activation using MI of complex body actions as a mental strategy. In addition, behavioural changes, i.e. shortening of motor reaction time was found in those participants. These results suggest that fMRI neurofeedback can assist participants to develop greater control over motor regions involved in motor-skill learning and it can be translated into an improvement in motor function. In summary, this PhD thesis extends and validates the usefulness of hyperalignment to align the fronto-parietal motor regions and explores its ability to generalise across different levels of motor representation. Furthermore, it sheds light on the dominant role of MI in the AO+MI condition by examining the neural representational similarity of AO+MI and MI tasks. In addition, the fMRI neurofeedback studies in this thesis provide proof-of-principle of using this technology to reduce pain in clinical applications and to enhance motor functions in a healthy population, with the potential for translation into the clinical environment

Deep Learning in Medical Image Analysis

The accelerating power of deep learning in diagnosing diseases will empower physicians and speed up decision making in clinical environments. Applications of modern medical instruments and digitalization of medical care have generated enormous amounts of medical images in recent years. In this big data arena, new deep learning methods and computational models for efficient data processing, analysis, and modeling of the generated data are crucially important for clinical applications and understanding the underlying biological process. This book presents and highlights novel algorithms, architectures, techniques, and applications of deep learning for medical image analysis