Multi-task learning for subthalamic nucleus identification in deep brain stimulation

Deep brain stimulation (DBS) of Subthalamic nucleus (STN) is the most successful treatment for advanced Parkinson’s disease. Localization of the STN through Microelectrode recordings (MER) is a key step during the surgery. However, it is a complex task even for a skilled neurosurgeon. Different researchers have developed methodologies for processing and classification of MER signals to locate the STN. Previous works employ the classical paradigm of supervised classification, assuming independence between patients. The aim of this paper is to introduce a patient-dependent learning scenario, where the predictive ability for STN identification at the level of a particular patient, can be used to improve the accuracy for STN identification in other patients. Our inspiration is the multi-task learning framework, that has been receiving increasing interest within the machine learning community in the last few years. To this end, we employ the multi-task Gaussian processes framework that exhibits state of the art performance in multi-task learning problems. In our context, we assume that each patient undergoing DBS is a different task, and we refer to the method as multi-patient learning. We show that the multi-patient learning framework improves the accuracy in the identification of STN in a range from 4.1 to 7.7%, compared to the usual patient-independent setup, for two different datasets. Given that MER are non stationary and noisy signals. Traditional approaches in machine learning fail to recognize accurately the STN during DBS. By contrast in our proposed method, we properly exploit correlations between patients with similar diseases, obtaining an additional information. This information allows to improve the accuracy not only for locating STN for DBS but also for other biomedical signal classification problems

파킨슨병에서 시상하핵 뇌심부자극술의 미세전극기록으로부터 딥러닝을 이용한 임상 결과 예측

학위논문(석사)--서울대학교 대학원 :의과대학 의학과,2019. 8. 백선하.(Objectives) Deep brain stimulation (DBS) of the subthalamic nucleus (STN) is an effective treatment to improve the motor symptoms of advanced Parkinson disease (PD). Accurate positioning of the stimulation electrodes to STN is mandatory for better clinical outcomes. However, the precise identification of the STN during the microelectrode recording (MER) is not easy. In this study, we analyzed deep learning based MER signals to better predict the clinical outcome of motor function improvement after bilateral STN DBS in patients with advanced PD. (Methods) 696 left MER segments of 4 seconds length from 34 PD patients with advanced PD who underwent bilateral STN DBS surgery under general anesthesia were included in this study. The datasets of thirty patients were assigned to the training set, and the datasets of four patients were assigned to the test set. The wavelet transformed MER and the ratio of DBS on and off Unified Parkinson's Disease Rating Scale(UPDRS) Part III score of the off-medication state were applied for deep learning. According to the ratio, the patients were divided into two groups, high-responder and moderate-responder group. Visual Geometry Group(VGG)-16 model with multi-task learning algorithm was used to estimate the bilateral effect of DBS. To apply the effect of the contralateral score more than ipsilateral score, the ratio of the loss function was varied. Gradient class activation map was used to marking the lesion of interest of CNN. (Results) When we divided MER according to the frequency band and transformed to wavelets, the maximal accuracy was the highest in the 50-500 Hz group, compared with 1-50 Hz and 500-5,000Hz groups. In addition, when the multitask-learning method was applied to 50-500Hz group, the stability of the model was prominently improved. The max accuracy was the highest(80.2%) when the loss ratio of right to left was given as 5:1 or 6:1 in the model. Area under the curve(AUC) was 0.88 in the receiver-operating characteristic(ROC) curve. Gradient class activation map showed that 80-200Hz band was the most commonly referenced area. (Conclusion) We confirmed that the clinical improvement of PD patients who underwent bilateral STN DBS could be predicted based on multi-task deep learning based MER analysis. The deep learning based MER analysis could be helpful for determining the position of the electrode, by predicting motor function improvement.연구 배경 시상하핵의 뇌심부자극술은 진행된 파킨슨병에서 운동 증상을 호전시키는 효과적인 치료이다. 좋은 임상적인 결과를 위해 자극 전극을 정확하게 위치시키는 것이 필요하다. 하지만 미세전극측정을 통해서도 시상하핵을 정확하게 식별하는 것이 쉽지 않다. 이 연구에서는 진행된 파킨슨병 환자에서 딥러닝을 기반으로 미세전극측정을 분석하여 양측 시상하핵 뇌심부자극술 후의 운동기능 호전 정도를 예측하였다. 연구 방법 이 연구에는 전신마취 하에서 양측 시상하핵 뇌심부자극술을 시행받은 34명의 환자로부터 측정된 4초 길이의 좌측 미세전극측정 분절이 포함되었다. 30명의 환자는 훈련군으로 4명의 환자는 실험군으로 구분하였다. 웨이브릿(wavelet) 변환된 미세전극측정 자료와 UPDRS(Unified Parkinson's Disease Rating Scale) 파트 III 중 오프-약물(Off-medication) 시기의 뇌심부자극/비자극 점수가 딥러닝에 사용되었다. 그 비율에 따라 고반응군과 중반응군으로 분류하였다. 다중작업학습 알고리즘을 이용한 VGG-16 모델이 DBS의 양측성 효과를 추정하기 위해 사용되었다. 동측의 점수보다 반대측의 점수를 크게 반영하도록 하기 위해 손실함수(loss function)의 비율을 다양하게 적용 하였다. CNN이 참조한 영역을 표시하기 위해 Grad-CAM을 사용하였다. 연구 결과 미세전극측정신호를 주파수 대역 별로 나누어 웨이브릿 변환하였을 때, 최대정확도는 1-50Hz와 500-5,000Hz와 비교하여 50-500Hz에서 가장 높았다. 게다가 다중작업학습을 적용하였을 때 모델의 안정도가 더 개선되었다. 최대 정확도는 좌우 손실함수의 비율이 5:1과 6:1 때 80.2%로 가장 높았다. 수신자 조작 특성 곡선(ROC curve)에서 곡선하 면적(AUC) 값은 0.88이었다. Grad-CAM에서는 80-200Hz 대역을 가장 흔히 참조한 것을 보여주었다. 연구 결론 미세전극측정의 다중작업학습을 통한 분석으로 파킨슨병 환자에서 양측 시상하핵 뇌심부자극술 시행 후 임상적 호전에 관한 예측이 가능할 것으로 판단하였다. 딥러닝으로 미세전극측정신호를 분석하여 수술 후 운동기능향상을 예측함으로써, 전극의 위치를 결정하는 데에 도움이 될 것으로 기대한다.Introduction 5 PD - DBS - UPDRS 5 Signal - CNN - Clinical outcome 5 Methods 7 Subjects 7 Surgical procedure 8 Microelectrode Recordings 9 Wavelet Transformation 9 Training set and Test set 10 Deep learning 11 Multi-task learning 12 Gradient class activation map 13 Statistical analysis 14 IRB 14 Results 15 Patient Data 15 MER & clinical outcome relation 21 Gradient Class Activation Map 24 Discussion 26 Single-task Learning 26 Multi-task Learning 27 Gradient Class Activation Map 28 Expected Clinical Usefulness 28 Limitation 29 Conclusion 31 References 32Maste

Uncovering the neurophysiology of mood, motivation and behavioral symptoms in Parkinson’s disease through intracranial recordings

Neuropsychiatric mood and motivation symptoms (depression, anxiety, apathy, impulse control disorders) in Parkinson’s disease (PD) are highly disabling, difficult to treat and exacerbated by current medications and deep brain stimulation therapies. High-resolution intracranial recording techniques have the potential to undercover the network dysfunction and cognitive processes that drive these symptoms, towards a principled re-tuning of circuits. We highlight intracranial recording as a valuable tool for mapping and desegregating neural networks and their contribution to mood, motivation and behavioral symptoms, via the ability to dissect multiplexed overlapping spatial and temporal neural components. This technique can be powerfully combined with behavioral paradigms and emerging computational techniques to model underlying latent behavioral states. We review the literature of intracranial recording studies investigating mood, motivation and behavioral symptomatology with reference to 1) emotional processing, 2) executive control 3) subjective valuation (reward & cost evaluation) 4) motor control and 5) learning and updating. This reveals associations between different frequency specific network activities and underlying cognitive processes of reward decision making and action control. If validated, these signals represent potential computational biomarkers of motivational and behavioural states and could lead to principled therapy development for mood, motivation and behavioral symptoms in PD

Bewegungsdekodierung für elektrophysiologisch gestützte intelligente adaptive tiefen Hirnstimulation bei der Parkinson-Krankheit

Deep Brain stimulation is an effective treatment for movement disorders such as Parkinson’s disease or essential tremor. Current therapy protocols do not adjust in real-time to the present need for treatment but instead rely on constant stimulation parameters. A novel concept called intelligent adaptive deep brain stimulation triggers stimulation based on decoding of a predefined state, such as movement, in a demand-driven way. Invasive Brain Computer Interfaces were previously presented for decoding behavioral states both using local field potential recordings from depth electrodes, primarily in movement disorder patients, and using electrocorticographic signals in epilepsy patients. Future brain implants may successfully treat different movement disorders using both modalities. A systematic brain signal decoding comparison of the two recording sites within patients was lacking. In this work, we analyzed invasive intraoperative recordings from Parkinson’s disease patients undergoing deep brain stimulation therapy. Subthalamic local field potentials and simultaneous electrocorticographic signals were recorded while the patients were performing a hand-gripping force task. We used these signals to develop a real-time-enabled feature estimation and decoding framework and investigated different hyperparameter-optimized machine learning approaches for the prediction of movement strength. We identified optimal temporal, spatial, and oscillatory decoding components. Our analysis showed for the first time that movement decoding performances of cortical recordings were superior to subcortical ones using different machine learning methods. We found that gradient-boosted decision trees showed the best performances for electrocorticographic recordings, while Wiener filters were optimal for subthalamic signals. Models from single electrode contacts were better performing than methods that combine data from multiple contacts. Decoding performances were negatively correlated to Parkinson's disease-specific symptom scores. Previously, subthalamic beta oscillations were reported to reflect Parkinson’s disease symptom severity, here we found that decoding performances were negatively correlated to elevated subthalamic beta oscillations. Additionally, we developed a movement decoding network that predicted contact-specific movement decoding performances using functional and structural connectivity profiles. In conclusion, we propose a computational framework based on invasive neurophysiology for brain signal decoding and highlight interactions of decoding performances with Parkinson’s disease symptom states, pathological symptom biomarkers, and whole-brain connectivity. This thesis, therefore, constitutes a significant contribution to the development of intelligent personalized medicine for adaptive deep brain stimulation.Tiefe Hirnstimulation ist eine effektive Behandlung von Bewegungsstörungen wie bei der Parkinson-Krankheit oder dem Essentiellen Tremor. Derzeitige Protokolle passen sich nicht in Echtzeit dem aktuellen Behandlungsbedarf an, sondern beruhen auf konstanten Stimulationsparametern. In einem neuen Therapieverfahren, der „intelligenten adaptiven tiefen Hirnstimulation“, wird die Stimulation bedarfsgerecht anhand eines vordefinierten Zustands, wie beispielsweise der Bewegung, angepasst. Invasive Brain Computer Interfaces konnten in vorigen Studien Verhaltenszustände mit elektrophysiologischen Aufnahmen dekodieren. Hier wurden entweder lokale Feldpotentiale, abgeleitet von Elektroden in tiefen Hirnregionen bei Patient*innen mit Bewegungsstörungen, oder elektrokortikographische Signale, bei Epilepsie-Patient*innen, verwendet. Beide Signal-Modalitäten könnten für zukünftige Hirnimplantate genutzt werden. Ein systematischer Vergleich der jeweiligen Dekodierleistung wurde bei denselben Patient*innen bisher nicht durchgeführt. Hier analysierten wir deshalb intraoperative Aufzeichnungen subthalamischer lokaler Feldpotentiale und gleichzeitige elektrokortikographische Ableitungen von Parkinson-Patient*innen während der Implantation des tiefen Hirnstimulators. Die Patient*innen führten Handbewegungen mit unterschiedlicher Greifkraft aus. Mittels echtzeitfähiger Feature Berechnung und Dekodierung untersuchten wir verschiedene Hyperparameter-optimierte maschinelle Lernverfahren zur Vorhersage der Bewegungsstärke. Wir identifizierten optimale temporale, oszillatorische und lokalisationsspezifische Parameter der Dekodierung. Unsere Studie zeigt zum ersten Mal, dass die Dekodierleistung von kortikalen gegenüber subkortikalen Signalen anhand von verschiedenen maschinellen Lernmethoden deutlich überlegen war. Gradient-boosted decision trees waren für elektrokortikographische Aufzeich-nungen die beste Dekodiermethode, während Wiener Filter für subthalamische Signale am geeignetsten waren. Modelle aus einzelnen Elektrodenkontakten zeigten bessere Dekodierleistungen als Modelle die Daten mehrerer Kontakte kombinierten. Die Dekodierleistung korrelierte negativ mit der Parkinson-Symptomschwere, und korrelierte zusätzlich negativ mit erhöhten subthalamischen Beta-Oszillationen, von denen bereits berichtet wurde, dass sie den Parkinson-Schweregrad widerspiegeln. Zusätzlich entwickelten wir ein Netzwerk für die Vorhersage der kontaktspezifischen Dekodierleistungen anhand von funktionellen und strukturellen Konnektivitätsprofilen. Zusammenfassend stellen wir ein computerbasiertes, neurophysiologisches Framework für die invasive Hirnsignal-Dekodierung vor. Wechselwirkungen der Dekodierleistung wurden mit der Parkinson-Symptomschwere, elektrophysiologischen Biomarkern pathologischer Symptome und der Konnektivität des gesamten Gehirns identifiziert. Diese Dissertation unterstützt daher die Entwicklung intelligenter, personalisierter Medizin für die adaptive tiefe Hirnstimulation

Clinical applications of magnetic resonance imaging based functional and structural connectivity

Advances in computational neuroimaging techniques have expanded the armamentarium of imaging tools available for clinical applications in clinical neuroscience. Non-invasive, in vivo brain MRI structural and functional network mapping has been used to identify therapeutic targets, define eloquent brain regions to preserve, and gain insight into pathological processes and treatments as well as prognostic biomarkers. These tools have the real potential to inform patient-specific treatment strategies. Nevertheless, a realistic appraisal of clinical utility is needed that balances the growing excitement and interest in the field with important limitations associated with these techniques. Quality of the raw data, minutiae of the processing methodology, and the statistical models applied can all impact on the results and their interpretation. A lack of standardization in data acquisition and processing has also resulted in issues with reproducibility. This limitation has had a direct impact on the reliability of these tools and ultimately, confidence in their clinical use. Advances in MRI technology and computational power as well as automation and standardization of processing methods, including machine learning approaches, may help address some of these issues and make these tools more reliable in clinical use. In this review, we will highlight the current clinical uses of MRI connectomics in the diagnosis and treatment of neurological disorders; balancing emerging applications and technologies with limitations of connectivity analytic approaches to present an encompassing and appropriate perspective

Machine learning based brain signal decoding for intelligent adaptive deep brain stimulation

Sensing enabled implantable devices and next-generation neurotechnology allow real-time adjustments of invasive neuromodulation. The identification of symptom and disease-specific biomarkers in invasive brain signal recordings has inspired the idea of demand dependent adaptive deep brain stimulation (aDBS). Expanding the clinical utility of aDBS with machine learning may hold the potential for the next breakthrough in the therapeutic success of clinical brain computer interfaces. To this end, sophisticated machine learning algorithms optimized for decoding of brain states from neural time-series must be developed. To support this venture, this review summarizes the current state of machine learning studies for invasive neurophysiology. After a brief introduction to the machine learning terminology, the transformation of brain recordings into meaningful features for decoding of symptoms and behavior is described. Commonly used machine learning models are explained and analyzed from the perspective of utility for aDBS. This is followed by a critical review on good practices for training and testing to ensure conceptual and practical generalizability for real-time adaptation in clinical settings. Finally, first studies combining machine learning with aDBS are highlighted. This review takes a glimpse into the promising future of intelligent adaptive DBS (iDBS) and concludes by identifying four key ingredients on the road for successful clinical adoption: i) multidisciplinary research teams, ii) publicly available datasets, iii) open-source algorithmic solutions and iv) strong world-wide research collaborations.Fil: Merk, Timon. Charité – Universitätsmedizin Berlin; AlemaniaFil: Peterson, Victoria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Matemática Aplicada del Litoral. Universidad Nacional del Litoral. Instituto de Matemática Aplicada del Litoral; Argentina. Harvard Medical School; Estados UnidosFil: Köhler, Richard. Charité – Universitätsmedizin Berlin; AlemaniaFil: Haufe, Stefan. Charité – Universitätsmedizin Berlin; AlemaniaFil: Richardson, R. Mark. Harvard Medical School; Estados UnidosFil: Neumann, Wolf Julian. Charité – Universitätsmedizin Berlin; Alemani