Behavior Classification Using Multi-site LFP and ECoG Signals

Abstract-Deep Brain Stimulation (DBS) is an effective therapy that alleviates the motor signs of Parkinson’s disease (PD). Existing DBS is open loop, providing a time invariant stimulation pulse train that may generate cognitive, speech, and balance side effects. A closed-loop DBS system that utilizes appropriate physiological control variables may improve therapeutic results, reduce stimulation side effects, and extend battery life of pulse generators. Furthermore, by customizing DBS to a patient’s behavioral goal, side effects of stimulation may arise only when they are non-detrimental to the patient’s current goals. Therefore, classification of human behavior using physiological signals is an important step in the design of the next generation of closed-loop DBS systems. Ten subjects who were undergoing DBS implantation were recruited for the study. DBS leads were used to record bilateral STN-LFP activity and an electrocorticography (ECoG) strip was used to record field potentials over left prefrontal cortex. Subjects were cued to perform voluntary behaviors including left and right hand movement, left and right arm movement, mouth movement, and speech. Two types of algorithms were used to classify the subjects’ behavior, support vector machine (SVM) using linear, polynomial, and RBF kernels as well as lp-norm multiple kernel learning (MKL). Behavioral classification was performed using only LFP channels, only ECoG channels, and both LFP and ECoG channels. Features were extracted from the time-frequency representation of the signals. Phase locking values (PLV) between ECoG and LFP channels were calculated to determine connectivity between sites and aid in feature selection. Classification performance improved when multi-site signals were used with either SVM or MKL algorithms. Our experiments further show that the lp-norm MKL outperforms single kernel SVM-based classifiers in classifying behavioral tasks. References [1] H. M. Golshan, A. O. Hebb, S. J. Hanrahan, J. Nedrud, and M. H. Mahoor, “A multiple kernel learning approach for human behavioral task classification using STN-LFP signal,” EMBC, 38th IEEE International Conference on., pp.1030-1033, 2016. [2] H. M. Golshan, A. O. Hebb, S. J. Hanrahan, J. Nedrud, and M. H. Mahoor, “An FFT-based synchronization approach to recognize human behaviors using STN-LFP signal,” To appear in ICASSP, 42nd IEEE International Conference on., 2017

Developing Machine Learning Algorithms for Behavior Recognition from Deep Brain Signals

Parkinson’s disease (PD) is a neurodegenerative condition and movement disorder that appears with symptoms such as tremor, rigidity of muscles and slowness of movements. Deep brain stimulation (DBS) is an FDA-approved surgical therapy for essential tremor and PD. Despite the fact that DBS substantially alleviates the motor signs of PD, it can cause cognitive side effects and speech malfunction mainly due to the lack of adaptivity and optimality of the stimulation signal to the patients’ current state. A behavior-adapted closed-loop DBS system may reduce the side effects and power consumption by adjusting the stimulation parameters to patients’ need. Behavior recognition based on physiological feedbacks plays a key role in designing the next generation of closed-loop DBS systems. Hence, this dissertation is concentrated on: 1. Investigating the capability of local field potential (LFP) signals recorded from Subthalamic nucleus (STN) in identifying behavioral activities 2. Developing advanced machine learning algorithms to recognize behavioral activities using LFP signals 3. Investigating the effects of medication and stimulation pulse on the behavior recognition task as well as characteristics of the LFP signal. STN-LFP is a great physiological signal candidate since the stimulation device itself can record it, eliminating the need for additional sensors. Continuous wavelet transform is utilized for time-frequency analysis of STN-LFPs. Experimental results demonstrate that different behaviors create different modulation patterns in STN within the beta frequency range. A hierarchical classification structure is proposed to perform the behavior classification through a multi-level framework. The beta frequency components of STN-LFPs recorded from all contacts of DBS leads are combined through an MKL-based SVM classifier for behavior classification. Alternatively, the inter-hemispheric synchronization of the LFP signals measured by an FFT-based synchronization approach is utilized to pair up the LFP signals from left and right STNs. Using these rearranged LFP signals reduces the computational cost significantly while keeping the classification ability almost unchanged. LFP-Net, a customized deep convolutional neural network (CNN) approach for behavior classification, is also proposed. CNNs learn different feature maps based on the beta power patterns associated with different behaviors. The features extracted by CNNs are passed through fully connected layers, and, then to the softmax layer for classification. The effect of medication and stimulation “off/on” conditions on characteristics of LFP signals and the behavior classification performance is studied. The beta power of LFP signals under different stimulation and medication paradigms is investigated. Experimental results confirm that the beta power is suppressed significantly when the patients take medication or therapeutic stimulation. The results also show that the behavior classification performance is not impacted by different medication or stimulation conditions. Identifying human behavioral activities from physiological signals is a stepping-stone toward adaptive closed-loop DBS systems. To design such systems, however, there are other open questions that need to be addressed, which are beyond the scope of this dissertation, such as developing event-related biomarkers, customizing the parameter of DBS system based on the patients’ current state, investigating the power consumption and computational complexity of the behavior recognition algorithms

Machine learning based brain signal decoding for intelligent adaptive deep brain stimulation

Sensing enabled implantable devices and next-generation neurotechnology allow real-time adjustments of invasive neuromodulation. The identification of symptom and disease-specific biomarkers in invasive brain signal recordings has inspired the idea of demand dependent adaptive deep brain stimulation (aDBS). Expanding the clinical utility of aDBS with machine learning may hold the potential for the next breakthrough in the therapeutic success of clinical brain computer interfaces. To this end, sophisticated machine learning algorithms optimized for decoding of brain states from neural time-series must be developed. To support this venture, this review summarizes the current state of machine learning studies for invasive neurophysiology. After a brief introduction to the machine learning terminology, the transformation of brain recordings into meaningful features for decoding of symptoms and behavior is described. Commonly used machine learning models are explained and analyzed from the perspective of utility for aDBS. This is followed by a critical review on good practices for training and testing to ensure conceptual and practical generalizability for real-time adaptation in clinical settings. Finally, first studies combining machine learning with aDBS are highlighted. This review takes a glimpse into the promising future of intelligent adaptive DBS (iDBS) and concludes by identifying four key ingredients on the road for successful clinical adoption: i) multidisciplinary research teams, ii) publicly available datasets, iii) open-source algorithmic solutions and iv) strong world-wide research collaborations.Fil: Merk, Timon. Charité – Universitätsmedizin Berlin; AlemaniaFil: Peterson, Victoria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Matemática Aplicada del Litoral. Universidad Nacional del Litoral. Instituto de Matemática Aplicada del Litoral; Argentina. Harvard Medical School; Estados UnidosFil: Köhler, Richard. Charité – Universitätsmedizin Berlin; AlemaniaFil: Haufe, Stefan. Charité – Universitätsmedizin Berlin; AlemaniaFil: Richardson, R. Mark. Harvard Medical School; Estados UnidosFil: Neumann, Wolf Julian. Charité – Universitätsmedizin Berlin; Alemani

Human Behavior Recognition Ssing Brain LFP Signal in the Presence of the Stimulation Pulse

Design and Methodology This study concentrates on human behavior classification task using local field potential (LFP) signals recorded from three subjects with Parkinson’s disease (PD). Existing approaches mainly employ the LFP signals acquired under the stimulation/off condition. In practical situations, however, it is necessary to design a classification method capable of recognizing different human activities under the stimulation/on condition, where the classification task is more complicated due to the artifacts imposed by the high amplitude stimulation pulse (~1-3volts). We utilize the time-frequency representation of the acquired LFPs in the Beta frequency range (~10-30Hz) to develop a feature space based on which the classification is efficiently performed while the high frequency stimulation pulse (~130-180Hz) has no/limited impact on the classification performance. Original Data and Results All three participants had undergone DBS surgery with implanted DBS leads (Medtronic 3389, Minneapolis, MN, USA) in the subthalamic nucleus of the brain. The recording sessions required the participants to do several repetitions of designed “button press” and “reach” trials under the condition of stimulation on/off. On average, 60 recordings were performed for each trial. Our analysis on the power spectral density (PSD) of the data showed that the stimulation pulse mostly impacts the frequency components around the stimulation frequency (~140Hz). Using a linear-kernel SVM classifier for classifying the aforementioned trials based on the proposed feature space, we obtained a classification accuracy of ~88% and ~87% respectively for stimulation off and on cases. Conclusion PD incidence increases with advancing age and peaks among people in their 60s and 70s. The cost of PD in the United States is estimated to be $25 billion per year. Thus, advanced techniques to improve the performance of existing devices are highly demanded. Human behavior classification from brain signals is essential in developing the next generation of closed-loop deep brain stimulation (DBS) systems. A closed-loop DBS system that utilizes appropriate physiological control variables may improve therapeutic results, reduce stimulation side effects, and extend battery life of pulse generators

Hyper-parameter tuning and feature extraction for asynchronous action detection from sub-thalamic nucleus local field potentials

Introduction: Decoding brain states from subcortical local field potentials (LFPs) indicative of activities such as voluntary movement, tremor, or sleep stages, holds significant potential in treating neurodegenerative disorders and offers new paradigms in brain-computer interface (BCI). Identified states can serve as control signals in coupled human-machine systems, e.g., to regulate deep brain stimulation (DBS) therapy or control prosthetic limbs. However, the behavior, performance, and efficiency of LFP decoders depend on an array of design and calibration settings encapsulated into a single set of hyper-parameters. Although methods exist to tune hyper-parameters automatically, decoders are typically found through exhaustive trial-and-error, manual search, and intuitive experience. Methods: This study introduces a Bayesian optimization (BO) approach to hyper-parameter tuning, applicable through feature extraction, channel selection, classification, and stage transition stages of the entire decoding pipeline. The optimization method is compared with five real-time feature extraction methods paired with four classifiers to decode voluntary movement asynchronously based on LFPs recorded with DBS electrodes implanted in the subthalamic nucleus of Parkinson’s disease patients. Results: Detection performance, measured as the geometric mean between classifier specificity and sensitivity, is automatically optimized. BO demonstrates improved decoding performance from initial parameter setting across all methods. The best decoders achieve a maximum performance of 0.74 ± 0.06 (mean ± SD across all participants) sensitivity-specificity geometric mean. In addition, parameter relevance is determined using the BO surrogate models. Discussion: Hyper-parameters tend to be sub-optimally fixed across different users rather than individually adjusted or even specifically set for a decoding task. The relevance of each parameter to the optimization problem and comparisons between algorithms can also be difficult to track with the evolution of the decoding problem. We believe that the proposed decoding pipeline and BO approach is a promising solution to such challenges surrounding hyper-parameter tuning and that the study’s findings can inform future design iterations of neural decoders for adaptive DBS and BCI

Towards Closed-Loop Deep Brain Stimulation: Behavior Recognition from Human STN

Deep brain stimulation (DBS) provides significant therapeutic benefit for movement disorders such as Parkinson’s disease (PD). Current DBS devices lack real-time feedback (thus are open loop) and stimulation parameters are adjusted during scheduled visits with a clinician. A closed-loop DBS system may reduce power consumption and side effects by adjusting stimulation parameters based on patient’s behavior. Thus behavior detection is a major step in designing such systems. Various physiological signals can be used to recognize the behaviors. Subthalamic Nucleus (STN) Local field Potential (LFP) is a great candidate signal for the neural feedback, because it can be recorded from the stimulation lead and does not require additional sensors. This thesis proposes novel detection and classification techniques for behavior recognition based on deep brain LFP. Behavior detection from such signals is the vital step in developing the next generation of closed-loop DBS devices. LFP recordings from 13 subjects are utilized in this study to design and evaluate our method. Recordings were performed during the surgery and the subjects were asked to perform various behavioral tasks. Various techniques are used understand how the behaviors modulate the STN. One method studies the time-frequency patterns in the STN LFP during the tasks. Another method measures the temporal inter-hemispheric connectivity of the STN as well as the connectivity between STN and Pre-frontal Cortex (PFC). Experimental results demonstrate that different behaviors create different modulation patterns in STN and it’s connectivity. We use these patterns as features to classify behaviors. A method for single trial recognition of the patient’s current task is proposed. This method uses wavelet coefficients as features and support vector machine (SVM) as the classifier for recognition of a selection of behaviors: speech, motor, and random. The proposed method is 82.4% accurate for the binary classification and 73.2% for classifying three tasks. As the next step, a practical behavior detection method which asynchronously detects behaviors is proposed. This method does not use any prior knowledge of behavior onsets and is capable of asynchronously detect the finger movements of PD patients. Our study indicates that there is a motor-modulated inter-hemispheric connectivity between LFP signals recorded bilaterally from STN. We utilize a non-linear regression method to measure this inter-hemispheric connectivity and to detect the finger movements. Our experimental results using STN LFP recorded from eight patients with PD demonstrate this is a promising approach for behavior detection and developing novel closed-loop DBS systems

Electrocorticography is superior to subthalamic local field potentials for movement decoding in Parkinson’s disease

Brain signal decoding promises significant advances in the development of clinical brain computer interfaces (BCI). In Parkinson's disease (PD), first bidirectional BCI implants for adaptive deep brain stimulation (DBS) are now available. Brain signal decoding can extend the clinical utility of adaptive DBS but the impact of neural source, computational methods and PD pathophysiology on decoding performance are unknown. This represents an unmet need for the development of future neurotechnology. To address this, we developed an invasive brain-signal decoding approach based on intraoperative sensorimotor electrocorticography (ECoG) and subthalamic LFP to predict grip-force, a representative movement decoding application, in 11 PD patients undergoing DBS. We demonstrate that ECoG is superior to subthalamic LFP for accurate grip-force decoding. Gradient boosted decision trees (XGBOOST) outperformed other model architectures. ECoG based decoding performance negatively correlated with motor impairment, which could be attributed to subthalamic beta bursts in the motor preparation and movement period. This highlights the impact of PD pathophysiology on the neural capacity to encode movement vigor. Finally, we developed a connectomic analysis that could predict grip-force decoding performance of individual ECoG channels across patients by using their connectomic fingerprints. Our study provides a neurophysiological and computational framework for invasive brain signal decoding to aid the development of an individualized precision-medicine approach to intelligent adaptive DBS

Decoding of Human Movements Based on Deep Brain Local Field Potentials Using Ensemble Neural Networks

Decoding neural activities related to voluntary and involuntary movements is fundamental to understanding human brain motor circuits and neuromotor disorders and can lead to the development of neuromotor prosthetic devices for neurorehabilitation. This study explores using recorded deep brain local field potentials (LFPs) for robust movement decoding of Parkinson’s disease (PD) and Dystonia patients. The LFP data from voluntary movement activities such as left and right hand index finger clicking were recorded from patients who underwent surgeries for implantation of deep brain stimulation electrodes. Movement-related LFP signal features were extracted by computing instantaneous power related to motor response in different neural frequency bands. An innovative neural network ensemble classifier has been proposed and developed for accurate prediction of finger movement and its forthcoming laterality. The ensemble classifier contains three base neural network classifiers, namely, feedforward, radial basis, and probabilistic neural networks. The majority voting rule is used to fuse the decisions of the three base classifiers to generate the final decision of the ensemble classifier. The overall decoding performance reaches a level of agreement (kappa value) at about 0.729±0.16 for decoding movement from the resting state and about 0.671±0.14 for decoding left and right visually cued movements