Neural activity results in chemical changes in the extracellular environment such as variation in pH or potassium/ sodium ion concentration. Higher signal to noise ratio make neurochemical signals an interesting biomarker for closed-loop neuromodulation systems. For such applications, it is important to reliably classify pH signatures to control stimulation timing and possibly dosage. For example, the activity of the subdiaphragmatic vagus nerve (sVN) branch can be monitored by measuring extracellular neural pH. More importantly, gut hormone cholecystokinin (CCK)-specific activity on the sVN can be used for controllably activating sVN, in order to mimic the gut-brain neural response to food intake. In this paper, we present a convolutional neural network (CNN) based classification system to identify CCK-specific neurochemical changes on the sVN, from non-linear background activity. Here we present a novel feature engineering approach which enables, after training, a high accuracy classification of neurochemical signals using CNN

Mirza, KB

Nikolic, K

Roever, P

Toumazou, C

English

Spiral - Imperial College Digital Repository

A Convolutional Neural Network for Classificationof Nerve Activity based on Action PotentialInduced Neurochemical SignaturesPaul Roever∗, Khalid B. Mirza†, Konstantin Nikolic†, Christofer Toumazou†∗Department of Bioengineering, Imperial College London† Centre for Bio-Inspired Technology, Imperial College London, SW7 2BT, UK.Email: {p.roever15,k.mirza,k.nikolic,c.toumazou}@imperial.ac.ukAbstract—Neural activity results in chemical changes in theextracellular environment such as variation in pH or potas-sium/sodium ion concentration. Higher signal to noise ratio makeneurochemical signals an interesting biomarker for closed-loopneuromodulation systems. For such applications, it is impor-tant to reliably classify pH signatures to control stimulationtiming and possibly dosage. For example, the activity of thesubdiaphragmatic vagus nerve (sVN) branch can be monitoredby measuring extracellular neural pH. More importantly, guthormone cholecystokinin (CCK)-specific activity on the sVN canbe used for controllably activating sVN, in order to mimic thegut-brain neural response to food intake. In this paper, we presenta convolutional neural network (CNN) based classification systemto identify CCK-specific neurochemical changes on the sVN,from non-linear background activity. Here we present a novelfeature engineering approach which enables, after training, ahigh accuracy classification of neurochemical signals using CNN.I. INTRODUCTIONClosed-loop neuromodulation systems are the next genera-tion frontier in neuromodulatory devices. The benefits includeimproved therapeutic, clinical and engineering outcomes suchas improved amelioration of symptoms, possibly better re-sponse to neuroplasticity and adaption, improved compliance,lower power consumption and longer battery lifetimes inimplant [1]. Furthermore, closed-loop neuromodulation offersthe oppurtunity to develop a more personalised approach to-wards neuromodulation therapies. However, a significant steptowards achieving an effective closed-loop system is selectionof an appropriate symptomatic biomarker [2]. It has also beenshown to offer better therapeutic outcome [1], [3].Neurochemical signals can be further classified into thoseoriginating from ions such as potassium (K+), sodium (Na+)and those originating from neurotransmitters such as dopamineserotonin etc. The former is directly to linked to electricalneural activity and can be observed in Central Nervous System(CNS) and Peripheral Nervous System (PNS). The latter isgenerally observed in CNS, more specifically in the presenceof cell bodies where inter-neuron response transmission oc-curs. More research has been performed in observing neu-rotransmitters compared to ionic responses with the primaryreason being difficulty in fabricating implantable, penetrating,ion-selective microelectrodes until recently [2].Fig. 1. A neural network can be used to accurately detect neurochemicalresponse and predict target biomarker concentration followed by indicationon whether to start stimulation or not.While performing neurochemical recording in vivo, bothlinear and non-linear background signals have been observedwith various background subtraction techniques being usedpreviously [4], [5]. The next step after background subtractionis response classification. This is a binary classification stagewhere the response is determined to be originating from thetarget physiological biomarker, in this case CCK, or not. Basedon the outcome of this classification, the next level consists ofmulti-level classification where the concentration of hormoneis determined based on response characteristics.Previous papers have explored modelling the relationshipbetween neurochemical levels and stimulation parameters [6].In this paper, a neural network based classification systemis described while considering various trade-offs and datapreparation techniques. After drift removal, the pre-processedsignal is labelled and used to train a neural network. The net-work is then optimised and used to perform classification. Theprecision and recall results for this network are presented. Thispaper focusses primarily on a methodology for identifying andenhancing features for noisy neurochemical signals.In this paper, Section II provides a brief introduction intoexisting closed-loop systems, our analysis of a suitable neuralnetwork architecture for classification of timeseries signalssuch as those described in this paper and the functional unitsof an implant processing unit.II. BACKGROUNDA. Neurochemical Changes in the Vagus NerveThe vagus nerve innervates the subdiaphragmatic regionon the ventral and dorsal gut regions. It links the gut tothe brain and plays a crucial role in appetite control throughinteraction of several gut hormones [7]. It is well-establishedthat cholecystokinin (CCK), secreted in the intestines, playsa crucial role in anorectic effects through the vagus nerve[8], [9]. In the experiments described in this paper, CCK isused to activate signalling in the vagus nerve by intravenousinjection. This results in activation of the vagus nerve resultingin changes in neurochemical pH.B. Closed Loop SystemsA closed-loop system consists of sensors/electrodes fol-lowed by data acquisition, pre-processing, possible dimen-sionality reduction followed by decision and stimulation dosetuning unit. The ultimate goal of a closed-loop system is toaccurately and autonomously answer when? and how much?to stimulate. Closed-loop systems can be divided into differenttypes, depending on functionality and type of control feedback.These are responsive: a partial closed-loop system whereone of the two questions (when?, how much?) is answeredon the basis of a single physiological biomarker and a pre-determined threshold of this biomarker, adaptive: where asingle biomarker is monitored but both when? and howmuch? are answered, complete closed-loop: where multiplebiomarkers are monitored and considered to deliver closed-loop functionality [2], [10].In order to implement autonomous detection, algorithmsneed to be trained using an experimental dataset [11], [5],[2]. The training step is performed on in vivo or in vitrodata. A variety of different algorithms have been used pre-viously [11], [5], [2]. These include machine learning (ML)algorithms such as principal component regression (PCR),which is a combination of principal component analysis andlinear regression. Artificial Neural networks (ANN) have beeninvestigated previously in for neurotransmitter release due toelectrical stimulation [11], [6]. ANN were used to model boththe relationship between stimulation-evoked dopamine releaseand vice-versa [6].III. METHODOLOGYThis section describes the methodology for training andvalidating a CNN for classification of neurochemical data.It consists of data preparation, labelling and optimisation fortraining and validation of the CNN. The experimental dataconsisted of neurochemical pH recorded in vivo consistingof one saline injection and four CCK injections of differentdosages (50, 100, 300, 1000 pmol/kg) [12].Fig. 3. (A)The drift in raw sensor data was removed using a techniquedescribed earlier [4]. The outliers were removed using the modified z-scoreprocess [13].A. Data PreparationThe raw neurochemical data consisted of biomarker inducedneurochemical changes, background neurochemical activityand electrode drift. The raw data is pre-processed to removedrift using the technique described in [4]. After drift removal,the outliers were identified and removed. For outlier removala modified z-score process was used, first described in [13](Figure III-A). This was used to remove biased scaling due tovery large data values.After outlier removal, the data was partitioned into temporalwindows of 15 seconds with an overlap of about 5 seconds.Following this the data was normalised to be such that its valuewas between 0 and 1. The normalisation was done using thefollowing formula.ynorm =yi − yminymax − ymin(1)The data was then labelled either 1 or 0 depending onwhether the datapoint was part of CCK-induced neurochemicalresponse or not, respectively (See Figure III-A).B. Class size ImbalanceClass imbalance is a common issue in ML-based classifica-tion problems, where the objects belonging to one class exceedthe number of objects in other classes. This problem can besolved through either creation or oversampling of minorityclass objects or undersampling of the majority class. In thispaper, the number of datapoints which are not part of theCCK-induced response far outnumber those which originatefrom the CCK-induced response. Hence, we use SyntheticMinority Oversampling Technique (SMOTE) to deal with thisissue [14]. In this case, SMOTE was applied to the data, post-windowing, such that there are equal number of positives (1)and negatives (0) in the training dataset.C. Model Training and ValidationA training data set of N experiments were prepared andprovided to models generated using Keras 2.2.4 with Ten-sorFlow 1.13.1 as a backend [15]. The model was providedwith a training data tensor of shape (Nwindows, Lwindows, 1),where Nwindows is the number of training windows, Lwindowsis the length of each window. After a model architecture wasselected, batches of training data were used to train the model.The batch size is used to determine the number of times perFig. 2. Different steps used to train and cross-validate a CNN for detection and classification of neurochemical response.epoch, the weights are updated. After one complete run oftraining a model i.e epoch, the model was cross-validated ona cross-validation dataset. If the model performed better thanthe previous one, then the model was saved to be used later.In order to prevent plateauing of model performance, alearning rate scheduler was used. Models in which the per-formance did not improve after 10 epochs had their learningrate halved. Models which did not improve after 40 epochswere discarded to reduce training time.D. Model Architecture and Hyperparameter OptimisationAfter completion of training and validation, objective eval-uation of different model hyperparameters and architectureswas validated with the same validation experiments using k-fold cross validation. Various model architectures, activationfunctions were tried ranging from dense neural networksto convolutional and recurrent neural networks were imple-mented using open-source Neural Network Intelligence (NNI),Microsoft. Smaller changes were made further to manuallyimprove model performance.TABLE IMODEL PARAMETERS OF CNN USED IN THIS PAPERParameters ValuesConvolutional Layers 4Channels per layer 16Kernels per Channel 3Stride Length 1Dropout Fraction 0.2Activation Function ReLUOptimisation Adam [16]Batch Size 64IV. RESULTSA number of different parameters were tried to achieveoptimal results with regards to training time and correctclassification results.Fig. 4. An optimal batch side is chosen based on optimal validation lossvalues.A. Network Parameters: Batch Size, Window Length, DropoutRate1) Batch Size: There is a trade-off involved in the selectionof batch size. If a large batch size is chosen i.e updatingweights happens less frequently per epoch has less perfor-mance overhead compared to a small batch size. The impactof different batch sizes is seen in Figure 4. Figure 4 shows themodel performance (validation loss) vs training time for eachbatch size. A batch size of 64 indicates optimal performancewith fast training time and good model performance.2) Window Length: The length of training windows werevaried to determine an optimal window size. This is significantbecause it will help determine how early a CCK response isdetected. A window length of 10 seconds with an overlapof 8 seconds is found to be optimal. Shorter time windowsdoes not allow enough temporal features to be captured, hencemodel performance drops. Longer windows results in excessdata bleeding in from the non-CCK areas.3) Dropout rate: Dropout is a tool used for regularisationby randomly deactivating a proportion of neurons in a modelduring each training step [17]. This is performed to reduceoverfitting and also to prevent neurons from relying too heavilyon specific features. This also means readjustment of neuronweights as neurons with connections to dropped-out neuronswill have to rely heavily on existing neurons. The effect onvalidation loss due to dropout is shown in Figure 5. A dropoutof 20% was chosen as an optimal value.Fig. 5. The effect of different dropout rates show the effect on validationloss (A) and training loss (B). This is used to determine an optimal value fordroput rate while training the model.B. Overall Performance and Improvements : Precision andRecallThe overall performance of the final model (parametersshown in Table I) indicates an accuracy of 93.27% (fractionof correct predictions compared to total predictions). Thiswas achieved with a minimum validation loss of 0.186. Theoptimisation algorithm used for the CNN was ADAM. Asample of prediction accuracy of the model is shown in Figure6 and overall results shown in Table II i.e number of correctcalls (CC), false negatives (FN) and false positives (FP).TABLE IIPRECISION AND RECALL RESULTS ON ENTIRE CROSS-VALIDATIONDATASETPredicted Label Non CCK Event CCK EventTarget LabelNon CCK Event 23010 (CC) 204 (FP)CCK Event 1151 (FN) 502 (CC)V. CONCLUSIONThis paper describes data pre-processing steps, feature en-hancement, development and optimisation of a convolutionalneural network model for detecting neurochemical changesin a real-time application. The advantage of using a CNN isthat it ensures early, more accurate detection of neurochemicalchanges. Although the model successfully detects neurochem-ical changes, however the accuracy, to some extent, is affectedby class-size imbalance for real-time detection. Hence, thereis a need either for better quality training data or additionalfeature information which can further improve the accuracy ofthe model.Fig. 6. (A) A cross-validation, experimental data set is labelled as shown. (B)Shows the datapoints predicted by the CNN model for different concentrationsof intravenously injected CCK.ACKNOWLEDGMENTThis work was supported by the European Research Council (Syn-ergy Grant No. 319818, i2MOVE), UK EPSRC grant EP/N002474/1,and EPSRC-Imperial Impact Acceleration Award (2019).REFERENCES[1] B. Rosin, M. Slovik, R. Mitelman, M. Rivlin-Etzion, S. N. Haber, Z. Is-rael, E. Vaadia, and H. Bergman, “Closed-loop deep brain stimulationis superior in ameliorating parkinsonism,” Neuron, vol. 72, no. 2, pp.370–384, 2011.[2] K. B. Mirza, N. Kulasekeram, Y. Liu, K. Nikolic, and C. Toumazou,“System on chip for closed loop neuromodulation based on dual modebiosignals,” Proceedings - IEEE International Symposium on Circuitsand Systems, vol. 2019-May, 2019.[3] F. T. Sun and M. J. Morrell, “Closed-loop Neurostimulation: TheClinical Experience,” Neurotherapeutics, vol. 11, no. 3, pp. 553–563,2014.[4] T. Ahmed, K. B. Mirza, and K. Nikolic, “Resource Efficient Pre-processor for Drift Removal in Neurochemical Signals,” in IEEE In-ternational Symposium on Circuits and Systems, 2018, pp. 1–5.[5] B. Bozorgzadeh, D. P. Covey, B. A. Heidenreich, P. A. Garris, andP. Mohseni, “Real-time processing of fast-scan cyclic voltammetry(FSCV) data using a field-programmable gate array (FPGA),” Interna-tional Conference of the IEEE Engineering in Medicine and BiologySociety (EMBS), vol. 2014, pp. 2036–2039, 2014.[6] P. J. Grahn, G. W. Mallory, O. U. Khurram, B. M. Berry, J. T. Hachmann,A. J. Bieber, K. E. Bennet, H.-k. Min, S.-y. Chang, K. H. Lee, andJ. L. Lujan, “A neurochemical closed-loop controller for deep brainstimulation : Towards individualized smart neuromodulation therapies,”vol. 8, no. June, pp. 1–11, 2014.[7] S. C. Cork, “The role of the vagus nerve in appetite control: Implicationsfor the pathogenesis of obesity,” Journal of Neuroendocrinology, vol. 30,no. 11, pp. 1–10, 2018.[8] S. Okano-Matsumoto, J. A. McRoberts, Y. Taché, and D. W. Adelson,“Electrophysiological evidence for distinct vagal pathways mediatingCCK-evoked motor effects in the proximal versus distal stomach.” TheJournal of physiology, vol. 589, no. Pt 2, pp. 371–93, 2011.[9] K. B. Mirza, A. Alenda, A. Eftekhar, N. Grossman, K. Nikolic, andR. Stephen, “Influence Of Cholecystokinin-8 On Compund Nerve ActionPotentials From Ventral Gastric Vagus In Rats,” International Journalof Neural Systems, vol. 28, no. 09, p. 1850006, 2018.[10] K. B. Mirza, K. Wildner, N. Kulasekeram, S. C. Cork, S. R. Bloom,K. Nikolic, and C. Toumazou, “Live Demo: Platform for Closed LoopNeuromodulation Based on Dual Mode Biosignals,” in IEEE 2017Biomedical Circuits and Systems Conference, BioCAS 2017, Milan,2017.[11] J. K. Trevathan, A. Yousefi, H. O. Park, J. J. Bartoletta, K. A. Ludwig,K. H. Lee, and J. L. Lujan, “Computational Modeling of Neurotrans-mitter Release Evoked by Electrical Stimulation: Nonlinear Approachesto Predicting Stimulation-Evoked Dopamine Release,” ACS ChemicalNeuroscience, vol. 18, no. 3, pp. 386–392, 2017.[12] S. C. Cork, A. Eftekhar, K. B. Mirza, C. Zuliani, K. Nikolic, J. V.Gardiner, S. R. Bloom, and C. Toumazou, “Extracellular pH monitoringfor use in closed-loop vagus nerve stimulation,” Journal of NeuralEngineering, vol. 15, no. 1, p. 016001, feb 2018.[13] A. Schmidt and Z. Bandar, “A modular neural network architecture withadditional generalization abilities for large input vectors,” in ArtificialNeural Nets and Genetic Algorithms. Springer, 1998, pp. 35–39.[14] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “Smote:synthetic minority over-sampling technique,” Journal of artificial intel-ligence research, vol. 16, pp. 321–357, 2002.[15] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S.Corrado, A. Davis, J. Dean, M. Devin et al., “Tensorflow: Large-scalemachine learning on heterogeneous distributed systems,” arXiv preprintarXiv:1603.04467, 2016.[16] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014.[17] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut-dinov, “Dropout: a simple way to prevent neural networks from over-fitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp.1929–1958, 2014.

Convolutional neural network for classification of nerve activity based on action potential induced neurochemical Signatures

Crossref

Video for A Convolutional Neural Network for Classification of Nerve Activity Based on Action Potential Induced Neurochemical Signatures

Neural activity results in chemical changes in the extracellular environment such as variation in pH or potassium/sodium ion concentration. Higher signal to noise ratio make neurochemical signals an interesting biomarker for closed-loop neuromodulation systems. For such applications, it is important to reliably classify pH signatures to control stimulation timing and possibly dosage. For example, the activity of the subdiaphragmatic vagus nerve (sVN) branch can be monitored by measuring extracellular neural pH. More importantly, gut hormone cholecystokinin (CCK)-specific activity on the sVN can be used for controllably activating sVN, in order to mimic the

gut-brain neural response to food intake. In this paper, we present a convolutional neural network (CNN) based classification system to identify CCK-specific neurochemical changes on the sVN,from non-linear background activity. Here we present a novel feature engineering approach which enables, after training, a high accuracy classification of neurochemical signals using CNN

Roever, Paul

Mirza, Khalid

Nikolic, Konstantin

Toumazou, Chris

UWL Repository

A Convolutional Neural Network for Classificationof Nerve Activity based on Action PotentialInduced Neurochemical SignaturesPaul Roever∗, Khalid B. Mirza†, Konstantin Nikolic†, Christofer Toumazou†∗Department of Bioengineering, Imperial College London† Centre for Bio-Inspired Technology, Imperial College London, SW7 2BT, UK.Email: {p.roever15,k.mirza,k.nikolic,c.toumazou}@imperial.ac.ukAbstract—Neural activity results in chemical changes in theextracellular environment such as variation in pH or potas-sium/sodium ion concentration. Higher signal to noise ratio makeneurochemical signals an interesting biomarker for closed-loopneuromodulation systems. For such applications, it is impor-tant to reliably classify pH signatures to control stimulationtiming and possibly dosage. For example, the activity of thesubdiaphragmatic vagus nerve (sVN) branch can be monitoredby measuring extracellular neural pH. More importantly, guthormone cholecystokinin (CCK)-specific activity on the sVN canbe used for controllably activating sVN, in order to mimic thegut-brain neural response to food intake. In this paper, we presenta convolutional neural network (CNN) based classification systemto identify CCK-specific neurochemical changes on the sVN,from non-linear background activity. Here we present a novelfeature engineering approach which enables, after training, ahigh accuracy classification of neurochemical signals using CNN.I. INTRODUCTIONClosed-loop neuromodulation systems are the next genera-tion frontier in neuromodulatory devices. The benefits includeimproved therapeutic, clinical and engineering outcomes suchas improved amelioration of symptoms, possibly better re-sponse to neuroplasticity and adaption, improved compliance,lower power consumption and longer battery lifetimes in im-plant [1]. Furthermore, closed-loop neuromodulation offers theoppurtunity to develop a more personalised approach towardsneuromodulation therapies. It has also been shown to offerbetter therapeutic outcome [1], [2]. However, a significant steptowards achieving an effective closed-loop system is selectionof an appropriate symptomatic biomarker [3].Neurochemical signals can be further classified into thoseoriginating from ions such as potassium (K+), sodium (Na+)and those originating from neurotransmitters such as dopamineserotonin etc. The former is directly to linked to electricalneural activity and can be observed in Central Nervous System(CNS) and Peripheral Nervous System (PNS). The latter isgenerally observed in CNS, more specifically in the presenceof cell bodies where inter-neuron response transmission oc-curs. More research has been performed in observing neu-rotransmitters compared to ionic responses with the primaryreason being difficulty in fabricating implantable, penetrating,ion-selective microelectrodes until recently [3].Fig. 1. A neural network can be used to accurately detect neurochemicalresponse and predict target biomarker concentration followed by indicationon whether to start stimulation or not.While performing neurochemical recording in vivo, bothlinear and non-linear background signals have been observedwith various background subtraction techniques being usedpreviously [4], [5]. The next step after background subtractionis response classification. This is a binary classification stagewhere the response is determined to be originating from thetarget physiological biomarker, in this case CCK, or not. Basedon the outcome of this classification, the next level consists ofmulti-level classification where the concentration of hormoneis determined based on response characteristics.Previous papers have explored modelling the relationshipbetween neurochemical levels and stimulation parameters [6].In this paper, a neural network based classification systemis described while considering various trade-offs and datapreparation techniques. After drift removal, the pre-processedsignal is labelled and used to train a neural network. Thenetwork is then optimised and used to perform classification.The precision and recall results for this network are presented .This paper focusses on primarily methodology for identifyingand enhancing features for noisy neurochemical signals.In this paper, Section II provides a brief introduction intoexisting closed-loop systems, and the functional units of animplant processing unit.II. BACKGROUNDA. Neurochemical Changes in the Vagus NerveThe vagus nerve innervates the subdiaphragmatic regionon the ventral and dorsal gut regions. It links the gut tothe brain and plays a crucial role in appetite control throughinteraction of several gut hormones [7]. It is well-establishedthat cholecystokinin (CCK), secreted in the intestines, playsa crucial role in anorectic effects through the vagus nerve[8], [9]. In the experiments described in this paper, CCK isused to activate signalling in the vagus nerve by intravenousinjection. This results in activation of the vagus nerve resultingin changes in neurochemical pH.B. Closed Loop SystemsA closed-loop system consists of sensors/electrodes fol-lowed by data acquisition, pre-processing, possible dimen-sionality reduction followed by decision and stimulation dosetuning unit. The ultimate goal of a closed-loop system is toaccurately and autonomously answer when? and how much?to stimulate. Closed-loop systems can be divided into differenttypes, depending on functionality and type of control feedback.These are responsive: a partial closed-loop system stimulationa partial closed-loop system where one of the two questions(when?, how much?) is answered on the basis of a singlephysiological biomarker and a predetermined threshold of thisbiomarker, adaptive: where a single biomarker is monitored butboth when? and how much? are answered, complete closed-loop: where multiple biomarkers are monitored and consideredto deliver closed-loop functionality [3], [10].In order to implement autonomous detection, algorithmsneed to be trained using an experimental dataset [11], [5],[3]. The training step is performed on in vivo or in vitrodata. A variety of different algorithms have been used pre-viously [11], [5], [3]. These include machine learning (ML)algorithms such as principal component regression (PCR),which is a combination of principal component analysis andlinear regression. Artificial Neural networks (ANN) have beeninvestigated previously for neurotransmitter release due toelectrical stimulation [11], [6]. ANN were used to model boththe relationship between stimulation-evoked dopamine releaseand vice-versa [6].C. CNN vs RNNFor timeseries classification such as the one described in thispaper, has been shown to be more suitable for ConvolutionalNeural Network (CNN) as compared to Recursive NeuralNetwork (RNN). It is dependant on a number of factors suchas size of pre-processing window.Furthermore, CNNs train about 20 times faster than RNN.On a batch size 64, RNNs take on an average of 1 ms whileCNNs take an average of 200 µs A hybrid of CNN andRNN was also tried. Even though a small improvement inclassification was observed, it was slower to test and train.Hence, RNNs were not considered an optimal solution.D. Processing UnitA closed-loop implant comprises of a processing unit whichdetermines when and how much to perform neural stimulation.The processing unit consists of a pre-processing unit whichhelps in removing data, followed by data labelling, dimension-ality reduction and classification. This paper primarily focusseson classification method based on low latency and relativelylow resource deep neural network (DNN).III. METHODOLOGYThis section describes the methodology for training andvalidating a CNN for classification of neurochemical data.It consists of data preparation, labelling and optimisation fortraining and validation of the CNN. The experimental dataconsisted of neurochemical pH recorded in vivo consistingof one saline injection and four CCK injections of differentdosages (50, 100, 300, 1000 pmol/kg) [12].A. Data PreparationThe raw neurochemical data consisted of biomarker inducedneurochemical changes, background neurochemical activityand electrode drift. The raw data is pre-processed to removedrift using the technique described in [4]. After drift removal,the outliers were identified and removed. For outlier removala modified z-score process was used, first described in [13](Figure III-A). This was used to remove biased scaling due tovery large data values.After outlier removal, the data was partitioned into temporalwindows of 15 seconds with an overlap of about 5 seconds.Following this the data was normalised to be such that itsvalue was between -1 to 0. The normalisation was done usingthe following formula.ynorm =yi − yminymax − ymin (1)The data was then labelled either 1 or 0 depending onwhether the datapoint was part of CCK-induced neurochemicalresponse or not, respectively (See Figure III-A).B. Class size ImbalanceClass imbalance is a common issue in ML-based classifica-tion problems, where the objects belonging to one class exceedthe number of objects in other classes. This problem can besolved through either creation or oversampling of minorityclass objects or undersampling of the majority class. In thispaper, the number of datapoints which are not part of theCCK-induced response far outnumber those which originatefrom the CCK-induced response. Hence, we use SyntheticMinority Oversampling Technique (SMOTE) to deal with thisissue [14]. In this case, SMOTE was applied to the data, post-windowing, such that there are equal number of positives(1)and negatives(0) in the training dataset.Fig. 2. Different steps used to train and cross-validate a CNN for detection and classification of neurochemical response.Fig. 3. (A)The drift in raw sensor data was removed using a techniquedescribed earlier ([4]). The outliers were removed using the modified z-scoreprocess [].C. Model Training and ValidationA training data set of N experiments were prepared andprovided to models generated using Keras 2.2.4 with Ten-sorFlow 1.13.1 as a backend [15]. The model was providedwith a training data tensor of shape (Nwindows, Lwindows, 1),where Nwindows is the number of training windows, Lwindowsis the length of each window. After a model architecture wasselected, batches of training data were used to train the model.The batch size is used to determine the number of times perepoch, the weights are updated. After one complete run oftraining a model i.e epoch, the model was cross-validated ona cross-validation dataset. If the model performed better thanthe previous one, then the model was saved to be used later.In order to prevent plateauing of model performance, alearning rate scheduler was used. Models in which the per-formance did not improve after 10 epochs had their learningrate halved. Models which did not improve after 40 epochswere discarded to reduce training time.D. Model Architecture and Hyperparameter OptimisationAfter completion of training and validation, objective eval-uation of different model hyperparameters and architectureswas validated with the same validation experiments using k-fold cross validation. Various model architectures, activationfunctions were tried ranging from dense neural networksto convolutional and recurrent neural networks were imple-mented using open-source Neural Network Intelligence (NNI),Microsoft. Smaller changes were made further to manuallyimprove model performance.TABLE IMODEL PARAMETERS OF CNN USED IN THIS PAPERParameters ValuesConvolutional Layers 4Channels per layer 16Kernels per Channel 3Stride Length 1Dropout Fraction 0.2Activation Function ReLUOptimisation ADAMBatch Size 64IV. RESULTSA number of different parameters were tried to achieveoptimal results with regards to training time and correctclassification results.A. Network Parameters: Batch Size, Window Length, DropoutRate1) Batch Size: There is a trade-off involved in the selectionof batch size. If a large batch size is chosen i.e updatingweights happens less frequently per epoch has less perfor-mance overhead compared to a small batch size. The impactof different batch sizes is seen in Figure 4. Figure 4 shows themodel performance (validation loss) vs training time for eachbatch size. A batch size of 64 indicates optimal performancewith fast training time and good model performance.2) Window Length: The length of training windows werevaried to determine an optimal window size. This is significantbecause it will help determine how early a CCK response isdetected. A window length of 10 seconds with an overlapof 8 seconds is found to be optimal. Shorter time windowsdoes not allow enough temporal features to be captured, henceFig. 4. An optimal batch side is chosen based on optimal validation lossvalues.model performance drops. Longer windows results in excessdata bleeding in from the non-CCK areas.3) Dropout rate: Dropout is a tool used for regularisationby randomly deactivating a proportion of neurons in a modelduring each training step [16]. This is performed to preventoverfitting and also to deny neuron from specialising heavilyon specific features. This also means readjustment of neuronweights as neurons with connections to dropped-out neuronswill have to rely heavily on existing neurons. The effect onvalidation loss due to dropout is shown in Figure 5. A dropoutof 20% was chosen as an optimal value.Fig. 5. The effect of different dropout rates show the effect on validationloss (A) and training loss (B). This is used to determine an optimal value fordroput rate while training the model.B. Overall Performance and Improvements : Precision andRecallThe overall performance of the final model (parametersshown in Table I) indicates an accuracy of 93.27% (fractionof correct predictions compared to total predictions). Thiswas achieved with a minimum validation loss of 0.186. Theoptimisation algorithm used for the CNN was ADAM. Asample of prediction accuracy of the model is shown in Figure6 and overall results shown in Table II. Table II indicatesperformance of the model on incoming real-time data whereclass-size imbalance negatively affects the model performance.TABLE IIPRECISION AND RECALL RESULTS ON ENTIRE CROSS-VALIDATIONDATASETPredicted Label Non CCK Event CCK EventTarget LabelNon CCK Event 23010 (TN) 204 (FP)CCK Event 1151 (FN) 502 (TP)Fig. 6. (A) A cross-validation, experimental data set is labelled as shown.(B) Shows the datapoints predicted by the CNN model.V. CONCLUSIONThis paper describes data pre-processing steps, feature en-hancement, development and optimisation of a convolutionalneural network model for detecting neurochemical changesin a real-time application. The advantage of using a CNN isthat it ensures early, more accurate detection of neurochemicalchanges. Although the model successfully detects neurochem-ical changes, however the accuracy is affected by class-sizeimbalance for real-time detection. Hence, there is a need eitherfor better quality training data or additional feature informationwhich can further improve the accuracy of the model.ACKNOWLEDGMENTThis work was supported by the European Research Coun-cil (Synergy Grant No. 319818, i2MOVE), UK EPSRCgrant EP/N002474/1, and EPSRC-Imperial Impact Accelera-tion Award (2019).REFERENCES[1] B. Rosin, M. Slovik, R. Mitelman, M. Rivlin-Etzion, S. N. Haber, Z. Is-rael, E. Vaadia, and H. Bergman, “Closed-loop deep brain stimulationis superior in ameliorating parkinsonism,” Neuron, vol. 72, no. 2, pp.370–384, 2011.[2] F. T. Sun and M. J. Morrell, “Closed-loop Neurostimulation: TheClinical Experience,” Neurotherapeutics, vol. 11, no. 3, pp. 553–563,2014.[3] K. B. Mirza, N. Kulasekeram, Y. Liu, K. Nikolic, and C. Toumazou,“System on chip for closed loop neuromodulation based on dual modebiosignals,” Proceedings - IEEE International Symposium on Circuitsand Systems, vol. 2019-May, 2019.[4] T. Ahmed, K. B. Mirza, and K. Nikolic, “Resource Efficient Pre-processor for Drift Removal in Neurochemical Signals,” in IEEE In-ternational Symposium on Circuits and Systems, 2018, pp. 1–5.[5] B. Bozorgzadeh, D. P. Covey, B. A. Heidenreich, P. A. Garris, andP. Mohseni, “Real-time processing of fast-scan cyclic voltammetry(FSCV) data using a field-programmable gate array (FPGA),” Confer-ence proceedings : ... Annual International Conference of the IEEEEngineering in Medicine and Biology Society. IEEE Engineering inMedicine and Biology Society. Annual Conference, vol. 2014, pp. 2036–2039, 2014.[6] P. J. Grahn, G. W. Mallory, O. U. Khurram, B. M. Berry, J. T. Hachmann,A. J. Bieber, K. E. Bennet, H.-k. Min, S.-y. Chang, K. H. Lee, andJ. L. Lujan, “A neurochemical closed-loop controller for deep brainstimulation : toward individualized smart neuromodulation therapies,”vol. 8, no. June, pp. 1–11, 2014.[7] S. C. Cork, “The role of the vagus nerve in appetite control: Implicationsfor the pathogenesis of obesity,” Journal of Neuroendocrinology, vol. 30,no. 11, pp. 1–10, 2018.[8] S. Okano-Matsumoto, J. A. McRoberts, Y. Tache´, andD. W. Adelson, “Electrophysiological evidence for distinctvagal pathways mediating CCK-evoked motor effects in theproximal versus distal stomach.” The Journal of physiology,vol. 589, no. Pt 2, pp. 371–93, 2011. [Online]. Available:http://doi.wiley.com/10.1113/jphysiol.2010.196832%5Cnhttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3043539&tool=pmcentrez&rendertype=abstract[9] K. B. Mirza, A. Alenda, A. Eftekhar, N. Grossman, K. Nikolic, andR. Stephen, “INFLUENCE OF CHOLECYSTOKININ-8 ON COM-POUND NERVE ACTION POTENTIALS FROM VENTRAL GAS-TRIC VAGUS IN RATS,” International Journal of Neural Systems, vol.Accepted, no. Accepted, pp. 1–15, 2018.[10] K. B. Mirza, K. Wildner, N. Kulasekeram, S. C. Cork, S. R. Bloom,K. Nikolic, and C. Toumazou, “Live Demo: Platform for Closed LoopNeuromodulation Based on Dual Mode Biosignals,” in IEEE 2017Biomedical Circuits and Systems Conference, BioCAS 2017 - Proceed-ings, Milan, 2017.[11] J. K. Trevathan, A. Yousefi, H. O. Park, J. J. Bartoletta, K. A. Ludwig,K. H. Lee, and J. L. Lujan, “Computational Modeling of Neurotrans-mitter Release Evoked by Electrical Stimulation: Nonlinear Approachesto Predicting Stimulation-Evoked Dopamine Release,” ACS ChemicalNeuroscience, vol. 18, no. 3, pp. 386–392, 2017.[12] S. C. Cork, A. Eftekhar, K. B. Mirza, C. Zuliani, K. Nikolic, J. V.Gardiner, S. R. Bloom, and C. Toumazou, “Extracellular pH monitoringfor use in closed-loop vagus nerve stimulation,” Journal of NeuralEngineering, vol. 15, no. 1, p. 016001, feb 2018.[13] A. Schmidt and Z. Bandar, “A modular neural network architecture withadditional generalization abilities for large input vectors,” in ArtificialNeural Nets and Genetic Algorithms. Springer, 1998, pp. 35–39.[14] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “Smote:synthetic minority over-sampling technique,” Journal of artificial intel-ligence research, vol. 16, pp. 321–357, 2002.[15] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S.Corrado, A. Davis, J. Dean, M. Devin et al., “Tensorflow: Large-scalemachine learning on heterogeneous distributed systems,” arXiv preprintarXiv:1603.04467, 2016.[16] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut-dinov, “Dropout: a simple way to prevent neural networks from over-fitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp.1929–1958, 2014.

Convolutional neural network for classification of nerve activity based on action potential induced neurochemical signatures

http://repository.uwl.ac.uk/id/eprint/7353/1/3_ISCAS-final.pdf

Convolutional neural network for classification of nerve activity based on action potential induced neurochemical Signatures

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository

Crossref

UWL Repository