Serious Mental Illnesses (SMIs) including schizophrenia and bipolar disorder are long term conditions which place major burdens on health and social care services. Locomotor activity is altered in many cases of SMI, and so in the long term wearable activity trackers could potentially aid in the early detection of SMI relapse, allowing early and targeted intervention. To move towards this goal, in this paper we use accelerometer activity tracking data collected from the UK Biobank to classify people as being either in a self-reported SMI group or an age and gender matched control group. Using an ensemble dense neural network algorithm we exploited hourly and average derived features from the wearable activity data and the created model obtained an accuracy of 91.3%

Casson, Alexander J.

Peek, Niels

Zebin, Tahmina

English

Casson, Alex

The University of Manchester - Institutional Repository

The University of Manchester ResearchPhysical activity based classification of serious mentalillness group participants in the UK Biobank usingensemble dense neural networksDOI:10.1109/EMBC.2019.8857532Document VersionAccepted author manuscriptLink to publication record in Manchester Research ExplorerCitation for published version (APA):Zebin, T., Peek, N., & Casson, A. (2019). Physical activity based classification of serious mental illness groupparticipants in the UK Biobank using ensemble dense neural networks. In IEEE EMBC 2019https://doi.org/10.1109/EMBC.2019.8857532Published in:IEEE EMBC 2019Citing this paperPlease note that where the full-text provided on Manchester Research Explorer is the Author Accepted Manuscriptor Proof version this may differ from the final Published version. If citing, it is advised that you check and use thepublisher's definitive version.General rightsCopyright and moral rights for the publications made accessible in the Research Explorer are retained by theauthors and/or other copyright owners and it is a condition of accessing publications that users recognise andabide by the legal requirements associated with these rights.Takedown policyIf you believe that this document breaches copyright please refer to the University of Manchester’s TakedownProcedures [http://man.ac.uk/04Y6Bo] or contact uml.scholarlycommunications@manchester.ac.uk providingrelevant details, so we can investigate your claim.Download date:21. Oct. 2022Physical activity based classification of serious mental illness groupparticipants in the UK Biobank using ensemble dense neural networksTahmina Zebin, Niels Peek and Alexander J. Casson, Senior Member, IEEEAbstract— Serious Mental Illnesses (SMIs) includingschizophrenia and bipolar disorder are long term conditionswhich place major burdens on health and social care services.Locomotor activity is altered in many cases of SMI, and soin the long term wearable activity trackers could potentiallyaid in the early detection of SMI relapse, allowing earlyand targeted intervention. To move towards this goal, in thispaper we use accelerometer activity tracking data collectedfrom the UK Biobank to classify people as being either in aself-reported SMI group or an age and gender matched controlgroup. Using an ensemble dense neural network algorithmwe exploited hourly and average derived features from thewearable activity data and the created model obtained anaccuracy of 91.3%.I. INTRODUCTIONSerious Mental Illnesses (SMIs) including schizophreniaand bipolar disorder are long term conditions which placemajor burdens on health and social care services, oftencoupled with significant reduction in quality of life for theindividual over a long term. Up to 40% of those dischargedafter hospital treatment for schizophrenia may relapse within1 year [1].To help reduce this rate there has long been an interestin creating smartphone based mHealth tools for helpingwith the management of SMIs. An example is ClinTouch,an Ecological Momentary Assessment (EMA) smartphoneapp [2], [3] which prompts the user to answer a structured setof questions multiple times per day about their thoughts andfeelings. Response histories are available within the app andare also uploaded to a central server for remote monitoringby healthcare professionals, allowing potential changes inbehaviour to be detected, and care planned and targeted asa result.Today, non-invasive body-worn sensors such as accelerom-eters and gyroscopes can also help to measure a wide rangeof behavioral factors to inform this process. Indeed, impairedmotor function is often found during episodes of SMI [4].For example, people with bipolar disorder or schizophreniacan be significantly more sedentary than age- and gender-matched healthy controls [5]. Further examples of the useof such passive sensing for behavioural monitoring areThis work was supported by the UK Engineering and Physical SciencesResearch Council grant number EP/P010148/1.This work was conducted at the University of Manchester and TahminaZebin is currently working with the School of Computing Sciences, The Uni-versity of East Anglia, Norwich, UK. Email: t.zebin@uea.ac.uk.A. J. Casson is with the School of Electrical and Electronic En-gineering, and N. Peek is with the Health e-Research Centre, TheUniversity of Manchester, Manchester, UK. Email: {alex.casson,niels.peek}@manchester.ac.uk.presented in [4], [5] which indicate that passively collectedbehavioral data, using wearables, presents a potentially scal-able and at present underutilized opportunity to help with thecare of people with SMI.However, long term wearable monitoring has only recentlybecome available and prospective controlled trials of wear-ables with people with SMI can be very time consuming toperform, as participants go through different periods of re-lapse and remission, and the wanted state may not be presentduring the monitoring period. In this paper we investigatemaking use of activity tracking (accelerometer) data from theUK Biobank [6], giving a large cross-sectional population toinvestigate. Within this large dataset, 63 participants self-identified as having an SMI, and we investigate whethermachine learning can be applied to the activity data fromthis population to classify the individuals separately froma matched control population also selected from the UKBiobank. Our classification scheme is based upon an ensem-ble of dense neural networks to obtain better performancecompared to using a single dense neural network.The rest of the paper is organized as follows. Section IIintroduces the UK Biobank data selection procedure for theclassification scenario of this paper. Section III presents ourproposed ensemble dense neural network learning architec-ture, with results presented and discussed in Section IV.Finally conclusions are drawn in Section V.II. UK BIOBANK DATA SELECTIONA. UK Biobank overviewThe UK Biobank is a very large scale open ac-cess database of a wide range genomic and bio-signalrecords, coming from more than 500,000 people forsome data fields. The public showcase of the data avail-able is online at http://biobank.ctsu.ox.ac.uk/showcase/label.cgi. In this paper we make use ofthe physical activity dataset (obtained through UK biobankapplication number 33693), which was collected from ap-proximately 100,000 people wearing an Activity AX3 3-axis accelerometer device [8] for a week. During this timeparticipants undertook their normal daily activities and theraw data is unlabelled. All data was sampled at 100 Hz, andboth the raw data (in units of g) and a number of derivedfeatures (as described below) are available.Also available is a web based questionnaire for theseparticipants, giving information regarding the presence andabsence of any mental health condition, including presentand past depression, bipolar affective disorder, generalizedanxiety disorder, harm behaviour, and subjective wellbeing.Fig. 1. Illustration of daily activity pattern for one participant. Images generated with toolbox provided by [7]. The colour code identifies moderateactivities, sleep, walking and sedentary patterns. (a) Participant from SMI group. (b) Participant from control group.Fig. 2. List of the accelerometer features used in this study and theassociated UK Biobank data fields.The outcomes of these questions are encoded in the Biobank(data field ID 21054) in sixteen fields and allows us to use theraw unlabelled activity dataset in a supervised classificationexperiment to group participants between SMI and controlgroups.B. Accelerometry featuresAn example of the accelerometery signals available in thedataset, processed using the open source toolbox from [7]to threshold the amount of activity into sedentary, moderate,and similar levels of activity, is shown in Fig. 1. In additionto the raw data a number of derived features based uponaveraging are available including: daily acceleration averages(7 measurements); per hour average acceleration (24 mea-surements); no-wear time (5 different metrics); categoricallyencoded age and gender (5 different metrics); overall weeklyacceleration average and standard deviation (2 features). Thelist of all of the features we used, and the associated UKBiobank data fields, is given in Fig. 2. As different featureshave different value ranges we have used a StandardScalernormalization applied to the raw data to avoid any potentialmodel bias due to larger values in any individual feature.C. SMI group and control group identificationAs reported in [9], there are 137 participants who self-indicated any form of mental health condition. Here wemake use of the responses to questionnaire A3 and A4 (UKbiobank data field 20544-0.1 and 20544-0.2) for which 63participants (21 female, 42 male) were identified as havingan SMI, and use this cohort as our SMI group.For the control group, potentially a very large number ofparticipants could be used (all of the remaining participants)but this is computationally impractical for the current work,and it is likely that some participants with undisclosed orun-diagnosed SMI would be present in such a control group.Moreover, it would give an extremely imbalanced data set forthe machine learning analysis. Here we overcome this byusing a one-sided Selection with synthetic Minority Over-sampling Technique (SMOTE) technique [10] to resamplethe data present. We randomly selected an age and gendermatched group of 200 participants from the Biobank to formthe control group (86 female, 114 male), giving a dataset ofa good size, which is also of a practical size to work withfor initial machine learning development.III. ACTIVITY BASED SMI CLASSIFICATIONIn this paper, we utilised an ensemble dense neuralnetwork architecture to process the hourly accelerometerfeatures available and classify participants as being in eitherthe SMI or control group. We opted for a Deep NeuralNetwork (DNN) architecture due to its capability to processraw time-series data, large error handling capability, andpotential for high model accuracy [11].Our model is implemented using the Keras open sourcelibrary in Python [12], and we have utilized the sequen-tial model and the dense, dropout, concetenate, and batchnormalization layers. At the very first layer, we fed the 43features from Fig. 2: the thirty-eight derived accelerometerfeatures, one-hot encoded gender (2 channels), and agefeatures (3 channels). These features create a non-linear,Fig. 3. Visualization of the proposed ensemble Dense Neural Network(DNN) model. The model handles a class imbalance using a syntheticminority oversampling technique.distributed representation of the input. A visualisation of theproposed network is given in Fig. 3.Ensemble methods are commonly used for achievingbetter performance compared to single models. As can beseen from Fig. 3, we started with three baseline models toobtain initial feature processing and classification. In eachof these networks there were four dense layers to learn fromthe features according to their priority. These baseline meta-classifiers were identical in nature for feature explorationpurposes, but fed with separate balanced training sets. As themajority (control) group instances (sample size = 200) werealmost three times than the minority SMI group (sample size= 63) we partitioned the the majority group into three equalparts to avoid the majority bias on the ensemble layers. Wethen applied the SMOTE technique on the minority group tofurther equalize the class imbalance. A balanced test set of 80participants containing both the groups was kept separatelyfor the purpose of model evaluation.After the three baseline classifiers a three layer denseneural network is employed (seen at the bottom of Fig. 3),trained by using the concatenated output of the three en-semble DNNs as inputs. The output layer uses a softmaxactivation, and all layers are fine-tuned using a holdoutdataset. The final model was trained with an Adam optimizerwith a learning rate of 0.002 and a binary cross-entropy. Thetraining of the model has been done offline on a PC with2.4 GHz Intel core i7 processor and 32 GB memory.To act as comparison cases, in Section IV we report theperformance of both the ensemble DNN, and a single (non-ensemble) DNN which is identical in set up to the above andtrained on the unbalanced training data set. We also report theperformance of a standard random forest classifier applied tothe data, implemented from the python sklearn library.Fig. 4. Comparative pair-plot for weekday and weekend average acceler-ation for the control (blue) and SMI (green) participants.Fig. 5. Top fifteen discriminatory features for this classification task(presented in an ascending order). f# indicates the feature number accordingto our internal numbering scheme.IV. RESULTS AND DISCUSSIONTo illustrate the two groupings, we present a comparativepair-plot for weekday and weekend average acceleration forthe control and SMI group participants in Fig. 4. The controlgroup is seen to have greater average daily acceleration thanthe SMI group. We have also provided an estimate of featureimportance for the classification task in hand in Fig. 5. Thishas identified age, gender, and some of the hourly averageacceleration features during the morning, evening and nighthours to be highly important in separating the two groups.Confusion matrices for the single DNN model and ensem-ble DNN model are given in Fig. 6. It can be seen that theensemble DNN substantially outperforms the single DNNwith an overall accuracy of 91.3% compared to 83.8%. Thenumber of records correctly classified into the control stateis in fact the same between the two DNN set ups, withthe ensemble DNN getting better performance due to moreparticipants being correctly classified into the SMI class.A comparison to the performance of a random forestclassifier is given in Table I. The random forest modeloutperformed the single DNN in overall performance, but notthe ensemble DNN. In addition, the random forest had muchTABLE IQUANTITATIVE COMPARISON OF ENSEMBLE DNN WITH TRADITIONAL SINGLE DNN AND A RANDOM FOREST CLASSIFIER.Classifier Overall accuracy Recall (SMI class) Recall (Control class) Precision (SMI class) Precision (Control class)Random forest 85.2% 67.1% 87.0% 64.2% 83.0%Single DNN 83.8% 85.3% 82.4% 84.4% 78.8%Ensemble DNN 91.3% 89.7% 92.7% 92.1% 90.5%Fig. 6. Class-wise confusion matrix of the test dataset for the (a) non-ensemble and (b) ensemble DNN.lower recall in the SMI class than either of the DNN basedapproaches. In terms of model execution time, the ensemble-DNN model took longer (10.9 seconds) in the training phasethan the single DNN (3.41 seconds) and random forest (9.66seconds).These high classification performances show that it ispotentially possible to differentiate between the two classesbased upon a week of activity data. This acts as a startingpoint to show that there is separability between the twoclasses, although we note that our control group was onlymatched for age and gender. Differences in socio-economicstatus and disease co-morbidity may be present which couldbe the underlying source of the separability reported here andshould be taken as a limitation of the current work. Futureinvestigations will investigate whether the accelerometry datasampled at different points in time could be used to givean indicator of potential relapse or remission events, withthe ultimate aim of using wearables to help with earlierinterventions.V. CONCLUSIONSIn this work we applied a data-driven ensemble dense neu-ral network technique aimed to separately classify a seriousmental illness participant group from a control group usingtheir physical activity data available from the UK Biobank.The results show that the two classes were potentiallyseparable based upon accelerometer activity recorded overa week. Using an ensemble classifier approach substantiallyimproved the performance compared to using a single denseneural network.REFERENCES[1] I. Barnett, J. Torous, P. Staples, et al., “Relapse prediction inschizophrenia through digital phenotyping: A pilot study,”Neuropsychopharmacology, vol. 43, no. 8, pp. 1660–1666,2018.[2] S. Shiffman, A. A. Stone, and M. R. Hufford, “Ecologicalmomentary assessment,” Annu. Rev. Clin. Psychol., vol. 4,no. 1, pp. 1–32, 2008.[3] P. Whelan, M. Machin, S. Lewis, et al., “Mobile earlydetection and connected intervention to coproduce bettercare in severe mental illness,” Stud. Health Technol. Inform.,vol. 216, no. 1, pp. 123–126, 2015.[4] E. Reinertsen and G. D. Clifford, “A review of physiologicaland behavioral monitoring with digital sensors for neuropsy-chiatric illnesses,” Physiol. Meas., vol. 39, no. 5, 05TR01,2018.[5] D. Vancampfort, J. Firth, F. B. Schuch, et al., “Seden-tary behavior and physical activity levels in people withschizophrenia, bipolar disorder and major depressive disor-der: A global systematic review and meta-analysis,” WorldPsychiatry, vol. 16, no. 3, pp. 308–315, 2017.[6] C. Sudlow, J. Gallacher, N. Allen, et al., “UK biobank: Anopen access resource for identifying the causes of a widerange of complex diseases of middle and old age,” PLoSMed., vol. 12, no. 3, e1001779, 2015.[7] A. Doherty, D. Jackson, N. Hammerla, et al., “Large scalepopulation assessment of physical activity using wrist wornaccelerometers: The UK biobank study,” PLoS One, vol. 12,no. 2, e0169649, 2017.[8] Axivity. (2019). AX3 3-axis logging accelerometer, [On-line]. Available: https://axivity.com/product/ax3/.[9] K. A. S. Davis, J. R. I. Coleman, M. Adams, et al., “Mentalhealth in UK biobank: Development, implementation andresults from an online questionnaire completed by 157 366participants,” BJPsych Open, vol. 4, no. 3, pp. 83–90, 2018.[10] I. H. Witten, E. Frank, M. A. Hall, et al., Data Mining:Practical machine learning tools and techniques. Burling-ton: Morgan Kaufmann, 2016.[11] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”Nature, vol. 521, no. 7553, p. 436, 2015.[12] F. Chollet. (2013). Keras: The Python Deep Learning library,[Online]. Available: https://keras.io/.

Physical activity based classification of serious mental illness group participants in the UK Biobank using ensemble dense neural networks

Crossref

University of East Anglia digital repository

Physical activity based classification of serious mental illness groupparticipants in the UK Biobank using ensemble dense neural networksTahmina Zebin, Niels Peek and Alexander J. Casson, Senior Member, IEEEAbstract— Serious Mental Illnesses (SMIs) includingschizophrenia and bipolar disorder are long term conditionswhich place major burdens on health and social care services.Locomotor activity is altered in many cases of SMI, and soin the long term wearable activity trackers could potentiallyaid in the early detection of SMI relapse, allowing earlyand targeted intervention. To move towards this goal, in thispaper we use accelerometer activity tracking data collectedfrom the UK Biobank to classify people as being either in aself-reported SMI group or an age and gender matched controlgroup. Using an ensemble dense neural network algorithmwe exploited hourly and average derived features from thewearable activity data and the created model obtained anaccuracy of 91.3%.I. INTRODUCTIONSerious Mental Illnesses (SMIs) including schizophreniaand bipolar disorder are long term conditions which placemajor burdens on health and social care services, oftencoupled with significant reduction in quality of life for theindividual over a long term. Up to 40% of those dischargedafter hospital treatment for schizophrenia may relapse within1 year [1].To help reduce this rate there has long been an interestin creating smartphone based mHealth tools for helpingwith the management of SMIs. An example is ClinTouch,an Ecological Momentary Assessment (EMA) smartphoneapp [2], [3] which prompts the user to answer a structured setof questions multiple times per day about their thoughts andfeelings. Response histories are available within the app andare also uploaded to a central server for remote monitoringby healthcare professionals, allowing potential changes inbehaviour to be detected, and care planned and targeted asa result.Today, non-invasive body-worn sensors such as accelerom-eters and gyroscopes can also help to measure a wide rangeof behavioral factors to inform this process. Indeed, impairedmotor function is often found during episodes of SMI [4].For example, people with bipolar disorder or schizophreniacan be significantly more sedentary than age- and gender-matched healthy controls [5]. Further examples of the useof such passive sensing for behavioural monitoring areThis work was supported by the UK Engineering and Physical SciencesResearch Council grant number EP/P010148/1.This work was conducted at the University of Manchester and TahminaZebin is currently working with the School of Computing Sciences, The Uni-versity of East Anglia, Norwich, UK. Email: t.zebin@uea.ac.uk.A. J. Casson is with the School of Electrical and Electronic En-gineering, and N. Peek is with the Health e-Research Centre, TheUniversity of Manchester, Manchester, UK. Email: {alex.casson,niels.peek}@manchester.ac.uk.presented in [4], [5] which indicate that passively collectedbehavioral data, using wearables, presents a potentially scal-able and at present underutilized opportunity to help with thecare of people with SMI.However, long term wearable monitoring has only recentlybecome available and prospective controlled trials of wear-ables with people with SMI can be very time consuming toperform, as participants go through different periods of re-lapse and remission, and the wanted state may not be presentduring the monitoring period. In this paper we investigatemaking use of activity tracking (accelerometer) data from theUK Biobank [6], giving a large cross-sectional population toinvestigate. Within this large dataset, 63 participants self-identified as having an SMI, and we investigate whethermachine learning can be applied to the activity data fromthis population to classify the individuals separately froma matched control population also selected from the UKBiobank. Our classification scheme is based upon an ensem-ble of dense neural networks to obtain better performancecompared to using a single dense neural network.The rest of the paper is organized as follows. Section IIintroduces the UK Biobank data selection procedure for theclassification scenario of this paper. Section III presents ourproposed ensemble dense neural network learning architec-ture, with results presented and discussed in Section IV.Finally conclusions are drawn in Section V.II. UK BIOBANK DATA SELECTIONA. UK Biobank overviewThe UK Biobank is a very large scale open ac-cess database of a wide range genomic and bio-signalrecords, coming from more than 500,000 people forsome data fields. The public showcase of the data avail-able is online at http://biobank.ctsu.ox.ac.uk/showcase/label.cgi. In this paper we make use ofthe physical activity dataset (obtained through UK biobankapplication number 33693), which was collected from ap-proximately 100,000 people wearing an Activity AX3 3-axis accelerometer device [8] for a week. During this timeparticipants undertook their normal daily activities and theraw data is unlabelled. All data was sampled at 100 Hz, andboth the raw data (in units of g) and a number of derivedfeatures (as described below) are available.Also available is a web based questionnaire for theseparticipants, giving information regarding the presence andabsence of any mental health condition, including presentand past depression, bipolar affective disorder, generalizedanxiety disorder, harm behaviour, and subjective wellbeing.Fig. 1. Illustration of daily activity pattern for one participant. Images generated with toolbox provided by [7]. The colour code identifies moderateactivities, sleep, walking and sedentary patterns. (a) Participant from SMI group. (b) Participant from control group.Fig. 2. List of the accelerometer features used in this study and theassociated UK Biobank data fields.The outcomes of these questions are encoded in the Biobank(data field ID 21054) in sixteen fields and allows us to use theraw unlabelled activity dataset in a supervised classificationexperiment to group participants between SMI and controlgroups.B. Accelerometry featuresAn example of the accelerometery signals available in thedataset, processed using the open source toolbox from [7]to threshold the amount of activity into sedentary, moderate,and similar levels of activity, is shown in Fig. 1. In additionto the raw data a number of derived features based uponaveraging are available including: daily acceleration averages(7 measurements); per hour average acceleration (24 mea-surements); no-wear time (5 different metrics); categoricallyencoded age and gender (5 different metrics); overall weeklyacceleration average and standard deviation (2 features). Thelist of all of the features we used, and the associated UKBiobank data fields, is given in Fig. 2. As different featureshave different value ranges we have used a StandardScalernormalization applied to the raw data to avoid any potentialmodel bias due to larger values in any individual feature.C. SMI group and control group identificationAs reported in [9], there are 137 participants who self-indicated any form of mental health condition. Here wemake use of the responses to questionnaire A3 and A4 (UKbiobank data field 20544-0.1 and 20544-0.2) for which 63participants (21 female, 42 male) were identified as havingan SMI, and use this cohort as our SMI group.For the control group, potentially a very large number ofparticipants could be used (all of the remaining participants)but this is computationally impractical for the current work,and it is likely that some participants with undisclosed orun-diagnosed SMI would be present in such a control group.Moreover, it would give an extremely imbalanced data set forthe machine learning analysis. Here we overcome this byusing a one-sided Selection with synthetic Minority Over-sampling Technique (SMOTE) technique [10] to resamplethe data present. We randomly selected an age and gendermatched group of 200 participants from the Biobank to formthe control group (86 female, 114 male), giving a dataset ofa good size, which is also of a practical size to work withfor initial machine learning development.III. ACTIVITY BASED SMI CLASSIFICATIONIn this paper, we utilised an ensemble dense neuralnetwork architecture to process the hourly accelerometerfeatures available and classify participants as being in eitherthe SMI or control group. We opted for a Deep NeuralNetwork (DNN) architecture due to its capability to processraw time-series data, large error handling capability, andpotential for high model accuracy [11].Our model is implemented using the Keras open sourcelibrary in Python [12], and we have utilized the sequen-tial model and the dense, dropout, concetenate, and batchnormalization layers. At the very first layer, we fed the 43features from Fig. 2: the thirty-eight derived accelerometerfeatures, one-hot encoded gender (2 channels), and agefeatures (3 channels). These features create a non-linear,Fig. 3. Visualization of the proposed ensemble Dense Neural Network(DNN) model. The model handles a class imbalance using a syntheticminority oversampling technique.distributed representation of the input. A visualisation of theproposed network is given in Fig. 3.Ensemble methods are commonly used for achievingbetter performance compared to single models. As can beseen from Fig. 3, we started with three baseline models toobtain initial feature processing and classification. In eachof these networks there were four dense layers to learn fromthe features according to their priority. These baseline meta-classifiers were identical in nature for feature explorationpurposes, but fed with separate balanced training sets. As themajority (control) group instances (sample size = 200) werealmost three times than the minority SMI group (sample size= 63) we partitioned the the majority group into three equalparts to avoid the majority bias on the ensemble layers. Wethen applied the SMOTE technique on the minority group tofurther equalize the class imbalance. A balanced test set of 80participants containing both the groups was kept separatelyfor the purpose of model evaluation.After the three baseline classifiers a three layer denseneural network is employed (seen at the bottom of Fig. 3),trained by using the concatenated output of the three en-semble DNNs as inputs. The output layer uses a softmaxactivation, and all layers are fine-tuned using a holdoutdataset. The final model was trained with an Adam optimizerwith a learning rate of 0.002 and a binary cross-entropy. Thetraining of the model has been done offline on a PC with2.4 GHz Intel core i7 processor and 32 GB memory.To act as comparison cases, in Section IV we report theperformance of both the ensemble DNN, and a single (non-ensemble) DNN which is identical in set up to the above andtrained on the unbalanced training data set. We also report theperformance of a standard random forest classifier applied tothe data, implemented from the python sklearn library.Fig. 4. Comparative pair-plot for weekday and weekend average acceler-ation for the control (blue) and SMI (green) participants.Fig. 5. Top fifteen discriminatory features for this classification task(presented in an ascending order). f# indicates the feature number accordingto our internal numbering scheme.IV. RESULTS AND DISCUSSIONTo illustrate the two groupings, we present a comparativepair-plot for weekday and weekend average acceleration forthe control and SMI group participants in Fig. 4. The controlgroup is seen to have greater average daily acceleration thanthe SMI group. We have also provided an estimate of featureimportance for the classification task in hand in Fig. 5. Thishas identified age, gender, and some of the hourly averageacceleration features during the morning, evening and nighthours to be highly important in separating the two groups.Confusion matrices for the single DNN model and ensem-ble DNN model are given in Fig. 6. It can be seen that theensemble DNN substantially outperforms the single DNNwith an overall accuracy of 91.3% compared to 83.8%. Thenumber of records correctly classified into the control stateis in fact the same between the two DNN set ups, withthe ensemble DNN getting better performance due to moreparticipants being correctly classified into the SMI class.A comparison to the performance of a random forestclassifier is given in Table I. The random forest modeloutperformed the single DNN in overall performance, but notthe ensemble DNN. In addition, the random forest had muchTABLE IQUANTITATIVE COMPARISON OF ENSEMBLE DNN WITH TRADITIONAL SINGLE DNN AND A RANDOM FOREST CLASSIFIER.Classifier Overall accuracy Recall (SMI class) Recall (Control class) Precision (SMI class) Precision (Control class)Random forest 85.2% 67.1% 87.0% 64.2% 83.0%Single DNN 83.8% 85.3% 82.4% 84.4% 78.8%Ensemble DNN 91.3% 89.7% 92.7% 92.1% 90.5%Fig. 6. Class-wise confusion matrix of the test dataset for the (a) non-ensemble and (b) ensemble DNN.lower recall in the SMI class than either of the DNN basedapproaches. In terms of model execution time, the ensemble-DNN model took longer (10.9 seconds) in the training phasethan the single DNN (3.41 seconds) and random forest (9.66seconds).These high classification performances show that it ispotentially possible to differentiate between the two classesbased upon a week of activity data. This acts as a startingpoint to show that there is separability between the twoclasses, although we note that our control group was onlymatched for age and gender. Differences in socio-economicstatus and disease co-morbidity may be present which couldbe the underlying source of the separability reported here andshould be taken as a limitation of the current work. Futureinvestigations will investigate whether the accelerometry datasampled at different points in time could be used to givean indicator of potential relapse or remission events, withthe ultimate aim of using wearables to help with earlierinterventions.V. CONCLUSIONSIn this work we applied a data-driven ensemble dense neu-ral network technique aimed to separately classify a seriousmental illness participant group from a control group usingtheir physical activity data available from the UK Biobank.The results show that the two classes were potentiallyseparable based upon accelerometer activity recorded overa week. Using an ensemble classifier approach substantiallyimproved the performance compared to using a single denseneural network.REFERENCES[1] I. Barnett, J. Torous, P. Staples, et al., “Relapse prediction inschizophrenia through digital phenotyping: A pilot study,”Neuropsychopharmacology, vol. 43, no. 8, pp. 1660–1666,2018.[2] S. Shiffman, A. A. Stone, and M. R. Hufford, “Ecologicalmomentary assessment,” Annu. Rev. Clin. Psychol., vol. 4,no. 1, pp. 1–32, 2008.[3] P. Whelan, M. Machin, S. Lewis, et al., “Mobile earlydetection and connected intervention to coproduce bettercare in severe mental illness,” Stud. Health Technol. Inform.,vol. 216, no. 1, pp. 123–126, 2015.[4] E. Reinertsen and G. D. Clifford, “A review of physiologicaland behavioral monitoring with digital sensors for neuropsy-chiatric illnesses,” Physiol. Meas., vol. 39, no. 5, 05TR01,2018.[5] D. Vancampfort, J. Firth, F. B. Schuch, et al., “Seden-tary behavior and physical activity levels in people withschizophrenia, bipolar disorder and major depressive disor-der: A global systematic review and meta-analysis,” WorldPsychiatry, vol. 16, no. 3, pp. 308–315, 2017.[6] C. Sudlow, J. Gallacher, N. Allen, et al., “UK biobank: Anopen access resource for identifying the causes of a widerange of complex diseases of middle and old age,” PLoSMed., vol. 12, no. 3, e1001779, 2015.[7] A. Doherty, D. Jackson, N. Hammerla, et al., “Large scalepopulation assessment of physical activity using wrist wornaccelerometers: The UK biobank study,” PLoS One, vol. 12,no. 2, e0169649, 2017.[8] Axivity. (2019). AX3 3-axis logging accelerometer, [On-line]. Available: https://axivity.com/product/ax3/.[9] K. A. S. Davis, J. R. I. Coleman, M. Adams, et al., “Mentalhealth in UK biobank: Development, implementation andresults from an online questionnaire completed by 157 366participants,” BJPsych Open, vol. 4, no. 3, pp. 83–90, 2018.[10] I. H. Witten, E. Frank, M. A. Hall, et al., Data Mining:Practical machine learning tools and techniques. Burling-ton: Morgan Kaufmann, 2016.[11] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”Nature, vol. 521, no. 7553, p. 436, 2015.[12] F. Chollet. (2013). Keras: The Python Deep Learning library,[Online]. Available: https://keras.io/.

https://ueaeprints.uea.ac.uk/id/eprint/73013/2/Accepted_Manuscript.pdf

Physical activity based classification of serious mental illness group participants in the UK Biobank using ensemble dense neural networks

Abstract

Similar works

Full text

Available Versions

The University of Manchester - Institutional Repository

Crossref

University of East Anglia digital repository