McConville, Ryan

Santos-Rodriguez, Raul

Twomey, Niall

Internet of Things (IoT) devices with embedded accelerometers continue to grow in popularity. These are often attached to individuals, whether they are a mobile phone in a pocket or a smartwatch on a wrist, and are constantly capturing data of a personal nature. In this work we propose a method for person identification using accelerometer data via supervised machine learning techniques. Further, we introduce the first unsupervised method for discovering individuals using the same accelerometer. We report the performance both in terms of classificationand clustering using a publicly available dataset covering a large number of activities of daily living. While this has numerous benefits in tasks such as activity recognition and biometrics, this work also motivates the debate and discussion around privacy concerns of the analysis of accelerometer data

Explore Bristol Research

                          McConville, R., Santos-Rodriguez, R., & Twomey, N. (2018). PersonIdentification and Discovery With Wrist Worn Accelerometer Data. InProceedings of the European Symposium on Artificial Neural Networks,Computational Intelligence and Machine Learning, 2018 (pp. 615-620)Publisher's PDF, also known as Version of recordLicense (if available):CC BY-SALink to publication record in Explore Bristol ResearchPDF-documentThis is the final published version of the article (version of record). It first appeared online via i6doc.com athttps://www.i6doc.com/en/ . Please refer to any applicable terms of use of the publisher.University of Bristol - Explore Bristol ResearchGeneral rightsThis document is made available in accordance with publisher policies. Please cite only the publishedversion using the reference above. Full terms of use are available:http://www.bristol.ac.uk/pure/about/ebr-termsPerson Identification and Discovery With WristWorn Accelerometer DataRyan McConville, Rau´l Santos-Rodr´ıguez, Niall TwomeyUniversity of Bristol, UKAbstract. Internet of Things (IoT) devices with embedded accelerome-ters continue to grow in popularity. These are often attached to individ-uals, whether they are a mobile phone in a pocket or a smartwatch ona wrist, and are constantly capturing data of a personal nature. In thiswork we propose a method for person identification using accelerometerdata via supervised machine learning techniques. Further, we introducethe first unsupervised method for discovering individuals using the sameaccelerometer. We report the performance both in terms of classificationand clustering using a publicly available dataset covering a large numberof activities of daily living. While this has numerous benefits in tasks suchas activity recognition and biometrics, this work also motivates the debateand discussion around privacy concerns of the analysis of accelerometerdata.1 IntroductionAs the proliferation of Internet of Things (IoT) devices continues, largeramounts of data are generated and stored on a daily basis. Increasingly, thishappens inside homes with data of a personal nature. Wearable devices whichcollect accelerometer data raise a specific type of often neglected privacy ques-tion. Can an on-body accelerometer, such as those typically found in wrist worndevices, e.g. smartwatches, gather data that can uniquely identify the individualthat it was generated from?The motivation behind this work originates from the H2020 EurValve project,which includes an ‘Smart Home in a Box’ strand utilizing a wrist-worn accelerom-eter [1, 2]. A key concern with such systems being remotely deployed and usedby participants themselves is to verify whether it is the actual participant iswearing the device, and not someone else. The potential to influence clinicaldecision making using this data means it is of the utmost importance that thecorrect participant is wearing the wearable throughout the study.Accelerometers provide a rich amount of information; they measure acceler-ation and deceleration in three dimensions; the vertical x, the anteroposterior yand the mediolateral z. These devices are popular and in widespread use dueto their very low cost yet broad applicability to tasks in different domains, fore.g. step counting in health and fitness. However, while other aspects of securityand privacy in IoT have been well studied [3], the use of accelerometer data hasattracted significantly less attention. Although there has been some recent at-tempts of using supervised machine learning methods to identify individuals [4],it appears that accelerometer data is still often overlooked as a personally iden-tifiable data source. We extend existing work by proposing both fully supervisedand unsupervised methods to respectively identify and discover individuals.ESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  615(a) 2 seconds (b) 30 seconds (c) 60 seconds (d) 720 secondsFigure 1: The progression of signal separation as the window size increases.We are the first to show, to the best of our knowledge, that it is possibleto perform unsupervised person discovery, accurately discovering clusters corre-sponding to individuals.2 Person Identification and DiscoveryIt is our hypothesis that each person has unique signature which can beextracted from their acceleration measured with a wrist worn device. In orderto investigate this in a fair manner, we require a dataset with the followingproperties. (a) It should consist of people each carrying out the same activitiesin a controlled environment (i.e. a script). This prevents detecting specificbehaviour patterns rather than a personal signature. (b) It should consist of thesame script being performed again on a different occasion by each participant.The SPHERE challenge dataset [5] is a recent publicly available dataset meetingthis criteria. Since the data is fully labelled, we will naturally first investigatesupervised classification methods. Following this we will explore unsupervisedmethods to extend the framework from identification to discovery.2.1 DatasetFirst, we provide an overview of the SPHERE challenge dataset1 which we usedin order to validate our hypothesis. The data was collected from 10 people ontwo different occasions in a house fitted with numerous sensors. There were 8males and 2 females, with 8 between the ages of 18 to 29 and 2 within the agesof 30 to 39. Each participant was wearing a wrist-worn accelerometer and wasasked to complete a series of scripted activities, taking around 25 to 30 minutesin total. The activities included in the data covered ambulation activities (e.g.walking), posture activities (e.g. standing) and transitional activities (e.g. sit tostand). This script was carried out twice in full by each participant on differentdays. Due to a data recording issue with one participant in one instance, weexclude them from the experiment and are left with 18 different sets of scriptedactivities, each of roughly 25 to 30 minutes in length, from 9 different people.2.1.1 Rotation correctionAs described in [6], several realistic anomalies occur in the dataset, includingdata missingness and here we will describe a method of correcting misorientated1We used a currently in preparation to be made public version of the dataset which providesinformation on who carried out each script. The current public version does not include thisinformation.ESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  616accelerometers to ensure that they conform to a standard reference. The tech-nique we introduce is an important preprocessing step for person identificationand activity recognition on the dataset at hand, but is also relevant in othersettings, e.g. mobile phones in pockets.We begin by computing the empirical distribution of each accelerometer se-quence using histograms with K non overlapping bins of width 0.5g over theinterval of −4g to 4g. A distance measure between probability distributions isrequired, and we have selected the Brier score [7] since it is a symmetric mea-sure (unlike e.g. the KL divergence). One sequence is selected (arbitrarily) asa reference, and a set of prespecified candidate 3D rotations are applied to theremaining sequences. Finally, for each sequence the rotation that achieves min-imal distance to the reference distribution is selected, and subsequently used inour analysis.2.2 Methods2s 4s 8s 12s 30s 60s 120s 240s 720s 780s 840s 1200swindow size0.200.250.300.350.400.450.500.550.60accuracyFigure 2: LR performanceover various window sizes.A natural way to approach this problem is to treatit as a multiclass classification problem where eachperson represents a different class. However, thisrelies on prior knowledge of which participant waswearing the wrist worn accelerometer which maynot always be accessible. Thus we will also investi-gate an unsupervised method where we will clusterall of the accelerometer data with the objective thateach discovered cluster will correspond to a single person.2.2.1 SupervisedFirst, for our supervised approach we train on each of the participants’ firstscripted instance and test on the second instance. While this is more challenging,it is also fair as it prevents issues such as learning to recognise the script thatthe participant is carrying out rather than the person themselves.We extract the following simple features over a one second window; the mean,min, max, variance and standard deviation of each axis. However, calculatingthe features over just one second of activity may not contain enough of anidentifiable signal. Thus we further compute the mean of each of these featuresover longer rolling time windows ranging from 2 seconds up to windows coveringsignificant portions of the sequence. The importance of this can be seen in Fig. 1where we can see the effect of increasing the window size (with PCA applied)for each of the 9 participants. With a 2 second window it is difficult to seeany differentiating structure in the data, however as the window size increasesto 30, 60 and then 720 seconds each individuals’ signal becomes much clearer.We subsequently use a large window size, 720 seconds, to evaluate a number ofclassification algorithms for the task of person identification.For each frame of each test set we use each model to predict which person(class label) the given frame belongs to. Due to the varied time it takes to carryout each script, the dataset is not equally balanced per script (person), althoughthe random classifier performance on the dataset of 9 people is still close to 11%.ESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  617XGB MLP NN Lin SVMRBF SVM RF Log Reg NBayesclassifier0.00.10.20.30.40.50.60.7accuracyFigure 3: Classification performanceover each script.All models perform comparably, withthe worst results around 2.5x betterthan random and the best over 5xbetter than random at 58% accuracy.The best model was a multinomi-nal Logistic Regression with cross-validated parameters closely followedby a Random Forest. We also evalu-ated each model in a different way; aswe knew that each set of accelerometer readings came from a single person, butnot who that person was, we took the majority class predicted over all of thesequence for a given person. The results can be seen in Fig. 3. Most modelsperformance improved in this scenario, with again the Logistic Regression andRandom Forest among the best performers with 68% and 67% accuracy respec-tively. Recall the visualization discussed in Fig. 1 showing the window size tobe very influential. In order to validate this, we plot the performance of theLogistic Regression model trained and tested on window sizes ranging from just2 seconds up to 1200 seconds in Fig. 2. It is clear that up until 840 secondsclassification accuracy improves consistently.0 1 2 3 4 5 6 7 8012345678030060090012001500Figure 4: Confusion ma-trix from the LR classifier.We further investigate the performance of theLogistic Regression (720s) model by analyzing theconfusion matrix and classification report. Fromthe confusion matrix in Fig. 4 it is clear that manypeople were correctly classified in the test set, withoccasional confusion with at most one or two otherpeople. We can see that person 4 is almost neverpredicted; investigation shows a recall of 0 and aprecision of just 0.01. The rest of the people are de-tected much more accurately, with F-Scores rangingfrom 0.19 up to 0.93, with an average of 0.55.2.2.2 UnsupervisedA more challenging scenario is to consider the problem as completely unsuper-vised; we have a set of accelerometer readings and we wish to discover whichreadings belong to which person.As before we calculate the features over a rolling window of 720 seconds.Due to the structured nature of the data apparent in Fig. 1 we first attempt touse a density based clustering algorithm. Using DBSCAN [8] with the featuresstandardized, removing the mean and scaled to unit variance, 65 clusters werefound, with high scores by most metrics and far from the 9 expected clusters.Thus, we evaluate an approach where we can specify the number of clusters inadvance. We use the agglomerative hierarchical clustering Ward [9] method andspecify the number of clusters as 9. We further specify connectivity constraintsbuilt using a K-Nearest Neighbour (kNN) graph where k is 50. By all metricswe achieve good performance in discovering clusters associated with each personESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  618All Trans. Posture Ambul. NoneHomogeneity 0.63 0.48 0.48 0.54 0.50Completeness 0.66 0.50 0.51 0.56 0.56Adj. Mut. Info 0.63 0.48 0.48 0.54 0.49Silhouette Coef. 0.13 0.21 0.19 0.28 0.34Purity 0.68 0.57 0.53 0.57 0.65Num. Points 32395 7855 23816 5082 1345Table 1: Cluster quality metrics for 9 clusters.across the two different sets (Table 1).We note that the same issue occurred with the clustering method as did withthe classification method; the person 4 was not reliably clustered into their owncluster. Instead they were often misclustered. We also found that the activitythat the participants were carrying out had no significant influence, with allperforming similarly (Table 1).As compared to the accuracy in classification methods, from around 50% witha 2 minute window to 58% with a 14 minute window, when it comes to unsuper-vised clustering methods we can achieve high quality clusters corresponding toparticipants with all quality metrics tried (when having access to the number ofindividuals expected). Interestingly, the cluster purity, which is most similar toa classification accuracy score, is around 68%, which is around a 10% improve-ment over the classification accuracy. This can be explained due to the fact thatthe unsupervised method had more data to learn from since only 50% of thedata for each person was utilized in classification for training.3 Related WorkWhile some existing supervised person identification work exists [10], per-haps the most relevant work is that of Hernandez et al. [4] where a method forboth posture and person identification using a wrist worn smartwatch (amongothers) is proposed. It extracts the ballistocardiography (BCG) signal from theparticipant using the wearable, then classifies the wearer as they carry out threedifferent stationary activities; specifically sitting, lying and standing over twodifferent one minute periods. Using a Galaxy Gear smartwatch sampling at100Hz they achieved an accuracy of 42.93% on their dataset, when the randomaccuracy would be 8.3%. Our work differs, and is an improvement in a numberof ways. First, we sample at 20Hz and use much less complex statistical featurescalculated on the accelerometer data. Their method relies on the participantstanding still for 10 seconds (while ours does not) in order to extract the BCGsignature. Our dataset is also significantly larger, with around 60 total minutesof data for each of our 9 participants. Further, our data is split into two differentperiods, where the same script consisting of 20 types of activities is repeated byeach participant twice. By training on one period, and testing on the other, weensure a fair evaluation, for e.g. that we avoid learning to recognise the specificoccasion rather than the person.ESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  619The task of identification based on time series is also common in related fieldssuch as speech processing (speaker recognition or verification [11, 12]).4 ConclusionIn this work we proposed a strategy for person identification and discoveryusing a wrist-worn accelerometer on a large dataset consisting of many activitiesof daily living using supervised and unsupervised methods. We were able toachieve classification scores over 5x better than the random baseline. Further weare the first to propose a method of unsupervised clustering of accelerometer datafor person discovery achieving a performance across many clustering evaluationmetrics, including 68% cluster purity. While this has many useful beneficialapplications in areas such as health, it also illustrates the argument for furtherdebate and work on privacy of pervasive accelerometer data.AcknowledgementThis work was supported in part by EurValve (Personalised Decision Support for HeartValve Disease), Project Number: H2020 PHC-30-2015, 689617 and the SPHERE IRC fundedby the UK Engineering and Physical Sciences Research Council (EPSRC), Grant EP/K031910/1.References[1] James Pope et al. Sphere in a box: Practical and scalable eurvalve activity monitoringsmart home kit. In IEEE Workshop on Networks of Sensors, Wearable, and MedicalDevices, 2017.[2] Ryan McConville et al. Understanding the quality of calibrations for indoor localisation.In IEEE 4th World Forum on Internet of Things, 2018.[3] S. Sicari et al. Security, privacy and trust in internet of things: The road ahead. ComputerNetworks, 76(Supplement C):146 – 164, 2015.[4] Javier Hernandez et al. Bioinsights: extracting personal data from still wearable motionsensors. In Wearable and Implantable Body Sensor Networks, pages 1–6, 2015.[5] Niall Twomey, Tom Diethe, et al. The sphere challenge: Activity recognition with mul-timodal sensor data. arXiv preprint arXiv:1603.00797, 2016.[6] Tom Diethe, Niall Twomey, Meelis Kull, Peter Flach, and Ian Craddock. Probabilisticsensor fusion for ambient assisted living. arXiv preprint arXiv:1702.01209, 2017.[7] Glenn W Brier. Verification of forecasts expressed in terms of probability. MontheyWeather Review, 78(1):1–3, 1950.[8] Martin Ester et al. A density-based algorithm for discovering clusters a density-basedalgorithm for discovering clusters in large spatial databases with noise. In KDD, 1996.[9] Joe H Ward Jr. Hierarchical grouping to optimize an objective function. Journal of theAmerican statistical association, 58(301):236–244, 1963.[10] Gary M. Weiss and Jeffrey W. Lockhart. Identifying user traits by mining smart phoneaccelerometer data. In SensorKDD, pages 61–69, 2011.[11] Tomi Kinnunen and Haizhou Li. An overview of text-independent speaker recognition:From features to supervectors. Speech Commun., 52(1):12–40, January 2010.[12] Dario Garc´ıa-Garc´ıa and Rau´l Santos-Rodr´ıguez. Sphere packing for clustering sets ofvectors in feature space. In IEEE ICASSP, pages 2092–2095, 2011.ESANN 2018 proceedings, European Symposium on Artificial Neural Networks, Computational  Intelligence and Machine Learning.  Bruges (Belgium), 25-27 April 2018, i6doc.com publ., ISBN 978-287587047-6. Available from http://www.i6doc.com/en/.  620

Person Identification and Discovery With Wrist Worn Accelerometer Data

https://research-information.bris.ac.uk/ws/files/168726131/es2018_190.pdf

Person Identification and Discovery With Wrist Worn Accelerometer Data

Abstract

Similar works

Full text

Available Versions

Explore Bristol Research