Brouwer, Anne Marie

Merasli, Alexandre

Stuldreher, Ivo V.

English

University of Twente Research Information

ProceedingsNovember 19–20, 2020Editors: Lu Cao, Walter Kosters and Jefrey LijffijtBNAIC/BeneLearn 2020ProceedingsLeiden, the NetherlandsNovember 19–20, 2020Editors: Lu Cao, Walter Kosters and Jefrey Lijffijthttp://www.bnaic.euUnsupervised clustering of groups with different selective attentional instructions using physiological synchrony Alexandre Merasli1, Ivo V Stuldreher1,2[0000-0002-6104-2162], Anne-Marie Brouwer1[0000-0003-1961-4291] 1  Netherlands Organisation for Applied Scientific Research, Soesterberg, The Netherlands 2  University of Twente, Enschede, The Netherlands alexandre.merasli@outlook.fr, ivo.stuldreher@tno.nl, anne-marie.brouwer@tno.nl Concurrent changes in physiological signals between people (Physiological Syn-chrony) can provide insight into attentional processes in groups, and individuals in re-lation to the group, for instance in educational settings. Earlier work showed that indi-viduals can be correctly classified into attentional groups (attending or not attending to a story) based on the degree of synchrony with members of known groups (either in-structed to attend or not attend to a story). Here we examine whether it is possible to find attentional groups from synchrony data using unsupervised learning, which may enable the identification of attentional groups without knowing anything about possible attentional foci beforehand as may be the case in real-world situations.  This study is based on data from [1] (publicly available on https://github.com/ivostuldreher/physiological-synchrony-selective-attention), where half of the participants were asked to focus on the story told by an audiobook, while the others were asked to focus on separate sound stimuli, occurring during the story. All participants heard the exact same audio file - only the instruction differed for the two attentional groups - and physiological signals were recorded during the experiment from three modalities (EEG - electroencephalography, EDA - electrodermal activity or skin conductance, and heart rate). From these signals, physiological synchrony was computed for each modality to assess the level of correlation between participants. Each synchrony coefficient represents the extent of synchronous change of physiological sig-nals between two participants [1]. First, we investigated whether it is possible to classify people according to their at-tentional focus, by clustering synchrony values between participants as input. Cluster-ing is a kind of method to find groups in the data without knowing any data labels (in this case, membership of the true attentional group). A straightforward approach is to directly apply specific clustering algorithms on the pre-processed synchrony data. Such kind of algorithms like hierarchical clustering or K-Medoids can cluster data without using coordinates but rather distances between points, which is the kind of information in the pre-processed synchrony data. The data appeared to be not easily clusterable, resulting in low performance (e.g. an accuracy of 65% when clustering EEG with hier-archical clustering). We then investigated adding a step before clustering, with different mapping methods which can be used to visualize and remove noise from the original synchrony data. Principal Coordinate Analysis (PCoA), Multi-Dimensional Scaling (MDS), or Uniform Manifold Approximation and Projection (UMAP) were used, with BNAIC/BeneLearn 2020 4282 the aim of finding coordinates which approximate the pre-processed synchrony data. Then, several clustering algorithms like K-Means, spectral clustering or hierarchical clustering were applied on these computed coordinates to provide two groups of par-ticipants. We studied each possible combination of mapping and clustering algorithms. We achieved the highest performance with EEG, with a classification accuracy of 85% obtained when mapping with PCoA and using spectral clustering. EDA and heart rate did not perform above chance level. After demonstrating that unsupervised detection of attentional groups is possible for EEG, we continued to look into combining the information coming from different mo-dalities (EEG, EDA and heart rate) in order to possibly enhance the information coming from only one modality. Mapping methods like Multi View Multi-Dimensional Scaling (MVMDS) or Multi View Spectral Clustering (MVSC), which can be seen as an exten-sion of respectively PCoA and spectral clustering, enabled us to combine the infor-mation present in several input matrices to create a map with all participants. We found that adding EDA to EEG did not improve the accuracy of the results, but made them more robust to varying pipelines than using EEG alone: the worst result using EEG alone was 54% accurate, whereas the worst result when EEG and EDA were combined was 73% accurate. From all combinations and pipelines, the best result was achieved by combining EEG and heart rate, where results were more robust and more accurate than EEG alone, with an accuracy of 92% when applying K-Means after MVMDS. Combining three modalities rather than only adding EDA or heart rate to EEG did not seem to further improve performance.  Finally, we investigated how to approach the problem of choosing the proper classi-fication pipeline for real world cases that our data may not generalize to. In the current study, we could use our known attentional groups to evaluate the clustering perfor-mance. However, in real-world applications, this ground truth information is usually not available. We therefore studied the silhouette coefficient to assess clustering qual-ity, and investigated its correspondence with the ground truth accuracy. Unfortunately, the data appear to be too noisy to successfully use this coefficient to choose good clus-tering pipelines among all methods.  In sum, our study indicates that it is possible to use unsupervised clustering on phys-iological synchrony data to identify groups with different attentional foci. Performance is close to that reached using knowledge of attentional groups, i.e., classifying an indi-vidual into a known attentional group that he or she correlates with most strongly [1]. Similar as in [1], we found that physiological synchrony in EEG is more informative than EDA and heart rate. However, adding heart rate or EDA to EEG results in classi-fication performance that depends less on the specific pipeline.  References 1. Stuldreher, I. V., Thammasan, N., van Erp, J. B., & Brouwer, A. M. (2020). Physiological synchrony in EEG, electrodermal activity and heart rate reflects shared selective auditory attention. Journal of Neural Engineering, 17(4), 046028. BNAIC/BeneLearn 2020 429

Unsupervised clustering of groups with different selective attentional instructions using physiological synchrony

https://ris.utwente.nl/ws/files/320116051/bnaic2020-merasli.pdf

Unsupervised clustering of groups with different selective attentional instructions using physiological synchrony

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information