Evolving, dynamic clustering of spatio/spectro-temporal data in 3D spiking neural network models and a case study on EEG data

Clustering is a fundamental data processing technique. While clustering of static (vector based) data and of fixed window size time series have been well explored, dynamic clustering of spatiotemporal data has been little researched if at all. Especially when patterns of changes (events) in the data across space and time have to be captured and understood. The paper presents novel methods for clustering of spatiotemporal data using the NeuCube spiking neural network (SNN) architecture. Clusters of spatiotemporal data were created and modified on-line in a continuous, incremental way, where spatiotemporal relationships of changes in variables are incrementally learned in a 3D SNN model and the model connectivity and spiking activity are incrementally clustered. Two clustering methods were proposed for SNN, one performed during unsupervised and one—during supervised learning models. Before submitted to the models, the data is encoded as spike trains, a spike representing a change in the variable value (an event). During the unsupervised learning, the cluster centres were predefined by the spatial locations of the input data variables in a 3D SNN model. Then clusters are evolving during the learning, i.e. they are adapted continuously over time reflecting the dynamics of the changes in the data. In the supervised learning, clusters represent the dynamic sequence of neuron spiking activities in a trained SNN model, specific for a particular class of data or for an individual instance. We illustrate the proposed clustering method on a real case study of spatiotemporal EEG data, recorded from three groups of subjects during a cognitive task. The clusters were referred back to the brain data for a better understanding of the data and the processes that generated it. The cluster analysis allowed to discover and understand differences on temporal sequences and spatial involvement of brain regions in response to a cognitive task

Personalised modelling with spiking neural networks integrating temporal and static information.

This paper proposes a new personalised prognostic/diagnostic system that supports classification, prediction and pattern recognition when both static and dynamic/spatiotemporal features are presented in a dataset. The system is based on a proposed clustering method (named d2WKNN) for optimal selection of neighbouring samples to an individual with respect to the integration of both static (vector-based) and temporal individual data. The most relevant samples to an individual are selected to train a Personalised Spiking Neural Network (PSNN) that learns from sets of streaming data to capture the space and time association patterns. The generated time-dependant patterns resulted in a higher accuracy of classification/prediction (80% to 93%) when compared with global modelling and conventional methods. In addition, the PSNN models can support interpretability by creating personalised profiling of an individual. This contributes to a better understanding of the interactions between features. Therefore, an end-user can comprehend what interactions in the model have led to a certain decision (outcome). The proposed PSNN model is an analytical tool, applicable to several real-life health applications, where different data domains describe a person's health condition. The system was applied to two case studies: (1) classification of spatiotemporal neuroimaging data for the investigation of individual response to treatment and (2) prediction of risk of stroke with respect to temporal environmental data. For both datasets, besides the temporal data, static health data were also available. The hyper-parameters of the proposed system, including the PSNN models and the d2WKNN clustering parameters, are optimised for each individual

Design of MRI Structured Spiking Neural Networks and Learning Algorithms for Personalized Modelling, Analysis, and Prediction of EEG Signals

Abstract This paper proposes a novel method and algorithms for the design of MRI structured personalized 3D spiking neural network models (MRI-SNN) for a better analysis, modeling, and prediction of EEG signals. It proposes a novel gradient-descent learning algorithm integrated with a spike-time-dependent-plasticity algorithm. The models capture informative personal patterns of interaction between EEG channels, contrary to single EEG signal modeling methods or to spike-based approaches which do not use personal MRI data to pre-structure a model. The proposed models can not only learn and model accurately measured EEG data, but they can also predict signals at 3D model locations that correspond to non-monitored brain areas, e.g. other EEG channels, from where data has not been collected. This is the first study in this respect. As an illustration of the method, personalized MRI-SNN models are created and tested on EEG data from two subjects. The models result in better prediction accuracy and a better understanding of the personalized EEG signals than traditional methods due to the MRI and EEG information integration. The models are interpretable and facilitate a better understanding of related brain processes. This approach can be applied for personalized modeling, analysis, and prediction of EEG signals across brain studies such as the study and prediction of epilepsy, peri-perceptual brain activities, brain-computer interfaces, and others

Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network.

The paper proposes a new method for deep learning and knowledge discovery in a brain-inspired Spiking Neural Networks (SNN) architecture that enhances the model’s explainability while learning from streaming spatiotemporal brain data (STBD) in an incremental and on-line mode of operation. This led to the extraction of spatiotemporal rules from SNN models that explain why a certain decision (output prediction) was made by the model. During the learning process, the SNN created dynamic neural clusters, captured as polygons, which evolved in time and continuously changed their size and shape. The dynamic patterns of the clusters were quantitatively analyzed to identify the important STBD features that correspond to the most activated brain regions. We studied the trend of dynamically created clusters and their spike-driven events that occur together in specific space and time. The research contributes to: (1) enhanced interpretability of SNN learning behavior through dynamic neural clustering; (2) feature selection and enhanced accuracy of classification; (3) spatiotemporal rules to support model explainability; and (4) a better understanding of the dynamics in STBD in terms of feature interaction. The clustering method was applied to a case study of Electroencephalogram (EEG) data, recorded from a healthy control group (n = 21) and opiate use (n = 18) subjects while they were performing a cognitive task. The SNN models of EEG demonstrated different trends of dynamic clusters across the groups. This suggested to select a group of marker EEG features and resulted in an improved accuracy of EEG classification to 92%, when compared with all-feature classification. During learning of EEG data, the areas of neurons in the SNN model that form adjacent clusters (corresponding to neighboring EEG channels) were detected as fuzzy boundaries that explain overlapping activity of brain regions for each group of subjects

Neurocomputing for spatio-/spectro temporal pattern recognition and early event prediction: methods, systems, applications

The talk presents a brief overview of contemporary methods for neurocomputation, including: evolving connections systems (ECOS) and evolving neuro-fuzzy systems [1]; evolving spiking neural networks (eSNN) [2-5]; evolutionary and neurogenetic systems [6]; quantum inspired evolutionary computation [7,8]; rule extraction from eSNN [9]. These methods are suitable for incremental adaptive, on-line learning from spatio-temporal data and for data mining. But the main focus of the talk is how they can learn to predict early the outcome of an input spatio-temporal pattern, before the whole pattern is entered in a system. This is demonstrated on several applications in bioinformatics, such as stroke occurrence prediction, and brain data modeling for brain-computer interfaces [10], on ecological and environmental modeling [11]. eSNN have proved superior for spatio-and spectro-temporal data analysis, modeling, pattern recognition and early event prediction as outcome of recognized patterns when partially presented

Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network

The paper proposes a new method for deep learning and knowledge discovery in a brain-inspired Spiking Neural Networks (SNN) architecture that enhances the model's explainability while learning from streaming spatiotemporal brain data (STBD) in an incremental and on-line mode of operation. This led to the extraction of spatiotemporal rules from SNN models that explain why a certain decision (output prediction) was made by the model. During the learning process, the SNN created dynamic neural clusters, captured as polygons, which evolved in time and continuously changed their size and shape. The dynamic patterns of the clusters were quantitatively analyzed to identify the important STBD features that correspond to the most activated brain regions. We studied the trend of dynamically created clusters and their spike-driven events that occur together in specific space and time. The research contributes to: (1) enhanced interpretability of SNN learning behavior through dynamic neural clustering; (2) feature selection and enhanced accuracy of classification; (3) spatiotemporal rules to support model explainability; and (4) a better understanding of the dynamics in STBD in terms of feature interaction. The clustering method was applied to a case study of Electroencephalogram (EEG) data, recorded from a healthy control group (n = 21) and opiate use (n = 18) subjects while they were performing a cognitive task. The SNN models of EEG demonstrated different trends of dynamic clusters across the groups. This suggested to select a group of marker EEG features and resulted in an improved accuracy of EEG classification to 92%, when compared with all-feature classification. During learning of EEG data, the areas of neurons in the SNN model that form adjacent clusters (corresponding to neighboring EEG channels) were detected as fuzzy boundaries that explain overlapping activity of brain regions for each group of subjects

Machine learning methods for the study of cybersickness: a systematic review

This systematic review offers a world-first critical analysis of machine learning methods and systems, along with future directions for the study of cybersickness induced by virtual reality (VR). VR is becoming increasingly popular and is an important part of current advances in human training, therapies, entertainment, and access to the metaverse. Usage of this technology is limited by cybersickness, a common debilitating condition experienced upon VR immersion. Cybersickness is accompanied by a mix of symptoms including nausea, dizziness, fatigue and oculomotor disturbances. Machine learning can be used to identify cybersickness and is a step towards overcoming these physiological limitations. Practical implementation of this is possible with optimised data collection from wearable devices and appropriate algorithms that incorporate advanced machine learning approaches. The present systematic review focuses on 26 selected studies. These concern machine learning of biometric and neuro-physiological signals obtained from wearable devices for the automatic identification of cybersickness. The methods, data processing and machine learning architecture, as well as suggestions for future exploration on detection and prediction of cybersickness are explored. A wide range of immersion environments, participant activity, features and machine learning architectures were identified. Although models for cybersickness detection have been developed, literature still lacks a model for the prediction of first-instance events. Future research is pointed towards goal-oriented data selection and labelling, as well as the use of brain-inspired spiking neural network models to achieve better accuracy and understanding of complex spatio-temporal brain processes related to cybersickness

Working towards improved conceptualisation and identification of gaming disorder and co-occurring addictions in gamers

This doctoral research thesis investigated the neurophysiological underpinnings of gaming disorder (GD), and the way in which co-occurrence can influence and correlate with GD in a clinical and a multi-cultural context. The unique contribution of knowledge was (i) the assessment of the neurophysiological expression of gamers using a novel spiking neural network (SNN) methodology; (ii) exploring co-occurrence in gamers and substance abstinent gamers; and (iii) exploring co-occurrence in gamers across three different individualistic countries (i.e., Australia, New Zealand, and the United Kingdom). The conceptualisation of GD and related methodologies were explored using multiple systematic research methods. A number of methodologies were then employed, including the use of electroencephalographic (EEG) data, a machine learning (ML) approach which utilised a novel SNN architecture (i.e., the NeuCube), and the use of surveys to reach a clinical cohort and three cohorts spanning three different countries in an effort to investigate the way in which co-occurrence may influence gamers and at-risk gamers. The results of the empirical studies indicated that: (i) problematic gamers experience different neurophysiological expression than those who recreational game and that ML methodologies are an effective method of classifying recreational and problematic gamers when using EEG data; (ii) maladaptive coping strategies were significantly associated to gaming scores, and that gamers appeared to experience co-occurrence more so than their non-gamer counterparts; (iii) at-risk and high-risk gamers may utilise gaming as a maladaptive coping strategy and other accompanying potentially addictive behaviour, or substance use may be influenced as a result; (iv) the manifestation of maladaptive coping strategies and potentially addictive behaviours can be influenced by the country in which an individual resides. Taken together, the present doctoral project further clarified the conceptualisation of GD, utilising a neurophysiological underpinning, which is further supported with observed behaviour as suggested by the National Institute of Mental Health. In addition, it places an emphasis on the importance of understanding co-occurrence and specific at-risk factors (e.g., coping) which may contribute to the development and maintenance of problematic or disordered gaming in both a clinical sample and general population samples