Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks

Human action recognition in 3D skeleton sequences has attracted a lot of research attention. Recently, Long Short-Term Memory (LSTM) networks have shown promising performance in this task due to their strengths in modeling the dependencies and dynamics in sequential data. As not all skeletal joints are informative for action recognition, and the irrelevant joints often bring noise which can degrade the performance, we need to pay more attention to the informative ones. However, the original LSTM network does not have explicit attention ability. In this paper, we propose a new class of LSTM network, Global Context-Aware Attention LSTM (GCA-LSTM), for skeleton based action recognition. This network is capable of selectively focusing on the informative joints in each frame of each skeleton sequence by using a global context memory cell. To further improve the attention capability of our network, we also introduce a recurrent attention mechanism, with which the attention performance of the network can be enhanced progressively. Moreover, we propose a stepwise training scheme in order to train our network effectively. Our approach achieves state-of-the-art performance on five challenging benchmark datasets for skeleton based action recognition

Sensor-based datasets for human activity recognition - a systematic review of literature

The research area of ambient assisted living has led to the development of activity recognition systems (ARS) based on human activity recognition (HAR). These systems improve the quality of life and the health care of the elderly and dependent people. However, before making them available to end users, it is necessary to evaluate their performance in recognizing activities of daily living, using data set benchmarks in experimental scenarios. For that reason, the scientific community has developed and provided a huge amount of data sets for HAR. Therefore, identifying which ones to use in the evaluation process and which techniques are the most appropriate for prediction of HAR in a specific context is not a trivial task and is key to further progress in this area of research. This work presents a systematic review of the literature of the sensor-based data sets used to evaluate ARS. On the one hand, an analysis of different variables taken from indexed publications related to this field was performed. The sources of information are journals, proceedings, and books located in specialized databases. The analyzed variables characterize publications by year, database, type, quartile, country of origin, and destination, using scientometrics, which allowed identification of the data set most used by researchers. On the other hand, the descriptive and functional variables were analyzed for each of the identified data sets: occupation, annotation, approach, segmentation, representation, feature selection, balancing and addition of instances, and classifier used for recognition. This paper provides an analysis of the sensor-based data sets used in HAR to date, identifying the most appropriate dataset to evaluate ARS and the classification techniques that generate better results

Applying machine learning to identify autistic adults using imitation: An exploratory study

Autism spectrum condition (ASC) is primarily diagnosed by behavioural symptoms including social, sensory and motor aspects. Although stereotyped, repetitive motor movements are considered during diagnosis, quantitative measures that identify kinematic characteristics in the movement patterns of autistic individuals are poorly studied, preventing advances in understanding the aetiology of motor impairment, or whether a wider range of motor characteristics could be used for diagnosis. The aim of this study was to investigate whether data-driven machine learning based methods could be used to address some fundamental problems with regard to identifying discriminative test conditions and kinematic parameters to classify between ASC and neurotypical controls. Data was based on a previous task where 16 ASC participants and 14 age, IQ matched controls observed then imitated a series of hand movements. 40 kinematic parameters extracted from eight imitation conditions were analysed using machine learning based methods. Two optimal imitation conditions and nine most significant kinematic parameters were identified and compared with some standard attribute evaluators. To our knowledge, this is the first attempt to apply machine learning to kinematic movement parameters measured during imitation of hand movements to investigate the identification of ASC. Although based on a small sample, the work demonstrates the feasibility of applying machine learning methods to analyse high-dimensional data and suggest the potential of machine learning for identifying kinematic biomarkers that could contribute to the diagnostic classification of autism

Sensor-based datasets for human activity recognition - a systematic review of literature

The research area of ambient assisted living has led to the development of activity recognition systems (ARS) based on human activity recognition (HAR). These systems improve the quality of life and the health care of the elderly and dependent people. However, before making them available to end users, it is necessary to evaluate their performance in recognizing activities of daily living, using data set benchmarks in experimental scenarios. For that reason, the scientific community has developed and provided a huge amount of data sets for HAR. Therefore, identifying which ones to use in the evaluation process and which techniques are the most appropriate for prediction of HAR in a specific context is not a trivial task and is key to further progress in this area of research. This work presents a systematic review of the literature of the sensor-based data sets used to evaluate ARS. On the one hand, an analysis of different variables taken from indexed publications related to this field was performed. The sources of information are journals, proceedings, and books located in specialized databases. The analyzed variables characterize publications by year, database, type, quartile, country of origin, and destination, using scientometrics, which allowed identification of the data set most used by researchers. On the other hand, the descriptive and functional variables were analyzed for each of the identified data sets: occupation, annotation, approach, segmentation, representation, feature selection, balancing and addition of instances, and classifier used for recognition. This paper provides an analysis of the sensor-based data sets used in HAR to date, identifying the most appropriate dataset to evaluate ARS and the classification techniques that generate better results

Phenotyping functional brain dynamics:A deep learning prespective on psychiatry

This thesis explores the potential of deep learning (DL) techniques combined with multi-site functional magnetic resonance imaging (fMRI) to enable automated diagnosis and biomarker discovery for psychiatric disorders. This marks a shift from the convention in the field of applying standard machine learning techniques on hand-crafted features from a single cohort.To enable this, we have focused on three main strategies: utilizing minimally pre-processed data to maintain spatio-temporal dynamics, developing sample-efficient DL models, and applying emerging DL training techniques like self-supervised and transfer learning to leverage large population-based datasets.Our empirical results suggest that DL models can sometimes outperform existing machine learning methods in diagnosing Autism Spectrum Disorder (ASD) and Major Depressive Disorder (MDD) from resting-state fMRI data, despite the smaller datasets and the high data dimensionality. Nonetheless, the generalization performance of these models is currently insufficient for clinical use, raising questions about the feasibility of applying supervised DL for diagnosis or biomarker discovery due to the highly heterogeneous nature of the disorders. Our findings suggest that normative modeling on functional brain dynamics provides a promising alternative to the current paradigm

Diagnosing Autism Spectrum Disorder through Brain Functional Magnetic Resonance Imaging

Autism spectrum disorder (ASD) is a neurodevelopmental condition that can be debilitating to social functioning. Previous functional Magnetic Resonance Imaging (fMRI) classification studies have included only small subject sample sizes (n 50) and have seen high classification accuracy. The recent release of the Autism Brain Imaging Data Exchange (ABIDE) provides fMRI data for over 1,100 subjects. In our research, we derive a subject\u27s functional network connectivity (FNC) from their fMRI data and develop a regularized logistic classifier to determine whether a subject has autism. We obtained up to 65% classification accuracy, similar to other studies using the ABIDE dataset, suggesting that generalizing a classifier over a large number of subjects is much more difficult than smaller studies. The connectivity among several brain regions of ASD subjects were highlighted in the model as abnormal compared to the control subjects which potentially warrants future investigations about how these regions affect ASD. Although the classification accuracy was lower than what could be considered as clinically applicable, this research contributes to the continuing development of an automated classifier for diagnosing autism