Riemannian manifold-valued part-based features and geodesic-induced kernel machine for activity classification dedicated to assisted living

In this paper, we address the problem of classifying human activities that are typical in a daily living environment from videos. We propose a novel method based on Riemannian manifolds that uses a tree structure of two layers, where nodes in each tree branch are on a Riemannian manifold. Each node corresponds to different part-based covariance features, and induces a geodesic-based kernel machine for classification. In the first layer, activities are classified according to the dynamics of body pose and the movement of hands or arms. Activities with similar body pose and motion but different human-object interaction are coarsely classified into the same category. In the second layer, the coarsely classified activities are further fine classified, according to the appearance of local image patches at hands in key frames. This is based on the observation that interacting objects as discriminative cues are likely to be attached to hands. The main novelties of this paper include: (i) Motion of body parts for each video activity is characterized by global features. More specifically, the features are distances between each pair of key points and the orientations of lines that connect them; (ii) Human-object interaction is described by local features. That is, the appearance of local regions around hands in key frames, where key frames are selected using the proximity of hands to other key points; (iii) Classification of human activities is formulated by a geodesic distance- induced kernel machine. This is done by exploiting pair-wise geodesics on Riemannian manifolds under the log-Euclidean metric. Experiments were conducted on 2 video datasets. The first dataset, made on our university campus, contains 8 activities with a total number of 943 videos. The second dataset is from a publicly available dataset, containing 7 activity classes and a total of 224 videos. Our test results on the first video dataset have shown high classification accuracy (average 94.27%), and small false alarm rate (average 0.80%). For the second video dataset, test results from the proposed method are compared with 6 existing methods. The proposed method has outperformed all these existing methods. Discussions are given on the impact of detected skeleton points from Kinect on the performance of activity classification

Machine Learning Methods for Image Analysis in Medical Applications, from Alzheimer\u27s Disease, Brain Tumors, to Assisted Living

Healthcare has progressed greatly nowadays owing to technological advances, where machine learning plays an important role in processing and analyzing a large amount of medical data. This thesis investigates four healthcare-related issues (Alzheimer\u27s disease detection, glioma classification, human fall detection, and obstacle avoidance in prosthetic vision), where the underlying methodologies are associated with machine learning and computer vision. For Alzheimer’s disease (AD) diagnosis, apart from symptoms of patients, Magnetic Resonance Images (MRIs) also play an important role. Inspired by the success of deep learning, a new multi-stream multi-scale Convolutional Neural Network (CNN) architecture is proposed for AD detection from MRIs, where AD features are characterized in both the tissue level and the scale level for improved feature learning. Good classification performance is obtained for AD/NC (normal control) classification with test accuracy 94.74%. In glioma subtype classification, biopsies are usually needed for determining different molecular-based glioma subtypes. We investigate non-invasive glioma subtype prediction from MRIs by using deep learning. A 2D multi-stream CNN architecture is used to learn the features of gliomas from multi-modal MRIs, where the training dataset is enlarged with synthetic brain MRIs generated by pairwise Generative Adversarial Networks (GANs). Test accuracy 88.82% has been achieved for IDH mutation (a molecular-based subtype) prediction. A new deep semi-supervised learning method is also proposed to tackle the problem of missing molecular-related labels in training datasets for improving the performance of glioma classification. In other two applications, we also address video-based human fall detection by using co-saliency-enhanced Recurrent Convolutional Networks (RCNs), as well as obstacle avoidance in prosthetic vision by characterizing obstacle-related video features using a Spiking Neural Network (SNN). These investigations can benefit future research, where artificial intelligence/deep learning may open a new way for real medical applications