Evidential Markov chains and trees with applications to non stationary processes segmentation

The triplet Markov chains (TMC) generalize the pairwise Markov chains (PMC), and the latter generalize the hidden Markov chains (HMC). Otherwise, in an HMC the posterior distribution of the hidden process can be viewed as a particular case of the so called "Dempster's combination rule" of its prior Markov distribution p with a probability q defined from the observations. When we place ourselves in the theory of evidence context by replacing p by a mass function m, the result of the Dempster's combination of m with q generalizes the conventional posterior distribution of the hidden process. Although this result is not necessarily a Markov distribution, it has been recently shown that it is a TMC, which renders traditional restoration methods applicable. Further, these results remain valid when replacing the Markov chains with Markov trees. We propose to extend these results to Pairwise Markov trees. Further, we show the practical interest of such combination in the unsupervised segmentation of non stationary hidden Markov chains, with application to unsupervised image segmentation.Les chaînes de Markov Triplet (CMT) généralisent les chaînes de Markov Couple (CMCouple), ces dernières généralisant les chaînes de Markov cachées (CMC). Par ailleurs, dans une CMC la loi a posteriori du processus caché, qui est de Markov, peut être vue comme une combinaison de Dempster de sa loi a priori p avec une probabilité q définie à partir des observations. Lorsque l'on se place dans le contexte de la théorie de l'évidence en remplaçant p par une fonction de masse m, sa combinaison de Dempster avec q généralise ainsi la probabilité a posteriori. Bien que le résultat de cette fusion ne soit pas nécessairement une chaîne de Markov, il a été récemment établi qu'il est une CMT, ce qui autorise les divers traitements d'intérêt. De plus, les résultats analogues restent valables lorsque l'on généralise les différentes chaînes de Markov aux arbres de Markov. Nous proposons d'étendre ces résultats aux arbres de Markov Couple, dans lesquels la loi du processus caché n'est pas nécessairement de Markov. Nous montrons également l'intérêt pratique de ce type de fusion dans la segmentation non supervisée des chaînes de Markov non stationnaires, avec application à la segmentation d'images

On the 3D point cloud for human-pose estimation

This thesis aims at investigating methodologies for estimating a human pose from a 3D point cloud that is captured by a static depth sensor. Human-pose estimation (HPE) is important for a range of applications, such as human-robot interaction, healthcare, surveillance, and so forth. Yet, HPE is challenging because of the uncertainty in sensor measurements and the complexity of human poses. In this research, we focus on addressing challenges related to two crucial components in the estimation process, namely, human-pose feature extraction and human-pose modeling. In feature extraction, the main challenge involves reducing feature ambiguity. We propose a 3D-point-cloud feature called viewpoint and shape feature histogram (VISH) to reduce feature ambiguity by capturing geometric properties of the 3D point cloud of a human. The feature extraction consists of three steps: 3D-point-cloud pre-processing, hierarchical structuring, and feature extraction. In the pre-processing step, 3D points corresponding to a human are extracted and outliers from the environment are removed to retain the 3D points of interest. This step is important because it allows us to reduce the number of 3D points by keeping only those points that correspond to the human body for further processing. In the hierarchical structuring, the pre-processed 3D point cloud is partitioned and replicated into a tree structure as nodes. Viewpoint feature histogram (VFH) and shape features are extracted from each node in the tree to provide a descriptor to represent each node. As the features are obtained based on histograms, coarse-level details are highlighted in large regions and fine-level details are highlighted in small regions. Therefore, the features from the point cloud in the tree can capture coarse level to fine level information to reduce feature ambiguity. In human-pose modeling, the main challenges involve reducing the dimensionality of human-pose space and designing appropriate factors that represent the underlying probability distributions for estimating human poses. To reduce the dimensionality, we propose a non-parametric action-mixture model (AMM). It represents high-dimensional human-pose space using low-dimensional manifolds in searching human poses. In each manifold, a probability distribution is estimated based on feature similarity. The distributions in the manifolds are then redistributed according to the stationary distribution of a Markov chain that models the frequency of human actions. After the redistribution, the manifolds are combined according to a probability distribution determined by action classification. Experiments were conducted using VISH features as input to the AMM. The results showed that the overall error and standard deviation of the AMM were reduced by about 7.9% and 7.1%, respectively, compared with a model without action classification. To design appropriate factors, we consider the AMM as a Bayesian network and propose a mapping that converts the Bayesian network to a neural network called NN-AMM. The proposed mapping consists of two steps: structure identification and parameter learning. In structure identification, we have developed a bottom-up approach to build a neural network while preserving the Bayesian-network structure. In parameter learning, we have created a part-based approach to learn synaptic weights by decomposing a neural network into parts. Based on the concept of distributed representation, the NN-AMM is further modified into a scalable neural network called NND-AMM. A neural-network-based system is then built by using VISH features to represent 3D-point-cloud input and the NND-AMM to estimate 3D human poses. The results showed that the proposed mapping can be utilized to design AMM factors automatically. The NND-AMM can provide more accurate human-pose estimates with fewer hidden neurons than both the AMM and NN-AMM can. Both the NN-AMM and NND-AMM can adapt to different types of input, showing the advantage of using neural networks to design factors

Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Fusion de Dempster–Shafer dans les chaînes triplet partiellement de Markov

International audienceHidden Markov Chains (HMC), Pairwise Markov Chains (PMC), and Triplet Markov Chains (TMC), allow one to estimate a hidden process X from an observed process Y. More recently, TMC have been generalized to Triplet Partially Markov chain (TPMC), where the estimation of X from Y remains workable. Otherwise, when introducing a Dempster-Shafer mass function instead of prior Markov distribution in classical HMC, the result of its Dempster-Shafer fusion with a distribution provided Y = y, which generalizes the posterior distribution of X, is a TMC. The aim of this Note is to generalize the latter result replacing HMC with multisensor TPMC.Les Chaînes de Markov Cachées (CMC), Chaînes de Markov Couple (CMCouple), ou Chaînes de Markov Triplet (CMT), permettent d'estimer un processus caché X à partir d'un processus observé Y. Récemment, les CMT ont été généralisées aux Chaînes Triplet Partiellement de Markov (CTPM), où l'estimation de X demeure possible. Par ailleurs, lorsque dans une CMC classique la loi a priori est remplacée par une masse de Dempster-Shafer, le résultat de la fusion de cette dernière avec une loi définie par Y = y, qui généralise la loi a posteriori de X, est une CMT. L'objet de cette Note est de généraliser ce dernier résultat de CMC aux CTPM multicapteu

Markov models in image processing

The aim of this paper is to present some aspects of Markov model based statistical image processing. After a brief review of statistical processing in image segmentation, classical Markov models (fields, chains, and trees) used in image processing are developed. Bayesian methods of segmentation are then described and different general parameter estimation methods are presented. More recent models and processing techniques, such as Pairwise and Triplet Markov models, Dempster-Shafer fusion in a Markov context, and generalized mixture estimation, are then discussed. We conclude with a nonexhaustive desciption of candidate extensions to multidimensional, multisensor, and multiresolution imagery. Connections with general graphical models are also highlighted.L'objet de l'article est de présenter divers aspects des traitements statistiques des images utilisant des modèles de Markov. En choisissant pour cadre la segmentation statistique nous rappelons brièvement la nature et l'intérêt des traitements probabilistes et présentons les modèles de Markov cachés classiques : champs, chaînes, et arbres. Les méthodes bayésiennes de segmentation sont décrites, ainsi que les grandes familles des méthodes d'apprentissage. Quelques modèles ou méthodes de traitements plus récents comme les modèles de Markov Couple et Triplet, la fusion de Dempster-Shafer dans le contexte markovien, ou l'estimation des mélanges généralisés sont également présentés. Nous terminons par une liste non exhaustive des divers prolongements des méthodes et modèles vers l'imagerie multidimensionnelle, multisenseurs, multirésolution. Des liens avec les modèles graphiques généraux sont également brièvement décrits