55 research outputs found

    Modelling, tracking and generating human interaction behaviours in video

    Get PDF
    Intelligent virtual characters are becoming increasingly popular in en­ tertainment, educational and simulation software. A virtual charac­ ter is the creation or re-creation of a human being in an image, using computer-generated imagery. It must act and react in the environment, drawing on the disciplines of automated reasoning and planning. Creating characters with human-like behaviours that respond interactively to a real person in a video, is still a serious challenge. There are several major reasons for this. First, human motion is very complex, which makes it particularly difficult to simulate. Second, the human form is also not straightforward to design due to the large number of degrees of freedom of the motion. Third, creating novel contextual movements for virtual characters in real time is a new research area.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

    Modelling, tracking and generating human interaction behaviours in video.

    Get PDF
    Intelligent virtual characters are becoming increasingly popular in en­ tertainment, educational and simulation software. A virtual charac­ ter is the creation or re-creation of a human being in an image, using computer-generated imagery. It must act and react in the environment, drawing on the disciplines of automated reasoning and planning. Cre­ ating characters with human-like behaviours that respond interactively to a real person in a video, is still a serious challenge. There are several major reasons for this. First, human motion is very complex, which makes it particularly difficult to simulate. Second, the human form is also not straightforward to design due to the large number of degrees of freedom of the motion. Third, creating novel contextual movements for virtual characters in real time is a new research area

    Minimal Infrastructure Radio Frequency Home Localisation Systems

    Get PDF
    The ability to track the location of a subject in their home allows the provision of a number of location based services, such as remote activity monitoring, context sensitive prompts and detection of safety critical situations such as falls. Such pervasive monitoring functionality offers the potential for elders to live at home for longer periods of their lives with minimal human supervision. The focus of this thesis is on the investigation and development of a home roomlevel localisation technique which can be readily deployed in a realistic home environment with minimal hardware requirements. A conveniently deployed Bluetooth ® localisation platform is designed and experimentally validated throughout the thesis. The platform adopts the convenience of a mobile phone and the processing power of a remote location calculation computer. The use of Bluetooth ® also ensures the extensibility of the platform to other home health supervision scenarios such as wireless body sensor monitoring. Central contributions of this work include the comparison of probabilistic and nonprobabilistic classifiers for location prediction accuracy and the extension of probabilistic classifiers to a Hidden Markov Model Bayesian filtering framework. New location prediction performance metrics are developed and signicant performance improvements are demonstrated with the novel extension of Hidden Markov Models to higher-order Markov movement models. With the simple probabilistic classifiers, location is correctly predicted 80% of the time. This increases to 86% with the application of the Hidden Markov Models and 88% when high-order Hidden Markov Models are employed. Further novelty is exhibited in the derivation of a real-time Hidden Markov Model Viterbi decoding algorithm which presents all the advantages of the original algorithm, while producing location estimates in real-time. Significant contributions are also made to the field of human gait-recognition by applying Bayesian filtering to the task of motion detection from accelerometers which are already present in many mobile phones. Bayesian filtering is demonstrated to enable a 35% improvement in motion recognition rate and even enables a floor recognition rate of 68% using only accelerometers. The unique application of time-varying Hidden Markov Models demonstrates the effect of integrating these freely available motion predictions on long-term location predictions

    Data Informed Health Simulation Modeling

    Get PDF
    Combining reliable data with dynamic models can enhance the understanding of health-related phenomena. Smartphone sensor data characterizing discrete states is often suitable for analysis with machine learning classifiers. For dynamic models with continuous states, high-velocity data also serves an important role in model parameterization and calibration. Particle filtering (PF), combined with dynamic models, can support accurate recurrent estimation of continuous system state. This thesis explored these and related ideas with several case studies. The first employed multivariate Hidden Markov models (HMMs) to identify smoking intervals, using time-series of smartphone-based sensor data. Findings demonstrated that multivariate HMMs can achieve notable accuracy in classifying smoking state, with performance being strongly elevated by appropriate data conditioning. Reflecting the advantages of dynamic simulation models, this thesis has contributed two applications of articulated dynamic models: An agent-based model (ABM) of smoking and E-Cigarette use and a hybrid multi-scale model of diabetes in pregnancy (DIP). The ABM of smoking and E-Cigarette use, informed by cross-sectional data, supports investigations of smoking behavior change in light of the influence of social networks and E-Cigarette use. The DIP model was evidenced by both longitudinal and cross-sectional data, and is notable for its use of interwoven ABM, system dynamics (SD), and discrete event simulation elements to explore the interaction of risk factors, coupled dynamics of glycemia regulation, and intervention tradeoffs to address the growing incidence of DIP in the Australia Capital Territory. The final study applied PF with an SD model of mosquito development to estimate the underlying Culex mosquito population using various direct observations, including time series of weather-related factors and mosquito trap counts. The results demonstrate the effectiveness of PF in regrounding the states and evolving model parameters based on incoming observations. Using PF in the context of automated model calibration allows optimization of the values of parameters to markedly reduce model discrepancy. Collectively, the thesis demonstrates how characteristics and availability of data can influence model structure and scope, how dynamic model structure directly affects the ways that data can be used, and how advanced analysis methods for calibration and filtering can enhance model accuracy and versatility

    Temporal Segmentation of Human Motion for Rehabilitation

    Get PDF
    Current physiotherapy practice relies on visual observation of patient movement for assessment and diagnosis. Automation of motion monitoring has the potential to improve accuracy and reliability, and provide additional diagnostic insight to the clinician, improving treatment quality, and patient progress. To enable automated monitoring, assessment, and diagnosis, the movements of the patient must be temporally segmented from the continuous measurements. Temporal segmentation is the process of identifying the starting and ending locations of movement primitives in a time-series data sequence. Most segmentation algorithms require training data, but a priori knowledge of the patient's movement patterns may not be available, necessitating the use of healthy population data for training. However, healthy population movement data may not generalize well to rehabilitation patients due to large differences in motion characteristics between the two demographics. In this thesis, four key contributions will be elaborated to enable accurate segmentation of patient movement data during rehabilitation. The first key contribution is the creation of a segmentation framework to categorize and compare different segmentation algorithms considering segment definitions, data sources, application specific requirements, algorithm mechanics, and validation techniques. This framework provides a structure for considering the factors that must be incorporated when constructing a segmentation and identification algorithm. The framework enables systematic comparison of different segmentation algorithms, provides the means to examine the impact of each algorithm component, and allows for a systematic approach to determine the best algorithm for a given situation. The second key contribution is the development of an online and accurate motion segmentation algorithm based on a classification framework. The proposed algorithm transforms the segmentation task into a classification problem by modelling the segment edge point directly. Given this formulation, a variety of feature transformation, dimensionality reduction and classifier techniques were investigated on several healthy and patient datasets. With proper normalization, the segmentation algorithm can be trained using healthy participant data and obtain high quality segments on patient data. Inter-participant and inter-primitive variability were assessed on a dataset of 30 healthy participants and 44 rehabilitation participants, demonstrating the generalizability and utility of the proposed approach for rehabilitation settings. The proposed approach achieves a segmentation accuracy of 83-100%. The third key contribution is the investigation of feature set generalizability of the proposed method. Nearly all segmentation techniques developed previously use a single sensor modality. The proposed method was applied to joint angles, electromyogram, motion capture, and force plate data to investigate how the choice of modality impacts segmentation performance. With proper normalization, the proposed method was shown to work with various input sensor types and achieved high accuracy on all sensor modalities examined. The proposed approach achieves a segmentation accuracy of 72-97%. The fourth key contribution is the development of a new feature set based on hypotheses about the optimality of human motion trajectory generation. A common hypothesis in human motor control is that human movement is generated by optimizing with respect to a certain criterion and is task dependent. In this thesis, a method to segment human movement by detecting changes to the optimization criterion being used via inverse trajectory optimization is proposed. The control strategy employed by the motor system is hypothesized to be a weighted sum of basis cost functions, with the basis weights changing with changes to the motion objective(s). Continuous time series data of movement is processed using a sliding fixed width window, estimating the basis weights of each cost function for each window by minimizing the Karush-Kuhn-Tucker optimality conditions. The quality of the cost function recovery is verified by evaluating the residual. The successfully estimated basis weights are averaged together to create a set of time varying basis weights that describe the changing control strategy of the motion and can be used to segment the movement with simple thresholds. The proposed algorithm is first demonstrated on simulation data and then demonstrated on a dataset of human subjects performing a series of exercise tasks. The proposed approach achieves a segmentation accuracy of 74-88%

    A Data Mining Toolbox for Collaborative Writing Processes

    Get PDF
    Collaborative writing (CW) is an essential skill in academia and industry. Providing support during the process of CW can be useful not only for achieving better quality documents, but also for improving the CW skills of the writers. In order to properly support collaborative writing, it is essential to understand how ideas and concepts are developed during the writing process, which consists of a series of steps of writing activities. These steps can be considered as sequence patterns comprising both time events and the semantics of the changes made during those steps. Two techniques can be combined to examine those patterns: process mining, which focuses on extracting process-related knowledge from event logs recorded by an information system; and semantic analysis, which focuses on extracting knowledge about what the student wrote or edited. This thesis contributes (i) techniques to automatically extract process models of collaborative writing processes and (ii) visualisations to describe aspects of collaborative writing. These two techniques form a data mining toolbox for collaborative writing by using process mining, probabilistic graphical models, and text mining. First, I created a framework, WriteProc, for investigating collaborative writing processes, integrated with the existing cloud computing writing tools in Google Docs. Secondly, I created new heuristic to extract the semantic nature of text edits that occur in the document revisions and automatically identify the corresponding writing activities. Thirdly, based on sequences of writing activities, I propose methods to discover the writing process models and transitional state diagrams using a process mining algorithm, Heuristics Miner, and Hidden Markov Models, respectively. Finally, I designed three types of visualisations and made contributions to their underlying techniques for analysing writing processes. All components of the toolbox are validated against annotated writing activities of real documents and a synthetic dataset. I also illustrate how the automatically discovered process models and visualisations are used in the process analysis with real documents written by groups of graduate students. I discuss how the analyses can be used to gain further insight into how students work and create their collaborative documents

    Tracking interacting targets in multi-modal sensors

    Get PDF
    PhDObject tracking is one of the fundamental tasks in various applications such as surveillance, sports, video conferencing and activity recognition. Factors such as occlusions, illumination changes and limited field of observance of the sensor make tracking a challenging task. To overcome these challenges the focus of this thesis is on using multiple modalities such as audio and video for multi-target, multi-modal tracking. Particularly, this thesis presents contributions to four related research topics, namely, pre-processing of input signals to reduce noise, multi-modal tracking, simultaneous detection and tracking, and interaction recognition. To improve the performance of detection algorithms, especially in the presence of noise, this thesis investigate filtering of the input data through spatio-temporal feature analysis as well as through frequency band analysis. The pre-processed data from multiple modalities is then fused within Particle filtering (PF). To further minimise the discrepancy between the real and the estimated positions, we propose a strategy that associates the hypotheses and the measurements with a real target, using a Weighted Probabilistic Data Association (WPDA). Since the filtering involved in the detection process reduces the available information and is inapplicable on low signal-to-noise ratio data, we investigate simultaneous detection and tracking approaches and propose a multi-target track-beforedetect Particle filtering (MT-TBD-PF). The proposed MT-TBD-PF algorithm bypasses the detection step and performs tracking in the raw signal. Finally, we apply the proposed multi-modal tracking to recognise interactions between targets in regions within, as well as outside the cameras’ fields of view. The efficiency of the proposed approaches are demonstrated on large uni-modal, multi-modal and multi-sensor scenarios from real world detections, tracking and event recognition datasets and through participation in evaluation campaigns

    Automatic annotation of musical audio for interactive applications

    Get PDF
    PhDAs machines become more and more portable, and part of our everyday life, it becomes apparent that developing interactive and ubiquitous systems is an important aspect of new music applications created by the research community. We are interested in developing a robust layer for the automatic annotation of audio signals, to be used in various applications, from music search engines to interactive installations, and in various contexts, from embedded devices to audio content servers. We propose adaptations of existing signal processing techniques to a real time context. Amongst these annotation techniques, we concentrate on low and mid-level tasks such as onset detection, pitch tracking, tempo extraction and note modelling. We present a framework to extract these annotations and evaluate the performances of different algorithms. The first task is to detect onsets and offsets in audio streams within short latencies. The segmentation of audio streams into temporal objects enables various manipulation and analysis of metrical structure. Evaluation of different algorithms and their adaptation to real time are described. We then tackle the problem of fundamental frequency estimation, again trying to reduce both the delay and the computational cost. Different algorithms are implemented for real time and experimented on monophonic recordings and complex signals. Spectral analysis can be used to label the temporal segments; the estimation of higher level descriptions is approached. Techniques for modelling of note objects and localisation of beats are implemented and discussed. Applications of our framework include live and interactive music installations, and more generally tools for the composers and sound engineers. Speed optimisations may bring a significant improvement to various automated tasks, such as automatic classification and recommendation systems. We describe the design of our software solution, for our research purposes and in view of its integration within other systems.EU-FP6-IST-507142 project SIMAC (Semantic Interaction with Music Audio Contents); EPSRC grants GR/R54620; GR/S75802/01
    corecore