Tracking interacting targets in multi-modal sensors

PhDObject tracking is one of the fundamental tasks in various applications such as surveillance, sports, video conferencing and activity recognition. Factors such as occlusions, illumination changes and limited field of observance of the sensor make tracking a challenging task. To overcome these challenges the focus of this thesis is on using multiple modalities such as audio and video for multi-target, multi-modal tracking. Particularly, this thesis presents contributions to four related research topics, namely, pre-processing of input signals to reduce noise, multi-modal tracking, simultaneous detection and tracking, and interaction recognition. To improve the performance of detection algorithms, especially in the presence of noise, this thesis investigate filtering of the input data through spatio-temporal feature analysis as well as through frequency band analysis. The pre-processed data from multiple modalities is then fused within Particle filtering (PF). To further minimise the discrepancy between the real and the estimated positions, we propose a strategy that associates the hypotheses and the measurements with a real target, using a Weighted Probabilistic Data Association (WPDA). Since the filtering involved in the detection process reduces the available information and is inapplicable on low signal-to-noise ratio data, we investigate simultaneous detection and tracking approaches and propose a multi-target track-beforedetect Particle filtering (MT-TBD-PF). The proposed MT-TBD-PF algorithm bypasses the detection step and performs tracking in the raw signal. Finally, we apply the proposed multi-modal tracking to recognise interactions between targets in regions within, as well as outside the cameras’ fields of view. The efficiency of the proposed approaches are demonstrated on large uni-modal, multi-modal and multi-sensor scenarios from real world detections, tracking and event recognition datasets and through participation in evaluation campaigns

Acoustic Speaker Localization with Strong Reverberation and Adaptive Feature Filtering with a Bayes RFS Framework

The thesis investigates the challenges of speaker localization in presence of strong reverberation, multi-speaker tracking, and multi-feature multi-speaker state filtering, using sound recordings from microphones. Novel reverberation-robust speaker localization algorithms are derived from the signal and room acoustics models. A multi-speaker tracking filter and a multi-feature multi-speaker state filter are developed based upon the generalized labeled multi-Bernoulli random finite set framework. Experiments and comparative studies have verified and demonstrated the benefits of the proposed methods

From Parsed Corpora to Semantically Related Verbs

A comprehensive repository of semantic relations between verbs is of great importance in supporting a large area of natural language applications. The aim of this paper is to automatically generate a repository of semantic relations between verb pairs using Distributional Memory (DM), a state-of-the-art framework for distributional semantics. The main idea of our method is to exploit relationships that are expressed through prepositions between a verbal and a nominal event in text to extract semantically related events. Then using these prepositions, we derive relation types including causal, temporal, comparison, and expansion. The result of our study leads to the construction of a resource for semantic relations, which consists of pairs of verbs associated with their probable arguments and significance scores based on our measures. Experimental evaluations show promising results on the task of extracting and categorising semantic relations between verbs

Civic engagement in young adulthood: Social capital and the mediating effects of postsecondary educational attainment

Civic Engagement has two purposes in American society. The first is to maintain democracy and democratic institutions; the second is to serve as a pathway for maturation into adulthood. Utilizing data from The National Longitudinal Study of Adolescent to Adult Health (Add Health), we investigate how adolescent experiences with schools and families impact young adult’s political (voting) and nonpolitical (volunteering) civic engagement and the mediating role postsecondary educational attainment may have in this process. Utilizing measures of adolescent bonding and bridging social capital, as well as human and financial capital, this investigation takes a life course perspective and relies heavily on theories of capital and emerging adulthood. Our study examines relationships between adolescence, emerging adulthood, and young adulthood with a sample of 6,872 respondents drawn from Waves I and IV of Add Health with a structural equation model and 10,000 bootstrap samples to test for mediation. We investigated these political and nonpolitical civic engagement. Our study used nationally representative, longitudinal data, to account for multiple important developmental pre-collegiate factors, assessed civic engagement in young adulthood, and included individuals who did not attend and/or complete higher education as well as those who did. In short, we found unique relationships exist for political and non-political civic engagement. Behavioral bonding and bridging social capital demonstrated direct and indirect effects to civic engagement in young adulthood. Postsecondary education was the strongest predictor civic engagement in young adulthood, suggesting greater levels of education are a powerful social structure to prepare members of society for civic participation