Audio-visual tracking of concurrent speakers

Audio-visual tracking of an unknown number of concurrent speakers in 3D is a challenging task, especially when sound and video are collected with a compact sensing platform. In this paper, we propose a tracker that builds on generative and discriminative audio-visual likelihood models formulated in a particle filtering framework. We localize multiple concurrent speakers with a de-emphasized acoustic map assisted by the image detection-derived 3D video observations. The 3D multimodal observations are either assigned to existing tracks for discriminative likelihood computation or used to initialize new tracks. The generative likelihoods rely on color distribution of the target and the de-emphasized acoustic map value. Experiments on AV16.3 and CAV3D datasets show that the proposed tracker outperforms the uni-modal trackers and the state-of-the-art approaches both in 3D and on the image plane

Bounded Influence Approaches to Constrained Mixed Vector Autoregressive Models

The proliferation of many clinical studies obtaining multiple biophysical signals from several individuals repeatedly in time is increasingly recognized, a recognition generating growth in statistical models that analyze cross-sectional time series data. In general, these statistical models try to answer two questions: (i) intra-individual dynamics of the response and its relation to some covariates; and, (ii) how this dynamics can be aggregated consistently in a group. In response to the first question, we propose a covariate-adjusted constrained Vector Autoregressive model, a technique similar to the STARMAX model (Stoffer, JASA 81, 762-772), to describe serial dependence of observations. In this way, the number of parameters to be estimated is kept minimal while offering flexibility for the model to explore higher order dependence. In response to (ii), we use mixed effects analysis that accommodates modelling of heterogeneity among cross-sections arising from covariate effects that vary from one cross-section to another. Although estimation of the model can proceed using standard maximum likelihood techniques, we believed it is advantageous to use bounded influence procedures in the modelling (such as choosing constraints) and parameter estimation so that the effects of outliers can be controlled. In particular, we use M-estimation with a redescending bounding function because its influence function is always bounded. Furthermore, assuming consistency, this influence function is useful to obtain the limiting distribution of the estimates. However, this distribution may not necessarily yield accurate inference in the presence of contamination as the actual asymptotic distribution might have wider tails. This led us to investigate bootstrap approximation techniques. A sampling scheme based on IID innovations is modified to accommodate the cross-sectional structure of the data. Then the M-estimation is applied to each bootstrap sample naively to obtain the asymptotic distribution of the estimates.We apply these strategies to the extracted BOLD activation from several regions of the brain from a group of individuals to describe joint dynamic behavior between these locations. We used simulated data with both innovation and additive outliers to test whether the estimation procedure is accurate despite contamination

Computer Vision for Fish Monitoring: Challenges and Possibilities

This master's thesis focuses on the evaluation and exploration of detection and tracking algorithms for fish in a dense underwater environment. The primary objectives were to achieve precise and accurate fish detection and to track fish over an extended period. The thesis explores the performance of two object detection algorithms, YOLOv4 and YOLOv8, as well as their integration with the DeepSORT tracking algorithm. The algorithms were trained and evaluated using a dataset collected from a densely populated underwater fish tank. The dataset was manually annotated using bounding box annotation techniques to accurately label the objects of interest. The results demonstrated the effectiveness of both YOLOv4 and YOLOv8 in detecting fish in densely populated environments. However, YOLOv8 achieved a significantly higher mAP50-95 score, indicating better localization and detection accuracy. It proved more adept at precisely locating the position of detected fish, leading to improved overall detection performance. In terms of fish tracking the combination of DeepSORT and YOLOv8 showed the best overall performance, as evidenced by higher MOTA and IDF1 scores, and lower MOTP scores. However, tracking individual fish over extended periods presented challenges due to occlusions and rapid trajectory changes, leading to a high number of identity switches. By evaluating and exploring the effectiveness of detection and tracking algorithms, this thesis contributes to the advancement of fish monitoring techniques in aquaculture. The findings provide valuable insights into the performance of YOLOv4 and YOLOv8 and the potential of DeepSORT for accurate and reliable fish detection and tracking. The results and methodologies presented in this study lay the groundwork for further research and development in the field, aiming to enhance fish welfare, optimize resource management, and improve efficiency in aquaculture practices

Orbital transfer vehicle launch operations study: Automated technology knowledge base, volume 4

A simplified retrieval strategy for compiling automation-related bibliographies from NASA/RECON is presented. Two subsets of NASA Thesaurus subject terms were extracted: a primary list, which is used to obtain an initial set of citations; and a secondary list, which is used to limit or further specify a large initial set of citations. These subject term lists are presented in Appendix A as the Automated Technology Knowledge Base (ATKB) Thesaurus