Keeping track of worm trackers

C. elegans is used extensively as a model system in the neurosciences due to its well defined nervous system. However, the seeming simplicity of this nervous system in anatomical structure and neuronal connectivity, at least compared to higher animals, underlies a rich diversity of behaviors. The usefulness of the worm in genome-wide mutagenesis or RNAi screens, where thousands of strains are assessed for phenotype, emphasizes the need for computational methods for automated parameterization of generated behaviors. In addition, behaviors can be modulated upon external cues like temperature, O2 and CO2 concentrations, mechanosensory and chemosensory inputs. Different machine vision tools have been developed to aid researchers in their efforts to inventory and characterize defined behavioral “outputs”. Here we aim at providing an overview of different worm-tracking packages or video analysis tools designed to quantify different aspects of locomotion such as the occurrence of directional changes (turns, omega bends), curvature of the sinusoidal shape (amplitude, body bend angles) and velocity (speed, backward or forward movement)

Automatic Scaffolding Productivity Measurement through Deep Learning

This study developed a method to automatically measure scaffolding productivity by extracting and analysing semantic information from onsite vision data

Video-Based Human Activity Recognition Using Deep Learning Approaches

Due to its capacity to gather vast, high-level data about human activity from wearable or stationary sensors, human activity recognition substantially impacts people’s day-to-day lives. Multiple people and things may be seen acting in the video, dispersed throughout the frame in various places. Because of this, modeling the interactions between many entities in spatial dimensions is necessary for visual reasoning in the action recognition task. The main aim of this paper is to evaluate and map the current scenario of human actions in red, green, and blue videos, based on deep learning models. A residual network (ResNet) and a vision transformer architecture (ViT) with a semi-supervised learning approach are evaluated. The DINO (self-DIstillation with NO labels) is used to enhance the potential of the ResNet and ViT. The evaluated benchmark is the human motion database (HMDB51), which tries to better capture the richness and complexity of human actions. The obtained results for video classification with the proposed ViT are promising based on performance metrics and results from the recent literature. The results obtained using a bi-dimensional ViT with long short-term memory demonstrated great performance in human action recognition when applied to the HMDB51 dataset. The mentioned architecture presented 96.7 ± 0.35% and 41.0 ± 0.27% in terms of accuracy (mean ± standard deviation values) in the train and test phases of the HMDB51 dataset, respectively

Design Of Computer Vision Systems For Optimizing The Threat Detection Accuracy

This dissertation considers computer vision (CV) systems in which a central monitoring station receives and analyzes the video streams captured and delivered wirelessly by multiple cameras. It addresses how the bandwidth can be allocated to various cameras by presenting a cross-layer solution that optimizes the overall detection or recognition accuracy. The dissertation presents and develops a real CV system and subsequently provides a detailed experimental analysis of cross-layer optimization. Other unique features of the developed solution include employing the popular HTTP streaming approach, utilizing homogeneous cameras as well as heterogeneous ones with varying capabilities and limitations, and including a new algorithm for estimating the effective medium airtime. The results show that the proposed solution significantly improves the CV accuracy. Additionally, the dissertation features an improved neural network system for object detection. The proposed system considers inherent video characteristics and employs different motion detection and clustering algorithms to focus on the areas of importance in consecutive frames, allowing the system to dynamically and efficiently distribute the detection task among multiple deployments of object detection neural networks. Our experimental results indicate that our proposed method can enhance the mAP (mean average precision), execution time, and required data transmissions to object detection networks. Finally, as recognizing an activity provides significant automation prospects in CV systems, the dissertation presents an efficient activity-detection recurrent neural network that utilizes fast pose/limbs estimation approaches. By combining object detection with pose estimation, the domain of activity detection is shifted from a volume of RGB (Red, Green, and Blue) pixel values to a time-series of relatively small one-dimensional arrays, thereby allowing the activity detection system to take advantage of highly capable neural networks that have been trained on large GPU clusters for thousands of hours. Consequently, capable activity detection systems with considerably fewer training sets and processing hours can be built

Computational ethology for primate sociality: a novel paradigm for computer-vision-based analysis of animal behaviour

Research in the biological and wildlife sciences is increasingly reliant on video data for measuring animal behaviour, however large-scale analysis is often limited by the time and resources it takes to process video archives. Computer vision holds serious potential to unlock these datasets to analyse behaviour at an unprecedented level of scale, depth and reliability, however thus far a framework for processing and analysing behaviour from large-scale video datasets is lacking. This thesis attempts to solve this problem by developing the theory and methods for capturing long-term sociality of animal populations from longitudinal video archives, laying the foundations for an emerging field; computational ethology of animals in the wild. It makes several key contributions by a) establishing the first unified longitudinal video dataset of wild chimpanzee stone tool use across a 30 year period, and building a framework for collaborative research using cloud-technology b) developing a set of computational tools to allow for processing of large volumes of video data for automated individual identification and behaviour recognition c) applying these automated methods to validate use for social network analysis and d) measuring the social dynamics and behaviour of a group of wild chimpanzees living in the forest of Bossou, Guinea, West Africa. In Chapter 1 I introduce the theoretical and historical context for the thesis, and outline the novel methodological framework for using computer vision to measure animal social behaviour in video. In Chapter 2 I introduce the methodology for processing and managing a longitudinal video archive, and future directions for a new framework for collaborative research workflows in the wildlife sciences using cloud technology. In Chapter 3 I lay the foundations of this framework for analysing behaviour and unlocking video datasets, using deep learning and face recognition. In Chapter 4 I evaluate the robustness of the method for modelling long-term sociality and social networks at Bossou and test whether life history variables predict individual-level sociality patterns. In Chapter 5 I introduce the final component to this framework for measuring long-term animal behaviour, through audiovisual behavioural recognition of chimpanzee nut-cracking. In my final chapter (6) I discuss the main contributions, limitations and future directions for research. Overall this thesis integrates a diverse range of interdisciplinary methods and concepts from primatology, ethology, engineering, and computer vision, to build the foundations for further exploration of cognition, ecology and evolution in wild animals using automated methods

Automatic visual detection of human behavior: a review from 2000 to 2014

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviors from video is a very recent research topic. In this paper, we perform a systematic and recent literature review on this topic, from 2000 to 2014, covering a selection of 193 papers that were searched from six major scientific publishers. The selected papers were classified into three main subjects: detection techniques, datasets and applications. The detection techniques were divided into four categories (initialization, tracking, pose estimation and recognition). The list of datasets includes eight examples (e.g., Hollywood action). Finally, several application areas were identified, including human detection, abnormal activity detection, action recognition, player modeling and pedestrian detection. Our analysis provides a road map to guide future research for designing automatic visual human behavior detection systems.This work is funded by the Portuguese Foundation for Science and Technology (FCT - Fundacao para a Ciencia e a Tecnologia) under research Grant SFRH/BD/84939/2012