AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild

Animals are capable of extreme agility, yet understanding their complex dynamics, which have ecological, biomechanical and evolutionary implications, remains challenging. Being able to study this incredible agility will be critical for the development of next-generation autonomous legged robots. In particular, the cheetah (acinonyx jubatus) is supremely fast and maneuverable, yet quantifying its whole-body 3D kinematic data during locomotion in the wild remains a challenge, even with new deep learning-based methods. In this work we present an extensive dataset of free-running cheetahs in the wild, called AcinoSet, that contains 119,490 frames of multi-view synchronized high-speed video footage, camera calibration files and 7,588 human-annotated frames. We utilize markerless animal pose estimation to provide 2D keypoints. Then, we use three methods that serve as strong baselines for 3D pose estimation tool development: traditional sparse bundle adjustment, an Extended Kalman Filter, and a trajectory optimization-based method we call Full Trajectory Estimation. The resulting 3D trajectories, human-checked 3D ground truth, and an interactive tool to inspect the data is also provided. We believe this dataset will be useful for a diverse range of fields such as ecology, neuroscience, robotics, biomechanics as well as computer vision.Comment: Code and data can be found at: https://github.com/African-Robotics-Unit/AcinoSe

Biomechanical Locomotion Heterogeneity in Synthetic Crowds

Synthetic crowd simulation combines rule sets at different conceptual layers to represent the dynamic nature of crowds while adhering to basic principles of human steering, such as collision avoidance and goal completion. In this dissertation, I explore synthetic crowd simulation at the steering layer using a critical approach to define the central theme of the work, the impact of model representation and agent diversity in crowds. At the steering layer, simulated agents make regular decisions, or actions, related to steering which are often responsible for the emergent behaviours found in the macro-scale crowd. Because of this bottom-up impact of a steering model's defining rule-set, I postulate that biomechanics and diverse biomechanics may alter the outcomes of dynamic synthetic-crowds-based outcomes. This would mean that an assumption of normativity and/or homogeneity among simulated agents and their mobility would provide an inaccurate representation of a scenario. If these results are then used to make real world decisions, say via policy or design, then those populations not represented in the simulated scenario may experience a lack of representation in the actualization of those decisions. A focused literature review shows that applications of both biomechanics and diverse locomotion representation at this layer of modelling are very narrow and often not present. I respond to the narrowness of this representation by addressing both biomechanics and heterogeneity separately. To address the question of performance and importance of locomotion biomechanics in crowd simulation, I use a large scale comparative approach. The industry standard synthetic crowd models are tested under a battery of benchmarks derived from prior work in comparative analysis of synthetic crowds as well as new scenarios derived from built environments. To address the question of the importance of heterogeneity in locomotion biomechanics, I define tiers of impact in the multi-agent crowds model at the steering layer--from the action space, to the agent space, to the crowds space. To this end, additional models and layers are developed to address the modelling and application of heterogeneous locomotion biomechanics in synthetic crowds. The results of both studies form a research arc which shows that the biomechanics in steering models provides important fidelity in several applications and that heterogeneity in the model of locomotion biomechanics directly impacts both qualitative and quantitative synthetic crowds outcomes. As well, systems, approaches, and pitfalls regarding the analysis of steering model and human mobility diversity are described

Learning Human Behaviour Patterns by Trajectory and Activity Recognition

The world’s population is ageing, increasing the awareness of neurological and behavioural impairments that may arise from the human ageing. These impairments can be manifested by cognitive conditions or mobility reduction. These conditions are difficult to be detected on time, relying only on the periodic medical appointments. Therefore, there is a lack of routine screening which demands the development of solutions to better assist and monitor human behaviour. The available technologies to monitor human behaviour are limited to indoors and require the installation of sensors around the user’s homes presenting high maintenance and installation costs. With the widespread use of smartphones, it is possible to take advantage of their sensing information to better assist the elderly population. This study investigates the question of what we can learn about human pattern behaviour from this rich and pervasive mobile sensing data. A deployment of a data collection over a period of 6 months was designed to measure three different human routines through human trajectory analysis and activity recognition comprising indoor and outdoor environment. A framework for modelling human behaviour was developed using human motion features, extracted in an unsupervised and supervised manner. The unsupervised feature extraction is able to measure mobility properties such as step length estimation, user points of interest or even locomotion activities inferred from an user-independent trained classifier. The supervised feature extraction was design to be user-dependent as each user may have specific behaviours that are common to his/her routine. The human patterns were modelled through probability density functions and clustering approaches. Using the human learned patterns, inferences about the current human behaviour were continuously quantified by an anomaly detection algorithm, where distance measurements were used to detect significant changes in behaviour. Experimental results demonstrate the effectiveness of the proposed framework that revealed an increase potential to learn behaviour patterns and detect anomalies

Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras

Despite the fact that personal privacy has become a major concern, surveillance technology is now becoming ubiquitous in modern society. This is mainly due to the increasing number of crimes as well as the essential necessity to provide secure and safer environment. Recent research studies have confirmed now the possibility of recognizing people by the way they walk i.e. gait. The aim of this research study is to investigate the use of gait for people detection as well as identification across different cameras. We present a new approach for people tracking and identification between different non-intersecting un-calibrated stationary cameras based on gait analysis. A vision-based markerless extraction method is being deployed for the derivation of gait kinematics as well as anthropometric measurements in order to produce a gait signature. The novelty of our approach is motivated by the recent research in biometrics and forensic analysis using gait. The experimental results affirmed the robustness of our approach to successfully detect walking people as well as its potency to extract gait features for different camera viewpoints achieving an identity recognition rate of 73.6 % processed for 2270 video sequences. Furthermore, experimental results confirmed the potential of the proposed method for identity tracking in real surveillance systems to recognize walking individuals across different views with an average recognition rate of 92.5 % for cross-camera matching for two different non-overlapping views.<br/

Artifacts Mapping: Multi-Modal Semantic Mapping for Object Detection and 3D Localization

Geometric navigation is nowadays a well-established field of robotics and the research focus is shifting towards higher-level scene understanding, such as Semantic Mapping. When a robot needs to interact with its environment, it must be able to comprehend the contextual information of its surroundings. This work focuses on classifying and localising objects within a map, which is under construction (SLAM) or already built. To further explore this direction, we propose a framework that can autonomously detect and localize predefined objects in a known environment using a multi-modal sensor fusion approach (combining RGB and depth data from an RGB-D camera and a lidar). The framework consists of three key elements: understanding the environment through RGB data, estimating depth through multi-modal sensor fusion, and managing artifacts (i.e., filtering and stabilizing measurements). The experiments show that the proposed framework can accurately detect 98% of the objects in the real sample environment, without post-processing, while 85% and 80% of the objects were mapped using the single RGBD camera or RGB + lidar setup respectively. The comparison with single-sensor (camera or lidar) experiments is performed to show that sensor fusion allows the robot to accurately detect near and far obstacles, which would have been noisy or imprecise in a purely visual or laser-based approach.Comment: Accepted to the 11th European Conference on Mobile Robots (ECMR) 202

Switching operation modes in the neocortex via cholinergic neuromodulation

In order to deal with the uncertainty in the world, our brains need to be able to flexibly switch between the exploration of new sensory representations and exploitation of previously acquired ones. This requires forming accurate estimations of what and how much something is expected. While modeling has allowed for the development of several ways to form predictions, how the brain could implement those is still under debate. Here, we recognize acetylcholine as one of the main neuromodulators driving learning based on uncertainty, promoting the exploration of new sensory representations. We identify its interactions with cortical inhibitory interneurons and derive a biophysically grounded computational model able to capture and learn from uncertainty. This model allows us to understand inhibition beyond gain control by suggesting that different interneuron subtypes either encode predictions or estimate their uncertainty, facilitating detection of unexpected cues. Moreover, we show how acetylcholine-like neuromodulation uniquely interacts with global and local sources of inhibition, disrupting perceptual certainty and promoting the rapid acquisition of new perceptual cues. Altogether, our model proposes that cortical acetylcholine favors sensory exploration over exploitation in a cortical microcircuit dedicated to estimating sensory uncertainty