Linear Regression and Unsupervised Learning For Tracking and Embodied Robot Control.

Computer vision problems, such as tracking and robot navigation, tend to be solved using models of the objects of interest to the problem. These models are often either hard-coded, or learned in a supervised manner. In either case, an engineer is required to identify the visual information that is important to the task, which is both time consuming and problematic. Issues with these engineered systems relate to the ungrounded nature of the knowledge imparted by the engineer, where the systems have no meaning attached to the representations. This leads to systems that are brittle and are prone to failure when expected to act in environments not envisaged by the engineer. The work presented in this thesis removes the need for hard-coded or engineered models of either visual information representations or behaviour. This is achieved by developing novel approaches for learning from example, in both input (percept) and output (action) spaces. This approach leads to the development of novel feature tracking algorithms, and methods for robot control. Applying this approach to feature tracking, unsupervised learning is employed, in real time, to build appearance models of the target that represent the input space structure, and this structure is exploited to partition banks of computationally efficient, linear regression based target displacement estimators. This thesis presents the first application of regression based methods to the problem of simultaneously modeling and tracking a target object. The computationally efficient Linear Predictor (LP) tracker is investigated, along with methods for combining and weighting flocks of LP’s. The tracking algorithms developed operate with accuracy comparable to other state of the art online approaches and with a significant gain in computational efficiency. This is achieved as a result of two specific contributions. First, novel online approaches for the unsupervised learning of modes of target appearance that identify aspects of the target are introduced. Second, a general tracking framework is developed within which the identified aspects of the target are adaptively associated to subsets of a bank of LP trackers. This results in the partitioning of LP’s and the online creation of aspect specific LP flocks that facilitate tracking through significant appearance changes. Applying the approach to the percept action domain, unsupervised learning is employed to discover the structure of the action space, and this structure is used in the formation of meaningful perceptual categories, and to facilitate the use of localised input-output (percept-action) mappings. This approach provides a realisation of an embodied and embedded agent that organises its perceptual space and hence its cognitive process based on interactions with its environment. Central to the proposed approach is the technique of clustering an input-output exemplar set, based on output similarity, and using the resultant input exemplar groupings to characterise a perceptual category. All input exemplars that are coupled to a certain class of outputs form a category - the category of a given affordance, action or function. In this sense the formed perceptual categories have meaning and are grounded in the embodiment of the agent. The approach is shown to identify the relative importance of perceptual features and is able to solve percept-action tasks, defined only by demonstration, in previously unseen situations. Within this percept-action learning framework, two alternative approaches are developed. The first approach employs hierarchical output space clustering of point-to-point mappings, to achieve search efficiency and input and output space generalisation as well as a mechanism for identifying the important variance and invariance in the input space. The exemplar hierarchy provides, in a single structure, a mechanism for classifying previously unseen inputs and generating appropriate outputs. The second approach to a percept-action learning framework integrates the regression mappings used in the feature tracking domain, with the action space clustering and imitation learning techniques developed in the percept-action domain. These components are utilised within a novel percept-action data mining methodology, that is able to discover the visual entities that are important to a specific problem, and to map from these entities onto the action space. Applied to the robot control task, this approach allows for real-time generation of continuous action signals, without the use of any supervision or definition of representations or rules of behaviour

Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)

The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.Comment: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist1

Towards accurate multi-person pose estimation in the wild

In this thesis we are concerned with the problem of articulated human pose estimation and pose tracking in images and video sequences. Human pose estimation is a task of localising major joints of a human skeleton in natural images and is one of the most important visual recognition tasks in the scenes containing humans with numerous applications in robotics, virtual and augmented reality, gaming and healthcare among others. Articulated human pose tracking requires tracking multiple persons in the video sequence while simultaneously estimating full body poses. This task is important for analysing surveillance footage, activity recognition, sports analytics, etc. Most of the prior work focused on the pose estimation of single pre-localised humans whereas here we address a case with multiple people in real world images which entails several challenges such as person-person overlaps in highly crowded scenes, unknown number of people or people entering and leaving video sequences. The first contribution is a multi-person pose estimation algorithm based on the bottom-up detection-by-grouping paradigm. Unlike the widespread top-down approaches our method detects body joints and pairwise relations between them in a single forward pass of a convolutional neural network. Multi-person parsing is performed by optimizing a joint objective based on a multicut graph partitioning framework. Secondly, we extend our pose estimation approach to articulated multi-person pose tracking in videos. Our approach performs multi-target tracking and pose estimation in a holistic manner by optimising a single objective. We further simplify and refine the formulation which allows us to reach close to the real-time performance. Thirdly, we propose a large scale dataset and a benchmark for articulated multi-person tracking. It is the first dataset of video sequences comprising complex multi-person scenes and fully annotated tracks with 2D keypoints. Our fourth contribution is a method for estimating 3D body pose using on-body wearable cameras. Our approach uses a pair of downward facing, head-mounted cameras and captures an entire body. This egocentric approach is free of limitations of traditional setups with external cameras and can estimate body poses in very crowded environments. Our final contribution goes beyond human pose estimation and is in the field of deep learning of 3D object shapes. In particular, we address the case of reconstructing 3D objects from weak supervision. Our approach represents objects as 3D point clouds and is able to learn them with 2D supervision only and without requiring camera pose information at training time. We design a differentiable renderer of point clouds as well as a novel loss formulation for dealing with camera pose ambiguity.In dieser Arbeit behandeln wir das Problem der Schätzung und Verfolgung artikulierter menschlicher Posen in Bildern und Video-Sequenzen. Die Schätzung menschlicher Posen besteht darin die Hauptgelenke des menschlichen Skeletts in natürlichen Bildern zu lokalisieren und ist eine der wichtigsten Aufgaben der visuellen Erkennung in Szenen, die Menschen beinhalten. Sie hat zahlreiche Anwendungen in der Robotik, virtueller und erweiterter Realität, in Videospielen, in der Medizin und weiteren Bereichen. Die Verfolgung artikulierter menschlicher Posen erfordert die Verfolgung mehrerer Personen in einer Videosequenz bei gleichzeitiger Schätzung vollständiger Körperhaltungen. Diese Aufgabe ist besonders wichtig für die Analyse von Video-Überwachungsaufnahmen, Aktivitätenerkennung, digitale Sportanalyse etc. Die meisten vorherigen Arbeiten sind auf die Schätzung einzelner Posen vorlokalisierter Menschen fokussiert, wohingegen wir den Fall mehrerer Personen in natürlichen Aufnahmen betrachten. Dies bringt einige Herausforderungen mit sich, wie die Überlappung verschiedener Personen in dicht gedrängten Szenen, eine unbekannte Anzahl an Personen oder Personen die das Sichtfeld der Video-Sequenz verlassen oder betreten. Der erste Beitrag ist ein Algorithmus zur Schätzung der Posen mehrerer Personen, welcher auf dem Paradigma der Erkennung durch Gruppierung aufbaut. Im Gegensatz zu den verbreiteten Verfeinerungs-Ansätzen erkennt unsere Methode Körpergelenke and paarweise Beziehungen zwischen ihnen in einer einzelnen Vorwärtsrechnung eines faltenden neuronalen Netzwerkes. Die Gliederung in mehrere Personen erfolgt durch Optimierung einer gemeinsamen Zielfunktion, die auf dem Mehrfachschnitt-Problem in der Graphenzerlegung basiert. Zweitens erweitern wir unseren Ansatz zur Posen-Bestimmung auf das Verfolgen mehrerer Personen und deren Artikulation in Videos. Unser Ansatz führt eine Verfolgung mehrerer Ziele und die Schätzung der zugehörigen Posen in ganzheitlicher Weise durch, indem eine einzelne Zielfunktion optimiert wird. Desweiteren vereinfachen und verfeinern wir die Formulierung, was unsere Methode nah an Echtzeit-Leistung bringt. Drittens schlagen wir einen großen Datensatz und einen Bewertungsmaßstab für die Verfolgung mehrerer artikulierter Personen vor. Dies ist der erste Datensatz der Video-Sequenzen von komplexen Szenen mit mehreren Personen beinhaltet und deren Spuren komplett mit zwei-dimensionalen Markierungen der Schlüsselpunkte versehen sind. Unser vierter Beitrag ist eine Methode zur Schätzung von drei-dimensionalen Körperhaltungen mittels am Körper tragbarer Kameras. Unser Ansatz verwendet ein Paar nach unten gerichteter, am Kopf befestigter Kameras und erfasst den gesamten Körper. Dieser egozentrische Ansatz ist frei von jeglichen Limitierungen traditioneller Konfigurationen mit externen Kameras und kann Körperhaltungen in sehr dicht gedrängten Umgebungen bestimmen. Unser letzter Beitrag geht über die Schätzung menschlicher Posen hinaus in den Bereich des tiefen Lernens der Gestalt von drei-dimensionalen Objekten. Insbesondere befassen wir uns mit dem Fall drei-dimensionale Objekte unter schwacher Überwachung zu rekonstruieren. Unser Ansatz repräsentiert Objekte als drei-dimensionale Punktwolken and ist im Stande diese nur mittels zwei-dimensionaler Überwachung und ohne Informationen über die Kamera-Ausrichtung zur Trainingszeit zu lernen. Wir entwerfen einen differenzierbaren Renderer für Punktwolken sowie eine neue Formulierung um mit uneindeutigen Kamera-Ausrichtungen umzugehen

Species partitioning in a temperate mountain chain: Segregation by habitat vs. interspecific competition

Disentangling the relative influence of the environment and biotic interactions in determining species coexistence patterns is a major challenge in ecology. The zonation occurring along elevation gradients, or at bioclimatic contact zones, offers a good opportunity to improve such understanding because the small scale at which the partitioning occurs facilitates inference based on experiments and ecological modelling. We studied the influence of abiotic gradients, habitat types, and interspecific competition in determining the spatial turnover between two pipit and two bunting species in NW Spain. We explored two independent lines of evidence to draw inference about the relative importance of environment and biotic interactions in driving range partitioning along elevation, latitude, and longitude. We combined occurrence data with environmental data to develop joint species distribution models (JSDM), in order to attribute co‐occurrence (or exclusion) to shared (or divergent) environmental responses and to interactions (attraction or exclusion). In the same region, we tested for interference competition by means of playback experiments in the contact zone. The JSDMs highlighted different responses for the two species pairs, although we did not find direct evidence of interspecific aggressiveness in our playback experiments. In pipits, partitioning was explained by divergent climate and habitat requirements and also by the negative correlations between species not explained by the environment. This significant residual correlation may reflect forms of competition others than direct interference, although we could not completely exclude the influence of unmeasured environmental predictors. When bunting species co‐occurred, it was because of shared habitat preferences, and a possible limitation to dispersal might cause their partitioning. Our results indicate that no single mechanism dominates in driving the distribution of our study species, but rather distributions are determined by the combination of many small forces including biotic and abiotic determinants of niche, whose relative strengths varied among species.Ministerio de Ciencia e Innovación, Grant/Award Number: BES-2012-053472, CGL2008-02749, CGL2011-28177 and CGL2014-53899-P; Fundación Biodiversidad; ARC Future Fellowship, Grant/Award Number: FT100100819; REMEDINAL3-CM, Grant/Award Number: P2013/MAE-2719Peer reviewe

Appl Ergon

Load carriage induces systematic alterations in gait patterns and pelvic-thoracic coordination. Leveraging this information, the objective of this study was to develop and assess a statistical prediction algorithm that uses body-worn inertial sensor data for classifying load carrying modes and load levels. Nine men participated in an experiment carrying a hand load in four modes: one-handed right and left carry, and two-handed side and anterior carry, each at 50% and 75% of the participant's maximum acceptable weight of carry, and a no-load reference condition. Twelve gait parameters calculated from inertial sensor data for each gait cycle, including gait phase durations, torso and pelvis postural sway, and thoracic-pelvic coordination were used as predictors in a two-stage hierarchical random forest classification model with Bayesian inference. The model correctly classified 96.9% of the carrying modes and 93.1% of the load levels. Coronal thoracic-pelvic coordination and pelvis postural sway were the most relevant predictors although their relative importance differed between carrying mode and load level prediction models. This study presents an algorithmic framework for combining inertial sensing with statistical prediction with potential use for quantifying physical exposures from load carriage.90IF0094/ACL/ACL HHS/United StatesT42 OH008455/OH/NIOSH CDC HHS/United States2020-03-18T00:00:00Z30642513PMC70792017374vault:3512

A new method for earthquake-induced damage identification in historic masonry towers combining OMA and IDA

AbstractThis paper presents a novel method for rapidly addressing the earthquake-induced damage identification task in historic masonry towers. The proposed method, termed DORI, combines operational modal analysis (OMA), FE modeling, rapid surrogate modeling (SM) and non-linear Incremental dynamic analysis (IDA). While OMA-based Structural Health Monitoring methods using statistical pattern recognition are known to allow the detection of small structural damages due to earthquakes, even far-field ones of moderate intensity, the combination of SM and IDA-based methods for damage localization and quantification is here proposed. The monumental bell tower of the Basilica of San Pietro located in Perugia, Italy, is considered for the validation of the method. While being continuously monitored since 2014, the bell tower experienced the main shocks of the 2016 Central Italy seismic sequence and the on-site vibration-based monitoring system detected changes in global dynamic behavior after the earthquakes. In the paper, experimental vibration data (continuous and seismic records), FE models and surrogate models of the structure are used for post-earthquake damage localization and quantification exploiting an ideal subdivision of the structure into meaningful macroelements. Results of linear and non-linear numerical modeling (SM and IDA, respectively) are successfully combined to this aim and the continuous exchange of information between the physical reality (monitoring data) and the virtual models (FE models and surrogate models) effectively enforces the Digital Twin paradigm. The earthquake-induced damage identified by both data-driven and model-based strategies is finally confirmed by in-situ visual inspections

Niche Evolution Along a Gradient of Ecological Specialization

The concept of the ecological niche is fundamental to understanding constraints on species distributions in space and time, and in explaining the origin and maintenance of biodiversity. A niche can be broadly defined to include all of the biotic and abiotic conditions that a species requires to persist. Niche breadth, or the degree of specialization, may influence how labile a species niche is, which can have broad implications for species ability to adapt to environmental change, and for explaining patterns of diversification. I investigated mechanism that facilitate or constrain niche evolution at multiple scales. First, I developed an index of specialization in bill morphology using museum specimens across a diverse New World Passerine clade. I used this index of specialization to evaluate the relative influence of geographic and ecological niche partitioning on speciation rates across islands and continents. I then examined evolutionary transition rates among generalist and specialist bill morphotypes to determine if specialization constrains further evolution over long time scales, thus creating an evolutionary dead end. My results suggest that specialization increases speciation rates, and that niche expansion allowing transitions from specialist back to a more generalist bill morphology were common. I further explored mechanisms that drive these broad scale patters by examining patterns of intraspecific niche partitioning in closely related tidal marsh passerines. I found that habitat characteristics that reflected a salinity gradient best explained parallel patterns of bill size divergence among populations of two closely related sparrow species. Lastly, I examined if the definition of specialization varies across niche axes. We found that niche breadth, or the degree of specialization, is correlated among functional, environmental, and competition axes among five species of Passerelid sparrows. By examining the influence of specialization on macroevolutionary patterns of diversification and patterns of niche partitioning within species we gain a more comprehensive understanding of how niches evolve across different temporal and taxonomic scales. I found specialization is associated with increased speciation rates that influence continental-scale patterns of diversification. I also provide evidence that specialists retain the potential for niche expansion at the species and population scale. Patterns of intraspecific niche partitioning along habitat gradient presented here also increase our understanding of how species might adapt to change at scales that are applicable to local conservation. My results suggest strategies to incorporate a diversity of habitat characteristics may be beneficial for conserving intraspecific variation and adaptive capacity of specialist species

Generalisable FPCA-based Models for Predicting Peak Power in Vertical Jumping using Accelerometer Data

Peak power in the countermovement jump is correlated with various measures of sports performance and can be used to monitor athlete training. The gold standard method for determining peak power uses force platforms, but they are unsuitable for field-based testing favoured by practitioners. Alternatives include predicting peak power from jump flight times, or using Newtonian methods based on body-worn inertial sensor data, but so far neither has yielded sufficiently accurate estimates. This thesis aims to develop a generalisable model for predicting peak power based on Functional Principal Component Analysis applied to body-worn accelerometer data. Data was collected from 69 male and female adults, engaged in sports at recreational, club or national levels. They performed up to 16 countermovement jumps each, with and without arm swing, 696 jumps in total. Peak power criterion measures were obtained from force platforms, and characteristic features from accelerometer data were extracted from four sensors attached to the lower back, upper back and both shanks. The best machine learning algorithm, jump type and sensor anatomical location were determined in this context. The investigation considered signal representation (resultant, triaxial or a suitable transform), preprocessing (smoothing, time window and curve registration), feature selection and data augmentation (signal rotations and SMOTER). A novel procedure optimised the model parameters based on Particle Swarm applied to a surrogate Gaussian Process model. Model selection and evaluation were based on nested cross validation (Monte Carlo design). The final optimal model had an RMSE of 2.5 W·kg-1, which compares favourably to earlier research (4.9 ± 1.7 W·kg-1 for flight-time formulae and 10.7 ± 6.3 W·kg-1 for Newtonian sensor-based methods). Whilst this is not yet sufficiently accurate for applied practice, this thesis has developed and comprehensively evaluated new techniques, which will be valuable to future biomechanical applications

Physically Explainable Deep Learning for Convective Initiation Nowcasting Using GOES-16 Satellite Observations

Convection initiation (CI) nowcasting remains a challenging problem for both numerical weather prediction models and existing nowcasting algorithms. In this study, object-based probabilistic deep learning models are developed to predict CI based on multichannel infrared GOES-R satellite observations. The data come from patches surrounding potential CI events identified in Multi-Radar Multi-Sensor Doppler weather radar products over the Great Plains region from June and July 2020 and June 2021. An objective radar-based approach is used to identify these events. The deep learning models significantly outperform the classical logistic model at lead times up to 1 hour, especially on the false alarm ratio. Through case studies, the deep learning model exhibits the dependence on the characteristics of clouds and moisture at multiple levels. Model explanation further reveals the model's decision-making process with different baselines. The explanation results highlight the importance of moisture and cloud features at different levels depending on the choice of baseline. Our study demonstrates the advantage of using different baselines in further understanding model behavior and gaining scientific insights