Data learning for human pose tracking

One of the most pressing problems in data-driven models is how to include latent data information in the model-building process. This could help to reduce the amount of data required for training in machine learning applications. Pose tracking is a field currently dominated by data-driven models and where the collection of large, labeled datasets is difficult and time intensive. We believe this is an application that could benefit significantly from the inclusion of structure in the interpretation of available data. Improving human pose tracking methods is significant, as existing methods are not robust enough for application in real-world scenarios such as remote physiotherapy or mobility monitoring. Data Assimilation and Machine Learning are both fields that allow a prediction to be made using a forecasting model. We propose to integrate methods from these two fields to improve the application of pose tracking methods in real-world scenarios. To the best of our knowledge, this is the first application of methods of this kind to the field of human pose tracking. In particular, this Thesis presents two areas of work applying Data Assimilation methods to human pose tracking. First, we show how to apply adjoint methods (from Data Assimilation (DA)) to 3D human pose tracking. This method successfully recovers 3D pose from only Inertial Measurement Units orientation data without the need for a learnt prior, joint limits or additional constraints. The method also does not require a full motion sequence for optimisation, allowing the algorithm to run online. The second collection of work is concerned with 2D human pose tracking in RGB images. Here we show three methods for Data Assimilation inspired modifications to traditional pose estimation methods. First, we apply a Kalman filter layer to traditional Convolutional Neural Network-based pose estimation methods to improve speed, consistency and accuracy of joint (or keypoint) location. Second, to additionally improve performance of these filters, we present a novel application of covariance extraction from feature heatmaps outputted by pose estimation Convolutional Neural Networks (CNNs). Third, we change the convolution function of the keypoint location network of the open-source pose estimation framework OpenPose to resemble a covariance calculation. There is significant potential for this work to continue to be applied in human pose tracking and other fields. To this end, the equations presented in this work are general to allow them to be applied to other areas with only minor modification.Open Acces

Real-time multi-person pose tracking using data assimilation

We propose a framework for the integration of data assimilation and machine learning methods in human pose estimation, with the aim of enabling any pose estimation method to be run in real-time, whilst also increasing consistency and accuracy. Data assimilation and machine learning are complementary methods: the former allows us to make use of information about the underlying dynamics of a system but lacks the flexibility of a data-based model, which we can instead obtain with the latter. Our framework presents a real-time tracking module for any single or multi-person pose estimation system. Specifically, tracking is performed by a number of Kalman filters initiated for each new person appearing in a motion sequence. This permits tracking of multiple skeletons and reduces the frequency that computationally expensive pose estimation has to be run, enabling online pose tracking. The module tracks for N frames while the pose estimates are calculated for frame (N+1). This also results in increased consistency of person identification and reduced inaccuracies due to missing joint locations and inversion of left-and right-side joints

Data learning: integrating data assimilation and machine learning

Data Assimilation (DA) is the approximation of the true state of some physical system by combining observations with a dynamic model. DA incorporates observational data into a prediction model to improve forecasted results. These models have increased in sophistication to better fit application requirements and circumvent implementation issues. Nevertheless, these approaches are incapable of fully overcoming their unrealistic assumptions. Machine Learning (ML) shows great capability in approximating nonlinear systems and extracting meaningful features from high-dimensional data. ML algorithms are capable of assisting or replacing traditional forecasting methods. However, the data used during training in any Machine Learning (ML) algorithm include numerical, approximation and round off errors, which are trained into the forecasting model. Integration of ML with DA increases the reliability of prediction by including information with a physical meaning. This work provides an introduction to Data Learning, a field that integrates Data Assimilation and Machine Learning to overcome limitations in applying these fields to real-world data. The fundamental equations of DA and ML are presented and developed to show how they can be combined into Data Learning. We present a number of Data Learning methods and results for some test cases, though the equations are general and can easily be applied elsewhere

Diabetes Affects Antibody Response to SARS-CoV-2 Vaccination in Older Residents of Long-term Care Facilities: Data From the GeroCovid Vax Study

Objective: Type 2 diabetes may affect the humoral immune response after vaccination, but data concerning coronavirus disease 19 (COVID-19) vaccines are scarce. We evaluated the impact of diabetes on antibody response to the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccination in older residents of long-term care facilities (LTCFs) and tested for differences according to antidiabetic treatment. Research design and methods: For this analysis, 555 older residents of LTCFs participating in the GeroCovid Vax study were included. SARS-CoV-2 trimeric S immunoglobulin G (anti-S IgG) concentrations using chemiluminescent assays were tested before the first dose and after 2 and 6 months. The impact of diabetes on anti-S IgG levels was evaluated using linear mixed models, which included the interaction between time and presence of diabetes. A second model also considered diabetes treatment: no insulin therapy (including dietary only or use of oral antidiabetic agents) and insulin therapy (alone or in combination with oral antidiabetic agents). Results: The mean age of the sample was 82.1 years, 68.1% were women, and 25.2% had diabetes. In linear mixed models, presence of diabetes was associated with lower anti-S IgG levels at 2 (β = -0.20; 95% CI -0.34, -0.06) and 6 months (β = -0.22; 95% CI -0.37, -0.07) after the first vaccine dose. Compared with those without diabetes, residents with diabetes not using insulin had lower IgG levels at 2- and 6-month assessments (β = -0.24; 95% CI -0.43, -0.05 and β = -0.30; 95% CI -0.50, -0.10, respectively), whereas no differences were observed for those using insulin. Conclusions: Older residents of LTCFs with diabetes tended to have weaker antibody response to COVID-19 vaccination. Insulin treatment might buffer this effect and establish humoral immunity similar to that in individuals without diabetes