Towards Automatic Human Body Model Fitting to a 3D Scan

This paper presents a method to automatically recover a realistic and accurate body shape of a person wearing clothing from a 3D scan. Indeed, in many practical situations, people are scanned wearing clothing. The underlying body shape is thus partially or completely occluded. Yet, it is very desirable to recover the shape of a covered body as it provides non-invasive means of measuring and analysing it. This is particularly convenient for patients in medical applications, customers in a retail shop, as well as in security applications where suspicious objects under clothing are to be detected. To recover the body shape from the 3D scan of a person in any pose, a human body model is usually fitted to the scan. Current methods rely on the manual placement of markers on the body to identify anatomical locations and guide the pose fitting. The markers are either physically placed on the body before scanning or placed in software as a postprocessing step. Some other methods detect key points on the scan using 3D feature descriptors to automate the placement of markers. They usually require a large database of 3D scans. We propose to automatically estimate the body pose of a person from a 3D mesh acquired by standard 3D body scanners, with or without texture. To fit a human model to the scan, we use joint locations as anchors. These are detected from multiple 2D views using a conventional body joint detector working on images. In contrast to existing approaches, the proposed method is fully automatic, and takes advantage of the robustness of state-of-art 2D joint detectors. The proposed approach is validated on scans of people in different poses wearing garments of various thicknesses and on scans of one person in multiple poses with known ground truth wearing close-fitting clothing

BODYFITR: Robust Automatic 3D Human Body Fitting

This paper proposes BODYFITR, a fully automatic method to fit a human body model to static 3D scans with complex poses. Automatic and reliable 3D human body fitting is necessary for many applications related to healthcare, digital ergonomics, avatar creation and security, especially in industrial contexts for large-scale product design. Existing works either make prior assumptions on the pose, require manual annotation of the data or have difficulty handling complex poses. This work addresses these limitations by providing a novel automatic fitting pipeline with carefully integrated building blocks designed for a systematic and robust approach. It is validated on the 3DBodyTex dataset, with hundreds of high-quality 3D body scans, and shown to outperform prior works in static body pose and shape estimation, qualitatively and quantitatively. The method is also applied to the creation of realistic 3D avatars from the high-quality texture scans of 3DBodyTex, further demonstrating its capabilities

3DBodyTex: Textured 3D Body Dataset

In this paper, a dataset, named 3DBodyTex, of static 3D body scans with high-quality texture information is presented along with a fully automatic method for body model fitting to a 3D scan. 3D shape modelling is a fundamental area of computer vision that has a wide range of applications in the industry. It is becoming even more important as 3D sensing technologies are entering consumer devices such as smartphones. As the main output of these sensors is the 3D shape, many methods rely on this information alone. The 3D shape information is, however, very high dimensional and leads to models that must handle many degrees of freedom from limited information. Coupling texture and 3D shape alleviates this burden, as the texture of 3D objects is complementary to their shape. Unfortunately, high-quality texture content is lacking from commonly available datasets, and in particular in datasets of 3D body scans. The proposed 3DBodyTex dataset aims to fill this gap with hundreds of high-quality 3D body scans with high-resolution texture. Moreover, a novel fully automatic pipeline to fit a body model to a 3D scan is proposed. It includes a robust 3D landmark estimator that takes advantage of the high-resolution texture of 3DBodyTex. The pipeline is applied to the scans, and the results are reported and discussed, showcasing the diversity of the features in the dataset

Effect of angiotensin-converting enzyme inhibitor and angiotensin receptor blocker initiation on organ support-free days in patients hospitalized with COVID-19

IMPORTANCE Overactivation of the renin-angiotensin system (RAS) may contribute to poor clinical outcomes in patients with COVID-19. Objective To determine whether angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) initiation improves outcomes in patients hospitalized for COVID-19. DESIGN, SETTING, AND PARTICIPANTS In an ongoing, adaptive platform randomized clinical trial, 721 critically ill and 58 non–critically ill hospitalized adults were randomized to receive an RAS inhibitor or control between March 16, 2021, and February 25, 2022, at 69 sites in 7 countries (final follow-up on June 1, 2022). INTERVENTIONS Patients were randomized to receive open-label initiation of an ACE inhibitor (n = 257), ARB (n = 248), ARB in combination with DMX-200 (a chemokine receptor-2 inhibitor; n = 10), or no RAS inhibitor (control; n = 264) for up to 10 days. MAIN OUTCOMES AND MEASURES The primary outcome was organ support–free days, a composite of hospital survival and days alive without cardiovascular or respiratory organ support through 21 days. The primary analysis was a bayesian cumulative logistic model. Odds ratios (ORs) greater than 1 represent improved outcomes. RESULTS On February 25, 2022, enrollment was discontinued due to safety concerns. Among 679 critically ill patients with available primary outcome data, the median age was 56 years and 239 participants (35.2%) were women. Median (IQR) organ support–free days among critically ill patients was 10 (–1 to 16) in the ACE inhibitor group (n = 231), 8 (–1 to 17) in the ARB group (n = 217), and 12 (0 to 17) in the control group (n = 231) (median adjusted odds ratios of 0.77 [95% bayesian credible interval, 0.58-1.06] for improvement for ACE inhibitor and 0.76 [95% credible interval, 0.56-1.05] for ARB compared with control). The posterior probabilities that ACE inhibitors and ARBs worsened organ support–free days compared with control were 94.9% and 95.4%, respectively. Hospital survival occurred in 166 of 231 critically ill participants (71.9%) in the ACE inhibitor group, 152 of 217 (70.0%) in the ARB group, and 182 of 231 (78.8%) in the control group (posterior probabilities that ACE inhibitor and ARB worsened hospital survival compared with control were 95.3% and 98.1%, respectively). CONCLUSIONS AND RELEVANCE In this trial, among critically ill adults with COVID-19, initiation of an ACE inhibitor or ARB did not improve, and likely worsened, clinical outcomes. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT0273570

Automatic Analysis, Representation and Reconstruction of Textured 3D Human Scans

Various practical applications in computer vision are related to the human body. These involve representing and modelling the body shape, pose, clothing and appearance with mathematical and statistical tools requiring datasets of examples, representative of the variation in the data. Three-dimensional (3D) data is especially important as it allows to simulate the physical world directly, for example to analyse and lift ambiguities in other prevalent data modalities, such as images. However, existing datasets of 3D human scans show limitations in their size, diversity, quality or annotation. This reduces their applicability in tackling research questions around the 3D human body. Two particular applications of interest that remain unanswered are the estimation of body shape under clothing, and the completion of textured shape of missing or defective data. This thesis proposes three main contributions. First, 3DBodyTex, a dataset of 3D human scans, which complements alternative datasets with real scans, body and clothing scans, hundreds of subjects, high-resolution texture information, dense annotations and aligned body shapes under the clothing. The aim is to enable and facilitate new research possibilities with learning-based methods, in 3D or using derived modalities. Second, to build this dataset automatically from raw scans, multiple robust 3D processing methods are proposed. These involve pose estimation, pose fitting, tight body shape fitting, and body shape estimation under clothing. The proposed methods show competitive or improved results on existing benchmarks and new proposed benchmarks based on 3DBodyTex. In particular, an alternative method is proposed to estimate the body shape under clothing from a single scan. On independent benchmarks, it is competitive with, or better than, methods requiring a full time sequence of scans. Third, the task of shape and texture completion of 3D human scans is tackled. A new method is proposed that completes the shape and the texture sequentially, and automatically identifies the missing regions. In particular, partial convolutions are extended to texture images (UV maps) for inpainting the colour of a 3D scan using a convolutional neural network. A new benchmark, based on 3DBodyTex, is proposed for the evaluation

3DBooSTeR: 3D Body Shape and Texture Recovery

We propose 3DBooSTeR, a novel method to recover a textured 3D body mesh from a textured partial 3D scan. With the advent of virtual and augmented reality, there is a demand for creating realistic and high-fidelity digital 3D human representations. However, 3D scanning systems can only capture the 3D human body shape up to some level of defects due to its complexity, including occlusion between bodyparts, varying levels of details, shape deformations and the articulated skeleton. Textured 3D mesh completion is thus important to enhance3D acquisitions. The proposed approach decouples the shape and texture completion into two sequential tasks. The shape is recovered by an encoder-decoder network deforming a template body mesh. The texture is subsequently obtained by projecting the partial texture onto the template mesh before inpainting the corresponding texture map with a novel approach. The approach is validated on the 3DBodyTex.v2 datase

Towards Generalization of 3D Human Pose Estimation In The Wild

In this paper, we propose 3DBodyTex.Pose, a dataset that addresses the task of 3D human pose estimation in-the-wild. Generalization to in-the-wild images remains limited due to the lack of adequate datasets. Existent ones are usually collected in indoor controlled environments where motion capture systems are used to obtain the 3D ground-truth annotations of humans. 3DBodyTex.Pose offers high quality and rich data containing 405 different real subjects in various clothing and poses, and 81k image samples with ground-truth 2D and 3D pose annotations. These images are generated from 200 viewpoints among which 70 challenging extreme viewpoints. This data was created starting from high resolution textured 3D body scans and by incorporating various realistic backgrounds. Retraining a state-of-the-art 3D pose estimation approach using data augmented with 3DBodyTex.Pose showed promising improvement in the overall performance, and a sensible decrease in the per joint position error when testing on challenging viewpoints. The 3DBodyTex.Pose is expected to offer the research community with new possibilities for generalizing 3D pose estimation from monocular in-the-wild images

SHARP 2020: The 1st Shape Recovery from Partial Textured 3D Scans Challenge Results

The SHApe Recovery from Partial textured 3D scans challenge, SHARP 2020, is the first edition of a challenge fostering and benchmarking methods for recovering complete textured 3D scans from raw incomplete data. SHARP 2020 is organized as a workshop in conjunction with ECCV 2020. There are two complementary challenges, the first one on 3D human scans, and the second one on generic objects. Challenge 1 is further split into two tracks, focusing, first, on large body and clothing regions, and, second, on fine body details. A novel evaluation metric is proposed to quantify jointly the shape reconstruction, the texture reconstruction, and the amount of completed data. Additionally, two unique datasets of 3D scans are proposed, to provide raw ground-truth data for the benchmarks. The datasets are released to the scientific community. Moreover, an accompanying custom library of software routines is also released to the scientific community. It allows for processing 3D scans, generating partial data and performing the evaluation. Results of the competition, analyzed in comparison to baselines, show the validity of the proposed evaluation metrics and highlight the challenging aspects of the task and of the datasets. Details on the SHARP 2020 challenge can be found at https://cvi2.uni.lu/sharp2020

Respiratory support in patients with severe COVID-19 in the International Severe Acute Respiratory and Emerging Infection (ISARIC) COVID-19 study: a prospective, multinational, observational study

Background: Up to 30% of hospitalised patients with COVID-19 require advanced respiratory support, including high-flow nasal cannulas (HFNC), non-invasive mechanical ventilation (NIV), or invasive mechanical ventilation (IMV). We aimed to describe the clinical characteristics, outcomes and risk factors for failing non-invasive respiratory support in patients treated with severe COVID-19 during the first two years of the pandemic in high-income countries (HICs) and low middle-income countries (LMICs). Methods: This is a multinational, multicentre, prospective cohort study embedded in the ISARIC-WHO COVID-19 Clinical Characterisation Protocol. Patients with laboratory-confirmed SARS-CoV-2 infection who required hospital admission were recruited prospectively. Patients treated with HFNC, NIV, or IMV within the first 24 h of hospital admission were included in this study. Descriptive statistics, random forest, and logistic regression analyses were used to describe clinical characteristics and compare clinical outcomes among patients treated with the different types of advanced respiratory support. Results: A total of 66,565 patients were included in this study. Overall, 82.6% of patients were treated in HIC, and 40.6% were admitted to the hospital during the first pandemic wave. During the first 24 h after hospital admission, patients in HICs were more frequently treated with HFNC (48.0%), followed by NIV (38.6%) and IMV (13.4%). In contrast, patients admitted in lower- and middle-income countries (LMICs) were less frequently treated with HFNC (16.1%) and the majority received IMV (59.1%). The failure rate of non-invasive respiratory support (i.e. HFNC or NIV) was 15.5%, of which 71.2% were from HIC and 28.8% from LMIC. The variables most strongly associated with non-invasive ventilation failure, defined as progression to IMV, were high leukocyte counts at hospital admission (OR [95%CI]; 5.86 [4.83-7.10]), treatment in an LMIC (OR [95%CI]; 2.04 [1.97-2.11]), and tachypnoea at hospital admission (OR [95%CI]; 1.16 [1.14-1.18]). Patients who failed HFNC/NIV had a higher 28-day fatality ratio (OR [95%CI]; 1.27 [1.25-1.30]). Conclusions: In the present international cohort, the most frequently used advanced respiratory support was the HFNC. However, IMV was used more often in LMIC. Higher leucocyte count, tachypnoea, and treatment in LMIC were risk factors for HFNC/NIV failure. HFNC/NIV failure was related to worse clinical outcomes, such as 28-day mortality. Trial registration This is a prospective observational study; therefore, no health care interventions were applied to participants, and trial registration is not applicable