13,665 research outputs found

    Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency

    Full text link
    Visual simultaneous localization and mapping (SLAM) systems face challenges in detecting loop closure under the circumstance of large viewpoint changes. In this paper, we present an object-based loop closure detection method based on the spatial layout and semanic consistency of the 3D scene graph. Firstly, we propose an object-level data association approach based on the semantic information from semantic labels, intersection over union (IoU), object color, and object embedding. Subsequently, multi-view bundle adjustment with the associated objects is utilized to jointly optimize the poses of objects and cameras. We represent the refined objects as a 3D spatial graph with semantics and topology. Then, we propose a graph matching approach to select correspondence objects based on the structure layout and semantic property similarity of vertices' neighbors. Finally, we jointly optimize camera trajectories and object poses in an object-level pose graph optimization, which results in a globally consistent map. Experimental results demonstrate that our proposed data association approach can construct more accurate 3D semantic maps, and our loop closure method is more robust than point-based and object-based methods in circumstances with large viewpoint changes

    VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs

    Full text link
    We introduce VIVE3D, a novel approach that extends the capabilities of image-based 3D GANs to video editing and is able to represent the input video in an identity-preserving and temporally consistent way. We propose two new building blocks. First, we introduce a novel GAN inversion technique specifically tailored to 3D GANs by jointly embedding multiple frames and optimizing for the camera parameters. Second, besides traditional semantic face edits (e.g. for age and expression), we are the first to demonstrate edits that show novel views of the head enabled by the inherent properties of 3D GANs and our optical flow-guided compositing technique to combine the head with the background video. Our experiments demonstrate that VIVE3D generates high-fidelity face edits at consistent quality from a range of camera viewpoints which are composited with the original video in a temporally and spatially consistent manner.Comment: CVPR 2023. Project webpage and video available at http://afruehstueck.github.io/vive3

    Hi4D: 4D Instance Segmentation of Close Human Interaction

    Full text link
    We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks.Comment: Project page: https://yifeiyin04.github.io/Hi4D

    Evaluating 3D human face reconstruction from a frontal 2D image, focusing on facial regions associated with foetal alcohol syndrome

    Get PDF
    Foetal alcohol syndrome (FAS) is a preventable condition caused by maternal alcohol consumption during pregnancy. The FAS facial phenotype is an important factor for diagnosis, alongside central nervous system impairments and growth abnormalities. Current methods for analysing the FAS facial phenotype rely on 3D facial image data, obtained from costly and complex surface scanning devices. An alternative is to use 2D images, which are easy to acquire with a digital camera or smart phone. However, 2D images lack the geometric accuracy required for accurate facial shape analysis. Our research offers a solution through the reconstruction of 3D human faces from single or multiple 2D images. We have developed a framework for evaluating 3D human face reconstruction from a single-input 2D image using a 3D face model for potential use in FAS assessment. We first built a generative morphable model of the face from a database of registered 3D face scans with diverse skin tones. Then we applied this model to reconstruct 3D face surfaces from single frontal images using a model-driven sampling algorithm. The accuracy of the predicted 3D face shapes was evaluated in terms of surface reconstruction error and the accuracy of FAS-relevant landmark locations and distances. Results show an average root mean square error of 2.62 mm. Our framework has the potential to estimate 3D landmark positions for parts of the face associated with the FAS facial phenotype. Future work aims to improve on the accuracy and adapt the approach for use in clinical settings. Significance: Our study presents a framework for constructing and evaluating a 3D face model from 2D face scans and evaluating the accuracy of 3D face shape predictions from single images. The results indicate low generalisation error and comparability to other studies. The reconstructions also provide insight into specific regions of the face relevant to FAS diagnosis. The proposed approach presents a potential cost-effective and easily accessible imaging tool for FAS screening, yet its clinical application needs further research

    Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Training

    Full text link
    Text-to-motion generation is an emerging and challenging problem, which aims to synthesize motion with the same semantics as the input text. However, due to the lack of diverse labeled training data, most approaches either limit to specific types of text annotations or require online optimizations to cater to the texts during inference at the cost of efficiency and stability. In this paper, we investigate offline open-vocabulary text-to-motion generation in a zero-shot learning manner that neither requires paired training data nor extra online optimization to adapt for unseen texts. Inspired by the prompt learning in NLP, we pretrain a motion generator that learns to reconstruct the full motion from the masked motion. During inference, instead of changing the motion generator, our method reformulates the input text into a masked motion as the prompt for the motion generator to ``reconstruct'' the motion. In constructing the prompt, the unmasked poses of the prompt are synthesized by a text-to-pose generator. To supervise the optimization of the text-to-pose generator, we propose the first text-pose alignment model for measuring the alignment between texts and 3D poses. And to prevent the pose generator from overfitting to limited training texts, we further propose a novel wordless training mechanism that optimizes the text-to-pose generator without any training texts. The comprehensive experimental results show that our method obtains a significant improvement against the baseline methods. The code is available at https://github.com/junfanlin/oohmg

    Northern Powerhouses: the homes of the industrial elite, c.1780-1875

    Get PDF
    This thesis explores the world of the industrial elites of Manchester and Liverpool in the period c.1780-1875, through their houses. The homes of the industrial elites, namely merchants and manufacturers, were extremely important tangible communicators of wealth, taste, and comfort. Whilst status-building was closely connected to the house, this thesis argues that the industrial elites carved their own identities into their domestic spheres and that emulation was not solely linked with aspiration. The findings of this thesis are based around its three research aims regarding the changing location of houses in Manchester and Liverpool in the eighteenth and nineteenth centuries, the appearance and use of houses, and the daily routines and involvement of the industrial elite in their domestic routines. An analysis of elite residential patterns in Manchester and Liverpool across the eighteenth and nineteenth centuries has created a more nuanced look at urban geographies of the region in this period. Though some residential patterns differed because of economic and political structure, a key finding has been that the process of suburbanisation in and around Manchester and Liverpool commenced earlier than previous scholarship has suggested. Suburbanisation among the elites began in the latter decades of the eighteenth century and into the early decades of the nineteenth century, with elite suburban communities being firmly established by the 1820s. This thesis discovered that despite socio-economic and political differences, the industrial elites of Manchester and Liverpool used their houses, gardens, and landed estates in very similar ways. This was a result of conformity which arose from emulation at both a community-based level and the emulation and aspiration of elite, gentrified lifestyle. Also, the merchants and manufacturers analysed within this work were involved in their home at every level of domesticity, from the construction of the house to the financial management of the household, although this latter theme was often a cooperative effort between spouses and family members, adding more to our understanding of gender, domesticity, and familial relations. Through detailed case studies and a combination of sources, the private lives of the industrial elites have been revaluated and redefined, including showing how their houses functions metaphorically and in reality

    Targeting Fusion Proteins of HIV-1 and SARS-CoV-2

    Get PDF
    Viruses are disease-causing pathogenic agents that require host cells to replicate. Fusion of host and viral membranes is critical for the lifecycle of enveloped viruses. Studying viral fusion proteins can allow us to better understand how they shape immune responses and inform the design of therapeutics such as drugs, monoclonal antibodies, and vaccines. This thesis discusses two approaches to targeting two fusion proteins: Env from HIV-1 and S from SARS-CoV-2. The first chapter of this thesis is an introduction to viruses with a specific focus on HIV-1 CD4 mimetic drugs and antibodies against SARS-CoV-2. It discusses the architecture of these viruses and fusion proteins and how small molecules, peptides, and antibodies can target these proteins successfully to treat and prevent disease. In addition, a brief overview is included of the techniques involved in structural biology and how it has informed the study of viruses. For the interested reader, chapter 2 contains a review article that serves as a more in-depth introduction for both viruses as well as how the use of structural biology has informed the study of viral surface proteins and neutralizing antibody responses to them. The subsequent chapters provide a body of work divided into two parts. The first part in chapter 3 involves a study on conformational changes induced in the HIV-1 Env protein by CD4-mimemtic drugs using single particle cryo-EM. The second part encompassing chapters 4 and 5 includes two studies on antibodies isolated from convalescent COVID-19 donors. The former involves classification of antibody responses to the SARS-CoV-2 S receptor-binding domain (RBD). The latter discusses an anti-RBD antibody class that binds to a conserved epitope on the RBD and shows cross-binding and cross-neutralization to other coronaviruses in the sarbecovirus subgenus.</p

    Cost-effective non-destructive testing of biomedical components fabricated using additive manufacturing

    Get PDF
    Biocompatible titanium-alloys can be used to fabricate patient-specific medical components using additive manufacturing (AM). These novel components have the potential to improve clinical outcomes in various medical scenarios. However, AM introduces stability and repeatability concerns, which are potential roadblocks for its widespread use in the medical sector. Micro-CT imaging for non-destructive testing (NDT) is an effective solution for post-manufacturing quality control of these components. Unfortunately, current micro-CT NDT scanners require expensive infrastructure and hardware, which translates into prohibitively expensive routine NDT. Furthermore, the limited dynamic-range of these scanners can cause severe image artifacts that may compromise the diagnostic value of the non-destructive test. Finally, the cone-beam geometry of these scanners makes them susceptible to the adverse effects of scattered radiation, which is another source of artifacts in micro-CT imaging. In this work, we describe the design, fabrication, and implementation of a dedicated, cost-effective micro-CT scanner for NDT of AM-fabricated biomedical components. Our scanner reduces the limitations of costly image-based NDT by optimizing the scanner\u27s geometry and the image acquisition hardware (i.e., X-ray source and detector). Additionally, we describe two novel techniques to reduce image artifacts caused by photon-starvation and scatter radiation in cone-beam micro-CT imaging. Our cost-effective scanner was designed to match the image requirements of medium-size titanium-alloy medical components. We optimized the image acquisition hardware by using an 80 kVp low-cost portable X-ray unit and developing a low-cost lens-coupled X-ray detector. Image artifacts caused by photon-starvation were reduced by implementing dual-exposure high-dynamic-range radiography. For scatter mitigation, we describe the design, manufacturing, and testing of a large-area, highly-focused, two-dimensional, anti-scatter grid. Our results demonstrate that cost-effective NDT using low-cost equipment is feasible for medium-sized, titanium-alloy, AM-fabricated medical components. Our proposed high-dynamic-range strategy improved by 37% the penetration capabilities of an 80 kVp micro-CT imaging system for a total x-ray path length of 19.8 mm. Finally, our novel anti-scatter grid provided a 65% improvement in CT number accuracy and a 48% improvement in low-contrast visualization. Our proposed cost-effective scanner and artifact reduction strategies have the potential to improve patient care by accelerating the widespread use of patient-specific, bio-compatible, AM-manufactured, medical components
    • …
    corecore