Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis

Recent works in implicit representations, such as Neural Radiance Fields (NeRF), have advanced the generation of realistic and animatable head avatars from video sequences. These implicit methods are still confronted by visual artifacts and jitters, since the lack of explicit geometric constraints poses a fundamental challenge in accurately modeling complex facial deformations. In this paper, we introduce Dynamic Tetrahedra (DynTet), a novel hybrid representation that encodes explicit dynamic meshes by neural networks to ensure geometric consistency across various motions and viewpoints. DynTet is parameterized by the coordinate-based networks which learn signed distance, deformation, and material texture, anchoring the training data into a predefined tetrahedra grid. Leveraging Marching Tetrahedra, DynTet efficiently decodes textured meshes with a consistent topology, enabling fast rendering through a differentiable rasterizer and supervision via a pixel loss. To enhance training efficiency, we incorporate classical 3D Morphable Models to facilitate geometry learning and define a canonical space for simplifying texture learning. These advantages are readily achievable owing to the effective geometric representation employed in DynTet. Compared with prior works, DynTet demonstrates significant improvements in fidelity, lip synchronization, and real-time performance according to various metrics. Beyond producing stable and visually appealing synthesis videos, our method also outputs the dynamic meshes which is promising to enable many emerging applications.Comment: CVPR 202

Visually Grounded Commonsense Knowledge Acquisition

Large-scale commonsense knowledge bases empower a broad range of AI applications, where the automatic extraction of commonsense knowledge (CKE) is a fundamental and challenging problem. CKE from text is known for suffering from the inherent sparsity and reporting bias of commonsense in text. Visual perception, on the other hand, contains rich commonsense knowledge about real-world entities, e.g., (person, can_hold, bottle), which can serve as promising sources for acquiring grounded commonsense knowledge. In this work, we present CLEVER, which formulates CKE as a distantly supervised multi-instance learning problem, where models learn to summarize commonsense relations from a bag of images about an entity pair without any human annotation on image instances. To address the problem, CLEVER leverages vision-language pre-training models for deep understanding of each image in the bag, and selects informative instances from the bag to summarize commonsense entity relations via a novel contrastive attention mechanism. Comprehensive experimental results in held-out and human evaluation show that CLEVER can extract commonsense knowledge in promising quality, outperforming pre-trained language model-based methods by 3.9 AUC and 6.4 mAUC points. The predicted commonsense scores show strong correlation with human judgment with a 0.78 Spearman coefficient. Moreover, the extracted commonsense can also be grounded into images with reasonable interpretability. The data and codes can be obtained at https://github.com/thunlp/CLEVER.Comment: Accepted by AAAI 202

Low-Temperature Gas Plasma Combined with Antibiotics for the Reduction of Methicillin-Resistant \u3ci\u3eStaphylococcus aureus\u3c/i\u3e Biofilm Both in Vitro and in Vivo

Biofilm infections in wounds seriously delay the healing process, and methicillin-resistant Staphylococcus aureus is a major cause of wound infections. In addition to inactivating micro-organisms, low-temperature gas plasma can restore the sensitivity of pathogenic microbes to antibiotics. However, the combined treatment has not been applied to infectious diseases. In this study, low-temperature gas plasma treatment promoted the effects of different antibiotics on the reduction of S. aureus biofilms in vitro. Low-temperature gas plasma combined with rifampicin also effectively reduced the S. aureus cells in biofilms in the murine wound infection model. The blood and histochemical analysis demonstrated the biosafety of the combined treatment. Our findings demonstrated that low-temperature gas plasma combined with antibiotics is a promising therapeutic strategy for wound infections

Pre‐symptomatic transmission of novel coronavirus in community settings

We used contact tracing to document how COVID‐19 was transmitted across 5 generations involving 10 cases, starting with an individual who became ill on January 27. We calculated the incubation period of the cases as the interval between infection and development of symptoms. The median incubation period was 6.0 days (interquartile range, 3.5‐9.5 days). The last two generations were infected in public places, 3 and 4 days prior to the onset of illness in their infectors. Both had certain underlying conditions and comorbidity. Further identification of how individuals transmit prior to being symptomatic will have important consequences.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/163478/2/irv12773.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163478/1/irv12773_am.pd

Exploration in Mapping Kernel-Based Home Range Models from Remote Sensing Imagery with Conditional Adversarial Networks

Kernel-based home range models are widely-used to estimate animal habitats and develop conservation strategies. They provide a probabilistic measure of animal space use instead of assuming the uniform utilization within an outside boundary. However, this type of models estimates the home ranges from animal relocations, and the inadequate locational data often prevents scientists from applying them in long-term and large-scale research. In this paper, we propose an end-to-end deep learning framework to simulate kernel home range models. We use the conditional adversarial network as a supervised model to learn the home range mapping from time-series remote sensing imagery. Our approach enables scientists to eliminate the persistent dependence on locational data in home range analysis. In experiments, we illustrate our approach by mapping the home ranges of Bar-headed Geese in Qinghai Lake area. The proposed framework outperforms all baselines in both qualitative and quantitative evaluations, achieving visually recognizable results and high mapping accuracy. The experiment also shows that learning the mapping between images is a more effective way to map such complex targets than traditional pixel-based schemes

Investigating Home Range, Movement Pattern, and Habitat Selection of Bar-headed Geese during Breeding Season at Qinghai Lake, China

The Bar-headed Goose is the only true goose species or Anserinae to migrate solely within the Central Asian Flyway, and thus, it is an ideal species for observing the effects of both land use and climate change throughout the flyway. In this paper, we investigate the home range, movement pattern, and habitat selection of Bar-headed Geese (Anser indicus) during the breeding season at Qinghai Lake, which is one of their largest breeding areas and a major migration staging area in the flyway. We identified several areas used by the geese during the breeding season along the shoreline of Qinghai Lake and found that most geese had more than one core use area and daily movements that provided insight into their breeding activity. We also observed the intensive use of specific wetlands and habitats near Qinghai Lake. These data provide interesting insights into the movement ecology of this important species and also provide critical information for managers seeking to understand and respond to conservation concerns threatening Bar-headed Geese, such as landscape and habitat changes

Exploiting Time-Series Image-to-Image Translation to Expand the Range of Wildlife Habitat Analysis

Characterizing wildlife habitat is one of the main topics in animal ecology. Locational data obtained from radio tracking and field observation are widely used in habitat analysis. However, such sampling methods are costly and laborious, and insufficient relocations often prevent scientists from conducting large-range and long-term research. In this paper, we innovatively exploit the image-to-image translation technology to expand the range of wildlife habitat analysis. We proposed a novel approach for implementing time-series imageto-image translation via metric embedding. A siamese neural network is used to learn the Euclidean temporal embedding from the image space. This embedding produces temporal vectors which bring time information into the adversarial network. The well-trained framework could effectively map the probabilistic habitat models from remote sensing imagery, helping scientists get rid of the persistent dependence on animal relocations. We illustrate our approach in a real-world application for mapping the habitats of Bar-headed Geese at Qinghai Lake breeding ground. We compare our model against several baselines and achieve promising results