121 research outputs found
ReliTalk: Relightable Talking Portrait Generation from a Single Video
Recent years have witnessed great progress in creating vivid audio-driven
portraits from monocular videos. However, how to seamlessly adapt the created
video avatars to other scenarios with different backgrounds and lighting
conditions remains unsolved. On the other hand, existing relighting studies
mostly rely on dynamically lighted or multi-view data, which are too expensive
for creating video portraits. To bridge this gap, we propose ReliTalk, a novel
framework for relightable audio-driven talking portrait generation from
monocular videos. Our key insight is to decompose the portrait's reflectance
from implicitly learned audio-driven facial normals and images. Specifically,
we involve 3D facial priors derived from audio features to predict delicate
normal maps through implicit functions. These initially predicted normals then
take a crucial part in reflectance decomposition by dynamically estimating the
lighting condition of the given video. Moreover, the stereoscopic face
representation is refined using the identity-consistent loss under simulated
multiple lighting conditions, addressing the ill-posed problem caused by
limited views available from a single monocular video. Extensive experiments
validate the superiority of our proposed framework on both real and synthetic
datasets. Our code is released in https://github.com/arthur-qiu/ReliTalk
Construction of a prognostic model of lung adenocarcinoma based on machine learning
In order to more accurately predict the prognosis and survival of lung adenocarcinoma patients, this paper used the gene expression and clinical information data of lung adenocarcinoma patients in the open database of TCGA to jointly construct a prognosis model of lung adenocarcinoma. Three difference analysis methods and univariate cox regression analysis were used as the preliminary screening method. By comparing the variable selection ability of lasso regression and random survival forest, comparing the performance of cox proportional risk regression model and random survival forest model, and integrating clinical data, a model that can more accurately predict the prognosis of lung adenocarcinoma patients was constructed. After comparison and selection, lasso regression was used to select variables and cox proportional risk model was used as the prediction model. The consistency index of the model reached 0.712. The AUC for 1-year, 3-year and 5-year survival of lung adenocarcinoma patients in the validation set were 0.808, 0.816 and 0.754, respectively. After the fusion of clinical data, the 1-year, 3-year and 5-year survival prediction AUC in the validation set were 0.840, 0.836 and 0.865, respectively, indicating that the model had good predictive performance
Evolution of emergent monopoles into magnetic skyrmion strings
Topological defects are fundamental concepts in physics, but little is known about the transition between distinct types across different dimensionalities. In topological magnetism, as in field theory, the transition between 1D strings and 0D monopoles is a key process whose observation has remained elusive. Here, we introduce a novel mechanism that allows for the controlled stabilization of emergent monopoles and show that magnetic skyrmion strings can be folded into monopoles. Conversely, they act as seeds out of which the entire string structure can unfold, containing its complete information. In chiral magnets, this process can be observed by resonant elastic X-ray scattering when the objects are in proximity to a polarized ferromagnet, whereby a pure monopole lattice is emerging on the surface. Our experimental proof of the reversible evolution from monopole to string sheds new light on topological defects and establishes the emergent monopole lattice as a new 3D topological phase
Detection of streptavidin using liquid crystal based whispering gallery mode microbubble
Protein is a complex chemical substance essential for human survival. Traditional protein detection methods, such as colorimetry, electrochemical analysis, and enzyme-linked immunosorbent assays, have shown good specificity and accuracy for the protein detection. However, all these methods require specialized instruments, and the detection procedures are laborious and time-consuming. As a result, a rapid, sensitive, label-free protein detection method is urgently needed. Herein, we have developed an ultra-sensitive biosensor for the detection of low-concentration protein molecules, employing liquid crystal (LC)-amplified optofluidic resonator. Since the orientations of LCs highly depend on the surface biomolecular binding processes, LCs can be employed to realize the extremely sensitive detection of biomolecules. Immobilized protein molecules interfere with the orientation of LCs by reducing the vertical anchoring force from the alignment layer in which the spectral wavelength shift was monitored as a sensing parameter. A biosensing platform based on an LC-amplified optofluidic whispering gallery mode (WGM) resonator was designed and studied accordingly. Due to the simultaneous interaction of the WGM and the LCs in the optofluidic resonator, the changes caused by the injection of protein molecules will be amplified, resulting in a shift in the resonance wavelength. Total wavelength shifts scale proportionally to the molecular concentrations of the protein within a certain range. The detection limit for streptavidin (SA) can reach as low as the femtometer level, which is significantly higher than the detection limit in the classic polarized optical microscope (POM) method visible with the naked eye. In addition to SA, the LC-based optofluidic resonator can also be applied to detect a variety of protein molecules. Our study demonstrates that LC-amplified optofluidic resonator can provide a novel solution for ultrasensitive real-time characterization of biosensing and biomolecular interactions
Model-enhanced Vector Index
Embedding-based retrieval methods construct vector indices to search for
document representations that are most similar to the query representations.
They are widely used in document retrieval due to low latency and decent recall
performance. Recent research indicates that deep retrieval solutions offer
better model quality, but are hindered by unacceptable serving latency and the
inability to support document updates. In this paper, we aim to enhance the
vector index with end-to-end deep generative models, leveraging the
differentiable advantages of deep retrieval models while maintaining desirable
serving efficiency. We propose Model-enhanced Vector Index (MEVI), a
differentiable model-enhanced index empowered by a twin-tower representation
model. MEVI leverages a Residual Quantization (RQ) codebook to bridge the
sequence-to-sequence deep retrieval and embedding-based models. To
substantially reduce the inference time, instead of decoding the unique
document ids in long sequential steps, we first generate some semantic virtual
cluster ids of candidate documents in a small number of steps, and then
leverage the well-adapted embedding vectors to further perform a fine-grained
search for the relevant documents in the candidate virtual clusters. We
empirically show that our model achieves better performance on the commonly
used academic benchmarks MSMARCO Passage and Natural Questions, with comparable
serving latency to dense retrieval solutions
Detection of DNA hybridization using liquid crystal based whispering gallery mode microbubble
DNA detection based on DNA hybridization has wide applications in various fields, such as clinical diagnostics, food safety, and environmental monitoring, etc. At present, common DNA detection approaches include fluorescence-based microarray technique, electrochemical method and surface plasmon resonance (SPR), etc. However, these methods require additional precise equipments and relatively complicated detection process. In this work, we developed a biosensor platform based on a liquid crystal (LC)-amplified optofluidic whispering gallery mode (WGM) resonator to achieve ultra-sensitive, label-free, and quick-response DNA hybridization detection. Liquid crystal is a material with high sensitivity, rapid response, and low cost. It exhibits significant directional and positional ordering, and it is sensitive and responsive to external stimuli. LC molecules exhibit a uniform orientation on the surface of the resonator, when the surface is covered with an appropriate amount of ssDNA. Once complementary DNA and ssDNA hybridize on the surface, the homotropic orientation of the LCs will be destroyed. Due to the simultaneous interaction of the WGM and the LCs in the optofluidic resonator, changes caused by the DNA hybridization can be amplified, resulting in a shift in the resonance wavelength. In this experiment, we used the spectral wavelength shift as a sensing parameter to achieve the detection of target DNA, and a lower detection limit compared to traditional DNA detection methods was obtained. At the same time, this biosensor platform also shows good selectivity. Our research results suggest that the LC based WGM optical microcavity sensing platform can provide an ultra-sensitive, label free solution for DNA detection
Label-free protein quantitation using liquid crystal-enhanced optofluidic biosensor
Protein detection plays an important role in the medical research. Liquid crystals (LCs), as a class of sensitive materials, exhibit a promising ability in the biosensing field. Herein, we exploited an ultrasensitive biosensor for protein detection, employing the whispering-gallery-mode (WGM) from the LC-amplified optofluidic micro-resonator. The biomolecules can trigger both light-matter interactions and the orientation transitions of LC molecules. The WGM spectral wavelength shift was recorded as the sensing indicator, and a detection limit of 15 fM for bovine serum albumin (BSA) was achieved. Our LC-amplified optofluidic biosensor provides a new solution for the ultrasensitive, real-time, and stable biological detection
PyPose: A Library for Robot Learning with Physics-based Optimization
Deep learning has had remarkable success in robotic perception, but its
data-centric nature suffers when it comes to generalizing to ever-changing
environments. By contrast, physics-based optimization generalizes better, but
it does not perform as well in complicated tasks due to the lack of high-level
semantic information and the reliance on manual parametric tuning. To take
advantage of these two complementary worlds, we present PyPose: a
robotics-oriented, PyTorch-based library that combines deep perceptual models
with physics-based optimization techniques. Our design goal for PyPose is to
make it user-friendly, efficient, and interpretable with a tidy and
well-organized architecture. Using an imperative style interface, it can be
easily integrated into real-world robotic applications. Besides, it supports
parallel computing of any order gradients of Lie groups and Lie algebras and
-order optimizers, such as trust region methods. Experiments
show that PyPose achieves 3-20 speedup in computation compared to
state-of-the-art libraries. To boost future research, we provide concrete
examples across several fields of robotics, including SLAM, inertial
navigation, planning, and control
PyPose v0.6: The Imperative Programming Interface for Robotics
PyPose is an open-source library for robot learning. It combines a
learning-based approach with physics-based optimization, which enables seamless
end-to-end robot learning. It has been used in many tasks due to its
meticulously designed application programming interface (API) and efficient
implementation. From its initial launch in early 2022, PyPose has experienced
significant enhancements, incorporating a wide variety of new features into its
platform. To satisfy the growing demand for understanding and utilizing the
library and reduce the learning curve of new users, we present the fundamental
design principle of the imperative programming interface, and showcase the
flexible usage of diverse functionalities and modules using an extremely simple
Dubins car example. We also demonstrate that the PyPose can be easily used to
navigate a real quadruped robot with a few lines of code
- …