SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field

Despite the great success in 2D editing using user-friendly tools, such as Photoshop, semantic strokes, or even text prompts, similar capabilities in 3D areas are still limited, either relying on 3D modeling skills or allowing editing within only a few categories. In this paper, we present a novel semantic-driven NeRF editing approach, which enables users to edit a neural radiance field with a single image, and faithfully delivers edited novel views with high fidelity and multi-view consistency. To achieve this goal, we propose a prior-guided editing field to encode fine-grained geometric and texture editing in 3D space, and develop a series of techniques to aid the editing process, including cyclic constraints with a proxy mesh to facilitate geometric supervision, a color compositing mechanism to stabilize semantic-driven texture editing, and a feature-cluster-based regularization to preserve the irrelevant content unchanged. Extensive experiments and editing examples on both real-world and synthetic data demonstrate that our method achieves photo-realistic 3D editing using only a single edited image, pushing the bound of semantic-driven editing in 3D real-world scenes. Our project webpage: https://zju3dv.github.io/sine/.Comment: Accepted to CVPR 2023. Project Page: https://zju3dv.github.io/sine

Cross-Lingual Taxonomy Alignment with Bilingual Biterm Topic Model

As more and more multilingual knowledge becomes available on the Web, knowledge sharing across languages has become an important task to benefit many applications. One of the most crucial kinds of knowledge on the Web is taxonomy, which is used to organize and classify the Web data. To facilitate knowledge sharing across languages, we need to deal with the problem of cross-lingual taxonomy alignment, which discovers the most relevant category in the target taxonomy of one language for each category in the source taxonomy of another language. Current approaches for aligning cross-lingual taxonomies strongly rely on domain-specific information and the features based on string similarities. In this paper, we present a new approach to deal with the problem of cross-lingual taxonomy alignment without using any domain-specific information. We first identify the candidate matched categories in the target taxonomy for each category in the source taxonomy using the cross-lingual string similarity. We then propose a novel bilingual topic model, called Bilingual Biterm Topic Model (BiBTM), to perform exact matching. BiBTM is trained by the textual contexts extracted from the Web. We conduct experiments on two kinds of real world datasets. The experimental results show that our approach significantly outperforms the designed state-of-the-art comparison methods

Design of a single-lens freeform prism based distortion-free stereovision system

10.1109/jphot.2019.2924458IEEE Photonics Journal1-

Noteworthy Consensus Effects of D/E Residues in Umami Peptides Used for Designing the Novel Umami Peptides

Aspartic acid (D) and glutamic acid (E) play vital roles in the umami peptides. To understand their exact mechanism of action, umami peptides were collected and cut into 1/2/3/4 fragments. Connecting D/E to the N/C-termini of the fragments formed D/E consensus effect groups (DEEGs), and all fragments containing DEEG were summarized according to the ratio and ranking obtained in the above four situations. The interaction patterns between peptides in DEEG and T1R1/T1R3-VFD were compared by statistical analysis and molecular docking, and the most conservative contacts were found to be HdB_277_ARG and HdB_148_SER. The molecular docking score of the effector peptides significantly dropped compared to that of their original peptides (−1.076 ± 0.658 kcal/mol, p value < 0.05). Six types of consensus fingerprints were set according to the Top7 contacts. The exponential of relative umami was linearly correlated with ΔGbind (R2 = 0.961). Under the D/E consensus effect, the electrostatic effect of the umami peptide was improved, and the energy gap between the highest occupied molecular orbital–the least unoccupied molecular orbital (HOMO–LUMO) was decreased. The shortest path map showed that the peptides had similar T1R1–T1R3 recognition pathways. This study helps to reveal umami perception rules and provides support for the efficient screening of umami peptides based on the material richness in D/E sequences