3D-aware Image Generation and Editing with Multi-modal Conditions

3D-consistent image generation from a single 2D semantic label is an important and challenging research topic in computer graphics and computer vision. Although some related works have made great progress in this field, most of the existing methods suffer from poor disentanglement performance of shape and appearance, and lack multi-modal control. In this paper, we propose a novel end-to-end 3D-aware image generation and editing model incorporating multiple types of conditional inputs, including pure noise, text and reference image. On the one hand, we dive into the latent space of 3D Generative Adversarial Networks (GANs) and propose a novel disentanglement strategy to separate appearance features from shape features during the generation process. On the other hand, we propose a unified framework for flexible image generation and editing tasks with multi-modal conditions. Our method can generate diverse images with distinct noises, edit the attribute through a text description and conduct style transfer by giving a reference RGB image. Extensive experiments demonstrate that the proposed method outperforms alternative approaches both qualitatively and quantitatively on image generation and editing

Robust Pose Transfer with Dynamic Details using Neural Video Rendering

Pose transfer of human videos aims to generate a high fidelity video of a target person imitating actions of a source person. A few studies have made great progress either through image translation with deep latent features or neural rendering with explicit 3D features. However, both of them rely on large amounts of training data to generate realistic results, and the performance degrades on more accessible internet videos due to insufficient training frames. In this paper, we demonstrate that the dynamic details can be preserved even trained from short monocular videos. Overall, we propose a neural video rendering framework coupled with an image-translation-based dynamic details generation network (D2G-Net), which fully utilizes both the stability of explicit 3D features and the capacity of learning components. To be specific, a novel texture representation is presented to encode both the static and pose-varying appearance characteristics, which is then mapped to the image space and rendered as a detail-rich frame in the neural rendering stage. Moreover, we introduce a concise temporal loss in the training stage to suppress the detail flickering that is made more visible due to high-quality dynamic details generated by our method. Through extensive comparisons, we demonstrate that our neural human video renderer is capable of achieving both clearer dynamic details and more robust performance even on accessible short videos with only 2k - 4k frames.Comment: Video link: https://www.bilibili.com/video/BV1y64y1C7ge

Probucol alleviates atherosclerosis and improves high density lipoprotein function

<p>Abstract</p> <p>Background</p> <p>Probucol is a unique hypolipidemic agent that decreases high density lipoprotein cholesterol (HDL-C). However, it is not definite that whether probucol hinders the progression of atherosclerosis by improving HDL function.</p> <p>Methods</p> <p>Eighteen New Zealand White rabbits were randomly divided into the control, atherosclerosis and probucol groups. Control group were fed a regular diet; the atherosclerosis group received a high fat diet, and the probucol group received the high fat diet plus probucol. Hepatocytes and peritoneal macrophages were isolated for [³H] labeled cholesterol efflux rates and expression of ABCA1 and SR-B1 at gene and protein levels; venous blood was collected for serum paraoxonase 1, myeloperoxidase activity and lipid analysis. Aorta were prepared for morphologic and immunohistochemical analysis after 12 weeks.</p> <p>Results</p> <p>Compared to the atherosclerosis group, the paraoxonase 1 activity, cholesterol efflux rates, expression of ABCA1 and SR-BI in hepatocytes and peritoneal macrophages, and the level of ABCA1 and SR-BI in aortic lesions were remarkably improved in the probucol group, But the serum HDL cholesterol concentration, myeloperoxidase activity, the IMT and the percentage plaque area of aorta were significantly decreased.</p> <p>Conclusion</p> <p>Probucol alleviated atherosclerosis by improving HDL function. The mechanisms include accelerating the process of reverse cholesterol transport, improving the anti-inflammatory and anti-oxidant functions.</p

Learning-based intrinsic reflectional symmetry detection

Reflectional symmetry is a ubiquitous pattern in nature. Previous works usually solve this problem by voting or sampling, suffering from high computational cost and randomness. In this paper, we propose a learning-based approach to intrinsic reflectional symmetry detection. Instead of directly finding symmetric point pairs, we parametrize this self-isometry using a functional map matrix, which can be easily computed given the signs of Laplacian eigenfunctions under the symmetric mapping. Therefore, we manually label the eigenfunction signs for a variety of shapes and train a novel neural network to predict the sign of each eigenfunction under symmetry. Our network aims at learning the global property of functions and consequently converts the problem defined on the manifold to the functional domain. By disentangling the prediction of the matrix into separated bases, our method generalizes well to new shapes and is invariant under perturbation of eigenfunctions. Through extensive experiments, we demonstrate the robustness of our method in challenging cases, including different topology and incomplete shapes with holes. By avoiding random sampling, our learning-based algorithm is over 20 times faster than state-of-the-art methods, and meanwhile, is more robust, achieving higher correspondence accuracy in commonly used metrics

Active exploration of large 3D model repositories

With broader availability of large-scale 3D model repositories, the need for efficient and effective exploration becomes more and more urgent. Existing model retrieval techniques do not scale well with the size of the database since often a large number of very similar objects are returned for a query, and the possibilities to refine the search are quite limited. We propose an interactive approach where the user feeds an active learning procedure by labeling either entire models or parts of them as “like” or “dislike” such that the system can automatically update an active set of recommended models. To provide an intuitive user interface, candidate models are presented based on their estimated relevance for the current query. From the methodological point of view, our main contribution is to exploit not only the similarity between a query and the database models but also the similarities among the database models themselves. We achieve this by an offline pre-processing stage, where global and local shape descriptors are computed for each model and a sparse distance metric is derived that can be evaluated efficiently even for very large databases. We demonstrate the effectiveness of our method by interactively exploring a repository containing over 100K models

Experimental Generation of Spin-Photon Entanglement in Silicon Carbide

A solid-state approach for quantum networks is advantages, as it allows the integration of nanophotonics to enhance the photon emission and the utilization of weakly coupled nuclear spins for long-lived storage. Silicon carbide, specifically point defects within it, shows great promise in this regard due to the easy of availability and well-established nanofabrication techniques. Despite of remarkable progresses made, achieving spin-photon entanglement remains a crucial aspect to be realized. In this paper, we experimentally generate entanglement between a silicon vacancy defect in silicon carbide and a scattered single photon in the zero-phonon line. The spin state is measured by detecting photons scattered in the phonon sideband. The photonic qubit is encoded in the time-bin degree-of-freedom and measured using an unbalanced Mach-Zehnder interferometer. Photonic correlations not only reveal the quality of the entanglement but also verify the deterministic nature of the entanglement creation process. By harnessing two pairs of such spin-photon entanglement, it becomes straightforward to entangle remote quantum nodes at long distance.Comment: 8 pages in total, 4 figures in the main text, 1 figure in the supplemental materia

Study on Small Simulation Device of Coal Spontaneous Combustion Process

AbstractConsidering advantages and disadvantages of present ignition stations, the double nested spontaneous furnace with better thermal insulation performance was developed, and electrical sand bath was taken as thermal insulation and gas preheating device, also the temperature acquisition and temperature control device with higher accuracy was selected in order to solve the problems of heat dissipation and temperature control. The ignition furnace and sand-bath attemperator of the device form the double insurance to protect the heat in the process of the coal spontaneous combustion reaction, and could guarantee the coal spontaneous combustion reaction process to be simulated scientifically and veritably. With this device, the coal sample of the SHIGANG COAL MINE was experimented to simulate the spontaneous combustion reaction process, proving that the device is feasible. The device can be used in the relevant experiments of coal spontaneous combustion

Lightweight text-driven image editing with disentangled content and attributes

Text-driven image editing aims to manipulate images with the guidance of natural language description. Text is much more natural and intuitive than many other interaction modes, and attracts more attention recently. However, compared with classical supervised learning tasks, there is no standard benchmark dataset for text-driven interactive image editing up to now. Therefore, it is hard to train an end-to-end model for pixel-aligned interactive image editing driven by text. Some methods follow the paradigm of text-to-image models by incorporating the target image into the process of text-to-image generation. However, these methods relying on cross-modal text-to-image generation involve complicated and expensive models, which can lead to inconsistent editing effects. In this article, a novel text-driven image editing method is proposed. Our key observation is that this task can be more efficiently learned using image-to-image translation. To ensure effective learning for image editing, our framework takes paired text and the corresponding images for training, and disentangles each image into content and attributes, such that the content is maintained while the attributes are modified according to the text. Our network is a lightweight encoder-decoder architecture that accomplishes pixel-aligned end-to-end training via cycle-consistent supervision. Quantitative and qualitative experimental results show that the proposed method achieves state-of-the-art performance

3D colored object reconstruction from a single view image through diffusion

In this paper we propose a novel 3D colored object reconstruction method from a single view image. Given a reference image, a conditional diffusion probabilistic model is built to reconstruct both a 3D point cloud shape and the corresponding color features at each point, and then images from arbitrary views can be synthesized using a volume rendering technique. The approach involves several sequential steps. First, the reference RGB image is encoded into separate shape and color latent variables. Then, a shape prediction module predicts reverse geometric noise based on the shape latent variable within the diffusion model. Next, a color prediction module predicts color features for each 3D point using information from the color latent variable. Finally, a volume rendering module transforms the generated colored point cloud into 2D image space, facilitating training based solely on a reference image. Experimental results demonstrate that the proposed method achieves competitive performance on colored 3D shape reconstruction and novel view image synthesis