Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap

Have you ever imagined how it would look if we placed new objects into paintings? For example, what would it look like if we placed a basketball into Claude Monet's ``Water Lilies, Evening Effect''? We propose Reference-based Painterly Inpainting, a novel task that crosses the wild reference domain gap and implants novel objects into artworks. Although previous works have examined reference-based inpainting, they are not designed for large domain discrepancies between the target and the reference, such as inpainting an artistic image using a photorealistic reference. This paper proposes a novel diffusion framework, dubbed RefPaint, to ``inpaint more wildly'' by taking such references with large domain gaps. Built with an image-conditioned diffusion model, we introduce a ladder-side branch and a masked fusion mechanism to work with the inpainting mask. By decomposing the CLIP image embeddings at inference time, one can manipulate the strength of semantic and style information with ease. Experiments demonstrate that our proposed RefPaint framework produces significantly better results than existing methods. Our method enables creative painterly image inpainting with reference objects that would otherwise be difficult to achieve. Project page: https://vita-group.github.io/RefPaint

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360{\deg} Views

Virtual reality and augmented reality (XR) bring increasing demand for 3D content. However, creating high-quality 3D content requires tedious work that a human expert must do. In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360{\deg} views that correspond well with the given reference image. By conditioning on the reference image, our model can fulfill the everlasting curiosity for synthesizing novel views of objects from images. Our technique sheds light on a promising direction of easing the workflows for 3D artists and XR designers. We propose a novel framework, dubbed NeuralLift-360, that utilizes a depth-aware neural radiance representation (NeRF) and learns to craft the scene guided by denoising diffusion models. By introducing a ranking loss, our NeuralLift-360 can be guided with rough depth estimation in the wild. We also adopt a CLIP-guided sampling strategy for the diffusion prior to provide coherent guidance. Extensive experiments demonstrate that our NeuralLift-360 significantly outperforms existing state-of-the-art baselines. Project page: https://vita-group.github.io/NeuralLift-360/Comment: Project page: https://vita-group.github.io/NeuralLift-360

Tin-graphene tubes as anodes for lithium-ion batteries with high volumetric and gravimetric energy densities.

Limited by the size of microelectronics, as well as the space of electrical vehicles, there are tremendous demands for lithium-ion batteries with high volumetric energy densities. Current lithium-ion batteries, however, adopt graphite-based anodes with low tap density and gravimetric capacity, resulting in poor volumetric performance metric. Here, by encapsulating nanoparticles of metallic tin in mechanically robust graphene tubes, we show tin anodes with high volumetric and gravimetric capacities, high rate performance, and long cycling life. Pairing with a commercial cathode material LiNi0.6Mn0.2Co0.2O2, full cells exhibit a gravimetric and volumetric energy density of 590 W h Kg-1 and 1,252 W h L-1, respectively, the latter of which doubles that of the cell based on graphite anodes. This work provides an effective route towards lithium-ion batteries with high energy density for a broad range of applications

Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation

For a complicated algorithm, its implementation by a human programmer usually starts with outlining a rough control flow followed by iterative enrichments, eventually yielding carefully generated syntactic structures and variables in a hierarchy. However, state-of-the-art large language models generate codes in a single pass, without intermediate warm-ups to reflect the structured thought process of "outline-then-detail". Inspired by the recent success of chain-of-thought prompting, we propose ChainCoder, a program synthesis language model that generates Python code progressively, i.e. from coarse to fine in multiple passes. We first decompose source code into layout frame components and accessory components via abstract syntax tree parsing to construct a hierarchical representation. We then reform our prediction target into a multi-pass objective, each pass generates a subsequence, which is concatenated in the hierarchy. Finally, a tailored transformer architecture is leveraged to jointly encode the natural language descriptions and syntactically aligned I/O data samples. Extensive evaluations show that ChainCoder outperforms state-of-the-arts, demonstrating that our progressive generation eases the reasoning procedure and guides the language model to generate higher-quality solutions. Our codes are available at: https://github.com/VITA-Group/ChainCoder.Comment: Accepted in ICML 202

Influence of flowering on the anatomical structure, chemical components and carbohydrate metabolism of Bambusa tuldoides culms at different ages

Bamboo forests, which have come to occupy large areas in recent years, naturally undergo the process of blooming. However, bamboo culms and rhizomes degenerate after the plants bloom, resulting in widespread loss of raw materials. Systematic research on the properties and physiology of bamboo culms after flowering is lacking, and whether flowering bamboo culms could be used as raw materials in industry is unclear. In this paper, we compared and measured the fiber morphology, chemical components, and sugar metabolism indexes of non-flowering and flowering Bambusa tuldoides culms at different ages. The results showed that the fibers in the middle internodes of both non-flowering and flowering B. tuldoides culms had the longest length. The fibers completed their elongation within 1 year, but the fiber walls were continually deposited with age. The levels of the chemical components in the nonflowering culms also continually increased with age. The nonstructural carbohydrate (NSC) content and sugar metabolism indexes showed the highest levels in the 2-year culms and then declined in the 3-year culms. Compared to young culms that had not yet flowered, the 3-month-old and 1-year-old flowering culms had a significant decrease in the fiber length and tangential diameter, and their holocellulose and lignin levels also decreased, while the levels of ash, SiO2, 1% NaOH extractives, and benzene-ethanol extractives increased. A correlation analysis showed that sugar catabolism was accelerated in the flowering cluster, which could lead to “starvation death” in bamboo and which had a significant negative impact on the anatomical and chemical properties of the bamboo culms. Generally, the flowering bamboo culms had shorter fibers, higher levels of extractives and ash, and lower holocellulose content, which indicated that bamboo flowering has an adverse effect on the application of such components in the production of pulp, in papermaking, and in other processing and utilization activities. This study revealed the physiological changes in flowering B. tuldoides culms and provided a theoretical basis to inform the utilization of culms in this species