R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation

Recent text-to-image (T2I) diffusion models have achieved remarkable progress in generating high-quality images given text-prompts as input. However, these models fail to convey appropriate spatial composition specified by a layout instruction. In this work, we probe into zero-shot grounded T2I generation with diffusion models, that is, generating images corresponding to the input layout information without training auxiliary modules or finetuning diffusion models. We propose a Region and Boundary (R&B) aware cross-attention guidance approach that gradually modulates the attention maps of diffusion model during generative process, and assists the model to synthesize images (1) with high fidelity, (2) highly compatible with textual input, and (3) interpreting layout instructions accurately. Specifically, we leverage the discrete sampling to bridge the gap between consecutive attention maps and discrete layout constraints, and design a region-aware loss to refine the generative layout during diffusion process. We further propose a boundary-aware loss to strengthen object discriminability within the corresponding regions. Experimental results show that our method outperforms existing state-of-the-art zero-shot grounded T2I generation methods by a large margin both qualitatively and quantitatively on several benchmarks.Comment: Preprint. Under review. Project page: https://sagileo.github.io/Region-and-Boundar

Uniaxial Tension Simulation Using Real Microstructure-based Representative Volume Elements Model of Dual Phase Steel Plate

AbstractDual-phase steels have become a favored material for car bodies. In this study, the deformation behavior of dual-phase steels under uniaxial tension is investigated by means of 2D Representative Volume Elements (RVE) model. The real metallographic graphs including particle geometry, distribution and morphology are considered in this RVE model. Stress and strain distributions between martensite and ferrite are analyzed. The results show that martensite undertakes most stress without significant strain while ferrite shares the most strain. The tensile failure is the result of the deforming inhomogeneity between martensite phase and ferrite phase, which is the key factor triggering the plastic strain localization on specimen section during the tensile test

Knowledge-guided Pairwise Reconstruction Network for Weakly Supervised Referring Expression Grounding

Weakly supervised referring expression grounding (REG) aims at localizing the referential entity in an image according to linguistic query, where the mapping between the image region (proposal) and the query is unknown in the training stage. In referring expressions, people usually describe a target entity in terms of its relationship with other contextual entities as well as visual attributes. However, previous weakly supervised REG methods rarely pay attention to the relationship between the entities. In this paper, we propose a knowledge-guided pairwise reconstruction network (KPRN), which models the relationship between the target entity (subject) and contextual entity (object) as well as grounds these two entities. Specifically, we first design a knowledge extraction module to guide the proposal selection of subject and object. The prior knowledge is obtained in a specific form of semantic similarities between each proposal and the subject/object. Second, guided by such knowledge, we design the subject and object attention module to construct the subject-object proposal pairs. The subject attention excludes the unrelated proposals from the candidate proposals. The object attention selects the most suitable proposal as the contextual proposal. Third, we introduce a pairwise attention and an adaptive weighting scheme to learn the correspondence between these proposal pairs and the query. Finally, a pairwise reconstruction module is used to measure the grounding for weakly supervised learning. Extensive experiments on four large-scale datasets show our method outperforms existing state-of-the-art methods by a large margin.Comment: Accepted by ACMMM 2019. arXiv admin note: text overlap with arXiv:1908.1056

Co-immobilization of multiple enzymes by metal coordinated nucleotide hydrogel nanofibers: improved stability and an enzyme cascade for glucose detection

Preserving enzyme activity and promoting synergistic activity via co-localization of multiple enzymes are key topics in bionanotechnology, materials science, and analytical chemistry. This study reports a facile method for co-immobilizing multiple enzymes in metal coordinated hydrogel nanofibers. Specifically, four types of protein enzymes, including glucose oxidase, Candida rugosa lipase, a-amylase, and horseradish peroxidase, were respectively encapsulated in a gel nanofiber made of Zn2+ and adenosine monophosphate (AMP) with a simple mixing step. Most enzymes achieved quantitative loading and retained full activity. At the same time, the entrapped enzymes were more stable against temperature variation (by 7.5 degrees C), protease attack, extreme pH (by 2-fold), and organic solvents. After storing for 15 days, the entrapped enzyme still retained 70% activity while the free enzyme nearly completely lost its activity. Compared to nanoparticles formed with AMP and lanthanide ions, the nanofiber gels allowed much higher enzyme activity. Finally, a highly sensitive and selective biosensor for glucose was prepared using the gel nanofiber to co-immobilize glucose oxidase and horseradish peroxidase for an enzyme cascade system. A detection limit of 0.3 mu M glucose with excellent selectivity was achieved. This work indicates that metal coordinated materials using nucleotides are highly useful for interfacing with biomolecules.Beijing Higher Education Young Elite Teacher Project [YETP0520]; Fundamental Research Funds for the Central Universities [YS1407]; Beijing Natural Science Foundation [2162030]; China Scholarship Council; Natural Sciences and Engineering Research Council of Canada (NSERC