Advancing Visual Grounding with Scene Knowledge: Benchmark and Method

Visual grounding (VG) aims to establish fine-grained alignment between vision and language. Ideally, it can be a testbed for vision-and-language models to evaluate their understanding of the images and texts and their reasoning abilities over their joint space. However, most existing VG datasets are constructed using simple description texts, which do not require sufficient reasoning over the images and texts. This has been demonstrated in a recent study~\cite{luo2022goes}, where a simple LSTM-based text encoder without pretraining can achieve state-of-the-art performance on mainstream VG datasets. Therefore, in this paper, we propose a novel benchmark of \underline{S}cene \underline{K}nowledge-guided \underline{V}isual \underline{G}rounding (SK-VG), where the image content and referring expressions are not sufficient to ground the target objects, forcing the models to have a reasoning ability on the long-form scene knowledge. To perform this task, we propose two approaches to accept the triple-type input, where the former embeds knowledge into the image features before the image-query interaction; the latter leverages linguistic structure to assist in computing the image-text matching. We conduct extensive experiments to analyze the above methods and show that the proposed approaches achieve promising results but still leave room for improvement, including performance and interpretability. The dataset and code are available at \url{https://github.com/zhjohnchan/SK-VG}.Comment: Computer Vision and Natural Language Processing. 21 pages, 14 figures. CVPR-202

Is synthetic data from generative models ready for image recognition?

Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images. Though the results are astonishing to human eyes, how applicable these generated images are for recognition tasks remains under-explored. In this work, we extensively study whether and how synthetic images generated from state-of-the-art text-to-image generation models can be used for image recognition tasks, and focus on two perspectives: synthetic data for improving classification models in data-scarce settings (i.e. zero-shot and few-shot), and synthetic data for large-scale model pre-training for transfer learning. We showcase the powerfulness and shortcomings of synthetic data from existing generative models, and propose strategies for better applying synthetic data for recognition tasks. Code: https://github.com/CVMI-Lab/SyntheticData.Comment: ICLR 2023, spotligh

Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation

Unsupervised learning of optical flow, which leverages the supervision from view synthesis, has emerged as a promising alternative to supervised methods. However, the objective of unsupervised learning is likely to be unreliable in challenging scenes. In this work, we present a framework to use more reliable supervision from transformations. It simply twists the general unsupervised learning pipeline by running another forward pass with transformed data from augmentation, along with using transformed predictions of original data as the self-supervision signal. Besides, we further introduce a lightweight network with multiple frames by a highly-shared flow decoder. Our method consistently gets a leap of performance on several benchmarks with the best accuracy among deep unsupervised methods. Also, our method achieves competitive results to recent fully supervised methods while with much fewer parameters.Comment: Accepted to CVPR 2020, https://github.com/lliuz/ARFlo

c-Abl Kinase Is a Regulator of αvβ3 Integrin Mediated Melanoma A375 Cell Migration.

Integrins are heterodimeric transmembrane receptors that physically link the extracellular matrix (ECM) to the intracellular actin cytoskeleton, and are also signaling molecules that transduce signals bi-directionally across the plasma membrane. Integrin regulation is essential for tumor cell migration in response to growth factors. c-Abl kinase is a nonreceptor tyrosine kinase and is critical for signaling transduction from various receptors. Here we show that c-Abl kinase is involved in A375 cell migration mediated by αvβ3 integrin in response to PDGF stimulation. c-Abl kinase colocalizes with αvβ3 integrin dynamically and affects αvβ3 integrin affinity by regulating its cluster. The interaction between c-Abl kinase and αvβ3 integrin was dependent on the activity of c-Abl kinase induced by PDGF stimulation, but was not dependent on the binding of αvβ3 integrin with its ligands, suggesting that c-Abl kinase is not involved in the outside-in signaling of αvβ3 integrin. Talin head domain was required for the interaction between c-Abl kinase and αvβ3 integrin, and the SH3 domain of c-Abl kinase was involved in its interaction with talin and αvβ3 integrin. Taken together, we have uncovered a novel and critical role of c-Abl kinase in αvβ3 integrin mediated melanoma cell migration

Diversity, knowledge, and valuation of plants used as fermentation starters for traditional glutinous rice wine by Dong communities in Southeast Guizhou, China

Abstract Background Beverages prepared by fermenting plants have a long history of use for medicinal, social, and ritualistic purposes around the world. Socio-linguistic groups throughout China have traditionally used plants as fermentation starters (or koji) for brewing traditional rice wine. The objective of this study was to evaluate traditional knowledge, diversity, and values regarding plants used as starters for brewing glutinous rice wine in the Dong communities in the Guizhou Province of China, an area of rich biological and cultural diversity. Methods Semi-structured interviews were administered for collecting ethnobotanical data on plants used as starters for brewing glutinous rice wine in Dong communities. Field work was carried out in three communities in Guizhou Province from September 2017 to July 2018. A total of 217 informants were interviewed from the villages. Results A total of 60 plant species were identified to be used as starters for brewing glutinous rice wine, belonging to 58 genera in 36 families. Asteraceae and Rosaceae are the most represented botanical families for use as a fermentation starter for rice wine with 6 species respectively, followed by Lamiaceae (4 species); Asparagaceae, Menispermaceae, and Polygonaceae (3 species respectively); and Lardizabalaceae, Leguminosae, Moraceae, Poaceae, and Rubiaceae (2 species, respectively). The other botanical families were represented by one species each. The species used for fermentation starters consist of herbs (60.0%), shrubs (23.3%), climbers (10.0%), and trees (6.7%). The parts used include the root (21.7%), leaf (20.0%), and the whole plant (16.7%). Findings indicate a significant relationship between knowledge of plants used as fermentation starters with age (P value < 0.001) and educational status (P value = 0.004) but not with gender (P value = 0.179) and occupation (P value = 0.059). The species that are most used by informants include Pueraria lobata var. montana (Lour.) van der Maesen (UV = 1.74; Leguminosae), Actinidia eriantha Benth. (UV = 1.51; Actinidiaceae), Oryza sativa L. var. glutinosa Matsum (UV = 1.5; Poaceae). Conclusion This study highlights that while most of the Dong informants continue to use a diverse range of plants as a fermentation starter for brewing glutinous rice wine, knowledge of these plants is being lost by the younger generations. Documentation of traditional ethnobotanical knowledge and outreach is thus needed to conserve biocultural diversity in the rural Dong communities in southern China

Pore structure and movable fluid characteristics of tight sandstone reservoirs in the Lower Shihezi Formation in the Hangjinqi area, Ordos Basin

Objective In recent years, new discoveries have been made in the He 1 Member of the Hangjinqi Gas Field. And the initial gas production of some wells by fracturing can reach 10×104 m3/d, which shows that the He 1 Member has great exploration potential. However, due to the strong heterogeneity of the He 1 Member, the gas production mechanism of the He 1 Member is not clear at present, which restricts its efficient development. To accurately and quantitatively characterize the microscopic pore structure and movable fluid characteristics of tight gas sandstone reservoirs. Methods In this paper, taking the tight sandstone reservoir in the He 1 Member of the Shangshihezi Formation in the Hangjinqi Gas Field as an example, NMR and CT tests were used to study the response characteristics and fluid identification ability of different types of pores in tight sandstone. Results The research shows that the porosity of the test samples is mainly distributed in 1.7%-10%, and the gas permeability is mainly distributed in 0.1×10-3-1.4×10-3 μm2, which belongs to the typical low-porosity and low-permeability porous tight sandstone reservoir. According to the T2 relaxation time curve of the saturation component before centrifugation, the pore type of the reservoir in the He 1 Member is bimodal (mainly the left peak, the right peak is not obvious), including 3 subtypes: micropore-small pore type, small pore-mesopore type, micropore-small pore-mesopore type. The T2 relaxation time intervals corresponding to the above three subtypes of pore types are 0.1-10 ms, 1-100 ms, and 0.1-100 ms, respectively. The results of 3D CT scans show that the micropore-mesopore reservoir has the best physical properties, followed by the micropore-small pore-mesopore type, and the micropore-small pore type reservoirs have relatively poor physical properties. The T2 cut-off values of the tested samples were mainly distributed between 1 and 14 ms, with an average value of 6.11 ms. There is a certain negative correlation between the T2 cut-off value and the movable fluid percentage of the rock samples. The movable fluid porosity and permeability have a very good positive correlation, reflecting that the amount of movable fluid is significantly affected by the reservoir permeability and the number of throats. The movable water saturation in the tight sandstone reservoirs of He 1 Member in the study area is mainly distributed at 4%-9%, with an average value of 5.8%. Conclusion Overall, the original movable water saturation of the He 1 Member is low and has great development potential

Enhanced Control of the Fungus Gnat Bradysia odoriphaga (Diptera: Sciaridae) by Co-Application of Clothianidin and Hexaflumuron

The fungus gnat is a major pest of chive in China. Its control has been relied heavily on the application of clothianidin. Due to the intensive application, its control efficacy become reduced. The present study was intended to evaluate co-drenching of clothianidin with hexaflumuron on absorption and dissipation of clothianidin in chive plants and soils and determine the effect of such application on control efficacies. Chive production fields in Guangdong and Hubei Provinces were drenched with clothianidin alone and a mixture of clothianidin and hexaflumuron at low application rates. Concentrations of clothianidin in chive plants and soils were analyzed by HPLC. Results showed that co-application had higher control efficacies against the fungus gnat than clothianidin alone. The co-application enhanced clothianidin absorption and dissipation and extended the half-lives of clothianidin in chive. It was likely that hexaflumuron protected chive roots from larva damage, and healthy roots absorbed more clothianidin, resulting in the extension of the half-lives. Additionally, the terminal residues of clothianidin in chive after 14 days of application were lower than the maximum residue limit in chive set by the Codex Alimentarius Commission. This study for the first time documented that co-application of clothianidin and hexaflumuron improved chive plants in absorption and dissipation of clothianidin and enhanced fungus gnat control efficacies