On Evaluating Adversarial Robustness of Large Vision-Language Models

Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses. Our findings provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice. Code is at https://github.com/yunqing-me/AttackVLM.Comment: NeurIPS 202

Collaborative Filtering With User-Item Co-Autoregressive Models

Deep neural networks have shown promise in collaborative filtering (CF). However, existing neural approaches are either user-based or item-based, which cannot leverage all the underlying information explicitly. We propose CF-UIcA, a neural co-autoregressive model for CF tasks, which exploits the structural correlation in the domains of both users and items. The co-autoregression allows extra desired properties to be incorporated for different tasks. Furthermore, we develop an efficient stochastic learning algorithm to handle large scale datasets. We evaluate CF-UIcA on two popular benchmarks: MovieLens 1M and Netflix, and achieve state-of-the-art performance in both rating prediction and top-N recommendation tasks, which demonstrates the effectiveness of CF-UIcA

Auxin-producing bacteria promote barley rhizosheath formation

Abstract The rhizosheath, or the layer of soil closely adhering to roots, can help plants to tolerate drought under moderate soil drying conditions. Rhizosheath formation is the result of poorly understood interactions between root exudates, microbes, and soil conditions. Here, we study the roles played by the soil microbiota in rhizosheath formation in barley (a dry crop). We show that barley rhizosheath formation is greater in acid soil than in alkaline soil, and inoculation with microbiota from acid soil enhances rhizosheath formation in alkaline soil. The rhizosheath-promoting activity is associated with the presence of Flavobacteriaceae and Paenibacillaceae bacteria that express genes for biosynthesis of indole-3-acetic acid (IAA, a common auxin), as determined by metagenomics and metatranscriptomics. Two bacterial strains isolated from rhizosheath (Chryseobacterium culicis and Paenibacillus polymyxa) produce IAA and enhance barley rhizosheath formation, while their IAA-defective mutants are unable to promote rhizosheath formation. Co-inoculation with the IAA-producing strains enhances barley grain yield in field experiments through an increase in spike number. Our findings contribute to our understanding of barley rhizosheath formation, and suggest potential strategies for crop improvement