Enhancing Cross-Prompt Transferability in Vision-Language Models through Contextual Injection of Target Tokens

Vision-language models (VLMs) seamlessly integrate visual and textual data to perform tasks such as image classification, caption generation, and visual question answering. However, adversarial images often struggle to deceive all prompts effectively in the context of cross-prompt migration attacks, as the probability distribution of the tokens in these images tends to favor the semantics of the original image rather than the target tokens. To address this challenge, we propose a Contextual-Injection Attack (CIA) that employs gradient-based perturbation to inject target tokens into both visual and textual contexts, thereby improving the probability distribution of the target tokens. By shifting the contextual semantics towards the target tokens instead of the original image semantics, CIA enhances the cross-prompt transferability of adversarial images.Extensive experiments on the BLIP2, InstructBLIP, and LLaVA models show that CIA outperforms existing methods in cross-prompt transferability, demonstrating its potential for more effective adversarial strategies in VLMs.Comment: 13 page

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which is applicable to different DNN types, sizes, and application scenarios. The algorithm part adopts the general block-circulant matrices to achieve a fine-grained tradeoff between accuracy and compression ratio. It applies to both fully-connected and convolutional layers and contains a mathematically rigorous proof of the effectiveness of the method. The proposed algorithm reduces computational complexity per layer from O($n^2$) to O($n\log n$) and storage complexity from O($n^2$) to O($n$), both for training and inference. The hardware part consists of highly efficient Field Programmable Gate Array (FPGA)-based implementations using effective reconfiguration, batch processing, deep pipelining, resource re-using, and hierarchical control. Experimental results demonstrate that the proposed framework achieves at least 152X speedup and 71X energy efficiency gain compared with IBM TrueNorth processor under the same test accuracy. It achieves at least 31X energy efficiency gain compared with the reference FPGA-based work.Comment: 6 figures, AAAI Conference on Artificial Intelligence, 201

A Meet-in-the-Middle Attack on ARIA

In this paper, we study the meet-in-the-middle attack against block cipher ARIA. We find some new 3-round and 4-round distinguish- ing properties of ARIA. Based on the 3-round distinguishing property, we can apply the meet-in-the-middle attack with up to 6 rounds for all versions of ARIA. Based on the 4-round distinguishing property, we can mount a successful attack on 8-round ARIA-256. Furthermore, the 4-round distinguishing property could be improved which leads to a 7-round attack on ARIA-192. The data and time complexities of 7-round attack are 2^120 and 2^185:3, respectively. The data and time complexities of 8-round attack are 2^56 and 2^251:6, respectively. Compared with the existing cryptanalytic results on ARIA, our 5-round attack has the lowest data and time complexities and the 6-round attack has the lowest data complexity. Moreover, it is shown that 8-round ARIA-256 is not immune to the meet-in-the-middle attack

Hierarchical Interaction Networks with Rethinking Mechanism for Document-level Sentiment Analysis

Document-level Sentiment Analysis (DSA) is more challenging due to vague semantic links and complicate sentiment information. Recent works have been devoted to leveraging text summarization and have achieved promising results. However, these summarization-based methods did not take full advantage of the summary including ignoring the inherent interactions between the summary and document. As a result, they limited the representation to express major points in the document, which is highly indicative of the key sentiment. In this paper, we study how to effectively generate a discriminative representation with explicit subject patterns and sentiment contexts for DSA. A Hierarchical Interaction Networks (HIN) is proposed to explore bidirectional interactions between the summary and document at multiple granularities and learn subject-oriented document representations for sentiment classification. Furthermore, we design a Sentiment-based Rethinking mechanism (SR) by refining the HIN with sentiment label information to learn a more sentiment-aware document representation. We extensively evaluate our proposed models on three public datasets. The experimental results consistently demonstrate the effectiveness of our proposed models and show that HIN-SR outperforms various state-of-the-art methods.Comment: 17 pages, accepted by ECML-PKDD 202

Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting

Learning to rank (LtR) applies supervised machine learning (SML) technologies to the ranking problems, aiming at optimizing the relevance of input document list. As regard to previous studies on the deep ranking model, the calculation of the relevance of the documents in the list is independent of each other, which lacks consideration of document interactions. In recent years, some new methods are devoted to mining the interaction between documents, such as groupwise scoring function (GSF), which learns multivariate scoring function to jointly judge the correlation, but most of these methods ignore the differences of the interaction between documents, and bring high calculation cost at the same time. In order to solve this problem, this paper proposes a weighted groupwise deep ranking model (W-GSF). In view of the deep interest network in the field of recommendation, this paper intro-duces the idea of adjusting the weight of historical behavior sequence according to the candidate products. On the basis of multivariate scoring method in learning to rank field, this method uses muti-layer feed forword neural networks as main structure, and adds an activation unit into it before the input module, taking advantage of neural networks to adjust the weight of input multiple variables adaptively, so as to mine the differences of cross document relationship. Experiments on the public benchmark dataset MSLR verify the effectiveness of the method. Compared with baseline ranking models, the introduction of activation strategy brings a significant improvement of ranking metrics, and the computational complexity is greatly reduced compared with the same effect learning to rank methods

The deubiquitinase USP6 affects memory and synaptic plasticity through modulating NMDA receptor stability

人类与其他动物相比的重要区别在于人类拥有高等认知能力，这种能力集中体现在学习记忆和语言表达方面。厦门大学医学院神经科学研究所王鑫教授团队发现人科动物特异性基因USP6作为一个新的NMDA受体调控因子，可通过去泛素化途径调节NMDA型谷氨酸受体的降解和稳定性，进而调控突触可塑性和学习记忆能力。 本研究工作由王鑫教授指导完成，博士生曾凡伟、马学海与硕士生朱琳为共同第一作者，王鑫教授为通讯作者。Ubiquitin-specific protease (USP) 6 is a hominoid deubiquitinating enzyme previously implicated in intellectual disability and autism spectrum disorder. Although these findings link USP6 to higher brain function, potential roles for USP6 in cognition have not been investigated. Here, we report that USP6 is highly expressed in induced human neurons and that neuron-specific expression of USP6 enhances learning and memory in a transgenic mouse model. Similarly, USP6 expression regulates N-methyl-D-aspartate-type glutamate receptor (NMDAR)-dependent long-term potentiation and long-term depression in USP6 transgenic mouse hippocampi. Proteomic characterization of transgenic USP6 mouse cortex reveals attenuated NMDAR ubiquitination, with concomitant elevation in NMDAR expression, stability, and cell surface distribution with USP6 overexpression. USP6 positively modulates GluN1 expression in transfected cells, and USP6 down-regulation impedes focal GluN1 distribution at postsynaptic densities and impairs synaptic function in neurons derived from human embryonic stem cells. Together, these results indicate that USP6 enhances NMDAR stability to promote synaptic function and cognition.This work was partially supported by the National Natural Science Foundation of China (31871077, 81822014, 81571176 to XW; 81701349 to Hongfeng Z.; 81701130 to QZ; and 81471160 to HS), the National Key R&D Program of China (2016YFC1305900 to XW and HS), the Natural Science Foundation of Fujian Province of China (2017J06021 to XW), the Fundamental Research Funds for the Chinese Central Universities (20720150061 to XW and 20720180040 to ZS), Open Research Fund of State Key Laboratory of Cellular Stress Biology, Xiamen University (SKLCSB2019KF012 to QZ), and China Postdoctoral Science Foundation (2017M612130 to QZ).该研究得到了国家自然科学基金面上项目和优秀青年基金项目的支持