Conditional Prompt Tuning for Multimodal Fusion

We show that the representation of one modality can effectively guide the prompting of another modality for parameter-efficient multimodal fusion. Specifically, we first encode one modality and use its representation as a prior to conditionally prompt all frozen layers of the other modality. This is achieved by disentangling the vanilla prompt vectors into three types of specialized prompts that adaptively capture global-level and instance-level features. To better produce the instance-wise prompt, we introduce the mixture of prompt experts (MoPE) to dynamically route each instance to the most suitable prompt experts for encoding. We further study a regularization term to avoid degenerated prompt expert routing. Thanks to our design, our method can effectively transfer the pretrained knowledge in unimodal encoders for downstream multimodal tasks. Compared with vanilla prompting, we show that our MoPE-based conditional prompting is more expressive, thereby scales better with training data and the total number of prompts. We also demonstrate that our prompt tuning is architecture-agnostic, thereby offering high modularity. Extensive experiments over three multimodal datasets demonstrate state-of-the-art results, matching or surpassing the performance achieved through fine-tuning, while only necessitating 0.7% of the trainable parameters. Code will be released: https://github.com/songrise/ConditionalPrompt.Comment: under revie

Did You Train on My Dataset? Towards Public Dataset Protection with Clean-Label Backdoor Watermarking

The huge supporting training data on the Internet has been a key factor in the success of deep learning models. However, this abundance of public-available data also raises concerns about the unauthorized exploitation of datasets for commercial purposes, which is forbidden by dataset licenses. In this paper, we propose a backdoor-based watermarking approach that serves as a general framework for safeguarding public-available data. By inserting a small number of watermarking samples into the dataset, our approach enables the learning model to implicitly learn a secret function set by defenders. This hidden function can then be used as a watermark to track down third-party models that use the dataset illegally. Unfortunately, existing backdoor insertion methods often entail adding arbitrary and mislabeled data to the training set, leading to a significant drop in performance and easy detection by anomaly detection algorithms. To overcome this challenge, we introduce a clean-label backdoor watermarking framework that uses imperceptible perturbations to replace mislabeled samples. As a result, the watermarking samples remain consistent with the original labels, making them difficult to detect. Our experiments on text, image, and audio datasets demonstrate that the proposed framework effectively safeguards datasets with minimal impact on original task performance. We also show that adding just 1% of watermarking samples can inject a traceable watermarking function and that our watermarking samples are stealthy and look benign upon visual inspection

Study on perception threshold for whole-body vibration

When people stay in the vibrating environment for a long time, the body may produce a series of physiological and psychological diseases. In order to evaluate the impact of vibration on the human body, the establishment of evaluation method or evaluation system is necessary. At present, most countries usually evaluate whole-body vibration based on the international standard ISO 2631-1 “Mechanical vibration and shock-Evaluation of human exposure to whole-body vibration-Part 1: General requirements”. In this paper, the experiments of perception threshold of whole-body vibration were taken as the breakthrough point of evaluation method, and 12 subjects participated in the experiments. Through the experiments, comparing the provisions of ISO 2631-1, we get some different conclusions about the distribution law of perception thresholds. This also provides some data support for further experimental research

Treatment of Esophagogastric Anastomotic Leak with Perianastomotic Drain

IntroductionThe most efficient treatment of such anastomotic leaks after esophagectomy remains controversial. Our objective was to evaluate the effectiveness of perianastomotic drains in anastomotic leaks.MethodsFive patients with intrathoracic anastomotic leaks had placement of perianastomotic drains through remanet esophagus and fitula into infected area. The other conservative methods of treatment were also chosen simultaneously.ResultsThe perianastomotic drains were placed successfully in all five patients. None of the patients underwent rethoracotomy. They were all cured. The median period to clinical healing was 33 days. The median hospital stay after the perianastomotic drainage procedure was 37 days.ConclusionThis procedure proved to be safe and effective in the treatment of esophagogastric anastomotic leak with perianastomotic drain through fistula

Shear response behavior of STF/kevlar composite fabric in picture frame test

The picture frame test was applied to compare Kevlar neat and STF/Kevlar composite fabrics. The digital image correlation markers method was applied to measure the shear deformation behavior of the fabric in real-time under three loading rates: 100, 500, and 1000 mm/min. A theoretical model was applied to evaluate the effect of STF on the shear deformation stiffness of the fabric and cells and on the energy absorption during shear deformation. The results show that the STF/Kevlar composite fabric has a larger load-carrying capacity than the neat fabric in the picture frame test, and has obvious loading rate dependence. The yarn cell of the fabric undergoes slip deformation and reaches a shear-locked state; the shear modulus and the cell spring torsion coefficient of the STF/Kevlar composite fabric are significantly higher than those of neat fabric. The shear thickening behavior of STF occurs at higher loading rates, and the composite fabric has the highest shear deformation stiffness and shear energy absorption level

Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypots

In the field of natural language processing, the prevalent approach involves fine-tuning pretrained language models (PLMs) using local samples. Recent research has exposed the susceptibility of PLMs to backdoor attacks, wherein the adversaries can embed malicious prediction behaviors by manipulating a few training samples. In this study, our objective is to develop a backdoor-resistant tuning procedure that yields a backdoor-free model, no matter whether the fine-tuning dataset contains poisoned samples. To this end, we propose and integrate a honeypot module into the original PLM, specifically designed to absorb backdoor information exclusively. Our design is motivated by the observation that lower-layer representations in PLMs carry sufficient backdoor features while carrying minimal information about the original tasks. Consequently, we can impose penalties on the information acquired by the honeypot module to inhibit backdoor creation during the fine-tuning process of the stem network. Comprehensive experiments conducted on benchmark datasets substantiate the effectiveness and robustness of our defensive strategy. Notably, these results indicate a substantial reduction in the attack success rate ranging from 10\% to 40\% when compared to prior state-of-the-art methods

Genetic mapping of QTL for maize leaf width combining RIL and IF2 populations

Leaf width is an important component of plant architecture that affects light capture during photosynthesis and wind circulation under dense planting conditions. To improve understanding of the genetic mechanisms involved in leaf width at different positions, a comprehensive evaluation using the RIL (Recombinant Inbred Line) and IF2 (Immortalized F2) populations and a subsequent meta-analysis were performed. Forty-seven QTL associated with leaf width at different positions below the tassel were detected. The individual effects of QTL explained 3.5% to 17.0% of the observed phenotypic variation, and ten QTL explained over 10%. The initial QTL were integrated into eight mQTL (meta-QTL) through a meta-analysis. Our results suggested that leaf widths at different positions may be affected by several of the same mQTL and may also be regulated by many different mQTL. These results provide useful information for breeding high density tolerant inbred lines and hybrid cultivars, as well as for using marker-assisted selection for important mQTL

Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models

AI Safety is a major concern in many deep learning applications such as autonomous driving. Given a trained deep learning model, an important natural problem is how to reliably verify the model's prediction. In this paper, we propose a novel framework -- deep verifier networks (DVN) to verify the inputs and outputs of deep discriminative models with deep generative models. Our proposed model is based on conditional variational auto-encoders with disentanglement constraints. We give both intuitive and theoretical justifications of the model. Our verifier network is trained independently with the prediction model, which eliminates the need of retraining the verifier network for a new model. We test the verifier network on out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation. We achieve state-of-the-art results in all of these problems.Comment: Accepted to AAAI 202