A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis

Aspect-Based Sentiment Analysis is a fine-grained sentiment analysis task, which focuses on detecting the sentiment polarity towards the aspect in a sentence. However, it is always sensitive to the multi-aspect challenge, where features of multiple aspects in a sentence will affect each other. To mitigate this issue, we design a novel training framework, called Contrastive Cross-Channel Data Augmentation (C3DA). A source sentence will be fed a domain-specific generator to obtain some synthetic sentences and is concatenated with these generated sentences to conduct supervised training and proposed contrastive training. To be specific, considering the limited ABSA labeled data, we also introduce some parameter-efficient approaches to complete sentences generation. This novel generation method consists of an Aspect Augmentation Channel (AAC) to generate aspect-specific sentences and a Polarity Augmentation (PAC) to generate polarity-inverted sentences. According to our extensive experiments, our C3DA framework can outperform those baselines without any augmentations by about 1\% on accuracy and Macro-F1

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation

Prompt Transfer (PoT) is a recently-proposed approach to improve prompt-tuning, by initializing the target prompt with the existing prompt trained on similar source tasks. However, such a vanilla PoT approach usually achieves sub-optimal performance, as (i) the PoT is sensitive to the similarity of source-target pair and (ii) directly fine-tuning the prompt initialized with source prompt on target task might lead to forgetting of the useful general knowledge learned from source task. To tackle these issues, we propose a new metric to accurately predict the prompt transferability (regarding (i)), and a novel PoT approach (namely PANDA) that leverages the knowledge distillation technique to alleviate the knowledge forgetting effectively (regarding (ii)). Extensive and systematic experiments on 189 combinations of 21 source and 9 target datasets across 5 scales of PLMs demonstrate that: 1) our proposed metric works well to predict the prompt transferability; 2) our PANDA consistently outperforms the vanilla PoT approach by 2.3% average score (up to 24.1%) among all tasks and model sizes; 3) with our PANDA approach, prompt-tuning can achieve competitive and even better performance than model-tuning in various PLM scales scenarios. We have publicly released our code in https://github.com/WHU-ZQH/PANDA.Comment: Accepted by IEEE TKD

Self-Evolution Learning for Discriminative Language Model Pretraining

Masked language modeling, widely used in discriminative language model (e.g., BERT) pretraining, commonly adopts a random masking strategy. However, random masking does not consider the importance of the different words in the sentence meaning, where some of them are more worthy to be predicted. Therefore, various masking strategies (e.g., entity-level masking) are proposed, but most of them require expensive prior knowledge and generally train from scratch without reusing existing model weights. In this paper, we present Self-Evolution learning (SE), a simple and effective token masking and learning method to fully and wisely exploit the knowledge from data. SE focuses on learning the informative yet under-explored tokens and adaptively regularizes the training by introducing a novel Token-specific Label Smoothing approach. Experiments on 10 tasks show that our SE brings consistent and significant improvements (+1.43~2.12 average scores) upon different PLMs. In-depth analyses demonstrate that SE improves linguistic knowledge learning and generalization.Comment: Accepted to Findings of ACL202

Vortex Dynamics in Rotating Rayleigh-B\'enard Convection

We investigate the spatial distribution and dynamics of the vortices in rotating Rayleigh-B\'enard convection in a reduced Rayleigh-number range $1.3{\le}Ra/Ra_{c}{\le}166$. Under slow rotations ($Ra{\gtrsim}10Ra_{c}$), the vortices are randomly distributed. The size-distribution of the Voronoi cells of the vortex centers is well described by the standard $\Gamma$ distribution. In this flow regime the vortices exhibit Brownian-type horizontal motion. The probability density functions of the vortex displacements are, however, non-Gaussian at short time scales. At modest rotating rates ($4Ra_{c}{\le}Ra{\lesssim}10Ra_{c}$) the centrifugal force leads to radial vortex motions, i.e., warm cyclones (cold anticyclones) moving towards (outward from) the rotation axis. The mean-square-displacements of the vortices increase faster than linearly at large time. This super-diffusive behavior can be satisfactorily explained by a Langevin model incorporating the centrifugal force. In the rapidly rotating regime ($1.6Ra_{c}{\le}Ra{\le}4Ra_{c}$) the vortices are densely distributed, with the size-distribution of their Voronoi cells differing significantly from the standard $\Gamma$ distribution. The hydrodynamic interaction of neighboring vortices results in formation of vortex clusters. Inside clusters the correlation of the vortex velocity fluctuations is scale free, with the correlation length being approximately $30\%$ of the cluster length. We examine the influence of cluster forming on the dynamics of individual vortex. Within clusters, cyclones exhibit inverse-centrifugal motion as they submit to the motion of strong anticyclones, while the velocity for outward motion of the anticyclones is increased. Our analysis show that the mobility of isolated vortices, scaled by their vorticity strength, is a simple power function of the Froude number

Synthesis and Anticancer Activity of 4β-Triazole-podophyllotoxin Glycosides

AbstractThe objective of present study was to investigate the effect of various sounds on the green mustard’s (Brassica Juncea) morphology characteristic and productivity. The plant has been subjected to three various sound, namely classical music (rhythmic violin music), machine and traffic noise, and mixed sound (classical music and traffic noise) with 70-75 dB sound pressure level, from germination to harvest for three hours (7-10 am.) each day. Six parameters, i.e. germination, plant height, leaf width, leaf lenght, total plant lenght, and fresh weight, related with growth and productivity of plant were been monitored on regular basis.The results showed classical music improves germination up to 15% for 36 hours, plant height 13,5%, leaf width 14,8%, leaf length 14,2%, and wet weight 57,1%. In general, exposure to classical music gives thebest results on the morphological characteristics and productivity of green mustard.Keywords: Sound exposure, plant morphology , productivity, green mustardAbstrakPenelitian ini bertujuan untuk menginvestigasi efek paparan variasi suara terhadap karakteristik morfologi dan produktivitas tanaman sawi hijau. suara yang dipaparkan antara lain musik klasik (suara biola), bising lalu lintas dan mesin industri (noise) dan campuran antara musik klasik dan noise. Level suara yang digunakan berkisar antara 70-75 dB dimulai sejak masa perkecambahan hingga panen selama 3 jam tiap harinya dimulai pukul 07.00-10.00. Enam parameter yang diamati dan diambil datanya meliputi, daya berkecambah, tinggi tanaman, lebar daun, panjang daun, panjang tanaman total dan berat basah. Hasil penelitian menunjukkan bahwa musik klasik meningkatkan daya berkecambah sebesar 15%, tinggi tanaman sebesar 13,5%, lebar daun sebesar 14,8%, panjang daun sebesar 14,2%, dan berat basah sebesar 57,1%. Secara umum paparan musik klasik memberikan hasil terbaik terhadap karakteristik morfologi dan produktivitas sawi hijau.Kata kunci: Paparan suara, morfologi, produktivitas, sawi hijauDiterima: 21 Oktober 2013;Disetujui: 28 Januari 201

Revisiting Token Dropping Strategy in Efficient BERT Pretraining

Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers. It can effectively reduce the training time without degrading much performance on downstream tasks. However, we empirically find that token dropping is prone to a semantic loss problem and falls short in handling semantic-intense tasks. Motivated by this, we propose a simple yet effective semantic-consistent learning method (ScTD) to improve the token dropping. ScTD aims to encourage the model to learn how to preserve the semantic information in the representation space. Extensive experiments on 12 tasks show that, with the help of our ScTD, token dropping can achieve consistent and significant performance gains across all task types and model sizes. More encouragingly, ScTD saves up to 57% of pretraining time and brings up to +1.56% average improvement over the vanilla token dropping.Comment: Accepted to ACL2023 Main Conferenc

Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks

Text classification tasks often encounter few shot scenarios with limited labeled data, and addressing data scarcity is crucial. Data augmentation with mixup has shown to be effective on various text classification tasks. However, most of the mixup methods do not consider the varying degree of learning difficulty in different stages of training and generate new samples with one hot labels, resulting in the model over confidence. In this paper, we propose a self evolution learning (SE) based mixup approach for data augmentation in text classification, which can generate more adaptive and model friendly pesudo samples for the model training. SE focuses on the variation of the model's learning ability. To alleviate the model confidence, we introduce a novel instance specific label smoothing approach, which linearly interpolates the model's output and one hot labels of the original samples to generate new soft for label mixing up. Through experimental analysis, in addition to improving classification accuracy, we demonstrate that SE also enhances the model's generalize ability

Towards Making the Most of ChatGPT for Machine Translation

ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g, low-resource and distant-language-pairs translation. However, they usually adopt simple prompts which can not fully elicit the capability of ChatGPT. In this report, we aim to further mine ChatGPT's translation ability by revisiting several aspects: temperature, task information, and domain information, and correspondingly propose two (simple but effective) prompts: Task-Specific Prompts (TSP) and Domain-Specific Prompts (DSP). We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information further improves ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community. We also explore the effects of advanced in-context learning strategies and find a (negative but interesting) observation: the powerful chain-of-thought prompt leads to word-by-word translation behavior, thus bringing significant translation degradation.Comment: Work in progress, 9 page