Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words

The performance of sentence encoders can be significantly improved through the simple practice of fine-tuning using contrastive loss. A natural question arises: what characteristics do models acquire during contrastive learning? This paper theoretically and experimentally shows that contrastive-based sentence encoders implicitly weight words based on information-theoretic quantities; that is, more informative words receive greater weight, while others receive less. The theory states that, in the lower bound of the optimal value of the contrastive learning objective, the norm of word embedding reflects the information gain associated with the distribution of surrounding words. We also conduct comprehensive experiments using various models, multiple datasets, two methods to measure the implicit weighting of models (Integrated Gradients and SHAP), and two information-theoretic quantities (information gain and self-information). The results provide empirical evidence that contrastive fine-tuning emphasizes informative words.Comment: 16 pages, 6 figures, accepted to EMNLP 2023 Findings (short paper

Sex-inducing effects toward planarians widely present among parasitic flatworms

Summary Various parasitic flatworms infect vertebrates for sexual reproduction, often causing devastating diseases in their hosts. Consequently, flatworms are of great socioeconomic and biomedical importance. Although the cessation of parasitic flatworm sexual reproduction is a major target of anti-parasitic drug design, little is known regarding bioactive compounds controlling flatworm sexual maturation. Using the planarian Dugesia ryukyuensis, we observed that sex-inducing substances found in planarians are also widespread in parasitic flatworms, such as monogeneans and flukes (but not in tapeworms). Reverse-phase HPLC analysis revealed the sex-inducing substance(s) eluting around the tryptophan retention time in the fluke Calicophoron calicophorum, consistent with previous studies on the planarian Bipalium nobile, suggesting that the substance(s) is likely conserved among flatworms. Moreover, six of the 18 ovary-inducing substances identified via transcriptome and metabolome analyses are involved in purine metabolism. Our findings provide a basis for understanding and modifying the life cycles of various parasitic flatworms.journal articl

18. asırda "Hırkai şerif" ziyareti

Taha Toros Arşivi, Dosya No: 120-Saraylar. Not: Gazetenin "Tarihten Sahifeler" köşesinde yayımlanmıştır.İstanbul Kalkınma Ajansı (TR10/14/YEN/0033) İstanbul Development Agency (TR10/14/YEN/0033

Use of Liquid Biopsy to Detect PIK3CA Mutation in Metastatic Breast Cancer

journal articl

Additional file 3: of A high-resolution physical map integrating an anchored chromosome with the BAC physical maps of wheat chromosome 6B

Final fingerprinting data used for contig assembly after BAC filtering. (PDF 33 kb