Mixed Distillation Helps Smaller Language Model Better Reasoning

While large language models (LLMs) have demonstrated exceptional performance in recent natural language processing (NLP) tasks, their deployment poses substantial challenges due to high computational and memory demands in real-world applications. Recent studies have focused on enhancing smaller models through knowledge distillation from LLMs, yielding promising results. However, these models often struggle to match the performance of LLMs, especially in tasks that require reasoning. In this work, we introduce Mixed Distillation (MD) framework, which capitalizes on the strengths of Program of Thought (PoT) and Chain of Thought (CoT) capabilities within LLMs, combining multiple prompting techniques and distilling these capabilities into smaller models. Our experimental results show that MD significantly enhances the single-path and multi-path reasoning ability of smaller models in various tasks. In terms of accuracy and generality of reasoning tasks, the model generated by it exceeds the comprehensive performance of two individually distilled models. Notably, LLaMA2-7B and CodeLlama-7B using MD achieved remarkable improvements of (84.5%) and (85.5%), respectively, outperforming GPT-3.5-Turbo by (2.5%) and (3.5%), on the SVAMP benchmark.Comment: Working in Progress, 17 pages, 16 figure

SRY gene transferred by extracellular vesicles accelerates atherosclerosis by promotion of leucocyte adherence to endothelial cells

Abstract We set out to investigate whether and how SRY (sex-determining region, Y) DNAs in plasma EVs (extracellular vesicles) is involved in the pathogenesis of atherosclerosis. PCR and gene sequencing found the SRY gene fragment in plasma EVs from male, but not female, patients; EVs from male patients with CAD (coronary artery disease) had a higher SRY GCN (gene copy number) than healthy subjects. Additional studies found that leucocytes, the major source of plasma EVs, had higher SRY GCN and mRNA and protein expression in male CAD patients than controls. After incubation with EVs from SRY-transfected HEK (human embryonic kidney)-293 cells, monocytes (THP-1) and HUVECs (human umbilical vein endothelial cells), which do not endogenously express SRY protein, were found to express newly synthesized SRY protein. This resulted in an increase in the adherence factors CD11-a in THP-1 cells and ICAM-1 (intercellular adhesion molecule 1) in HUVECs. EMSA showed that SRY protein increased the promoter activity of CD11-a in THP-1 cells and ICAM-1 in HUVECs. There was an increase in THP-1 cells adherent to HUVECs after incubation with SRY-EVs. SRY DNAs transferred from EVs have pathophysiological significance in vivo; injection of SRY EVs into ApoE −/− (apolipoprotein-knockout) mice accelerated atherosclerosis. The SRY gene in plasma EVs transferred to vascular endothelial cells may play an important role in the pathogenesis of atherosclerosis; this mechanism provides a new approach to the understanding of inheritable CAD in men

Tracing dynamic evolution from cross-dialect comparisons : a case study of tonal systems in Songzi area

Songzi of Hubei, located at a boundary area influenced by Xiang and Gan, now is classified into Subgroup Changhe of Southwestern Mandarin though it is quite distinct from the other members in this subgroup. It is also at the southern tip of a narrow area dividing Hubei into two dialectal parts with or without Rusheng (Entering Tone). This paper attempts to find out the evolutionary process of Rusheng and the phonological systems among Songzi dialects. Based on the Pan-chronic Phonology, the methods of Acoustic and Auditory Phonetics and a broad fieldwork, the study reveals that Songzi dialect can be further divided into three phonological systems, i.e. Weishui, Nanhai and Songzihe, which manifest a dynamic tonal evolution. Except for an extra Rusheng category in Weishui, Weishui and Nanhai are identical in their tonal systems; then a tonal flip-flop of Yinping and Qusheng occurs in Nanhai and Songzihe. In addition, according to the four parameters to define Rusheng, the Rusheng category of Weishui is undergoing the following changes: (1) this Rusheng in falsetto or fortis voice falls in two ways, which implies its downward movement from Register High to Register Mid; (2) though the glottal stop ending ? is at the edge of a total disappearance, it still exists and is realized in two ways; (3) its duration almost approximates to the other non-Rusheng tones; (4) as to the pitch of Weishui's Rusheng, two contours co-occur, which are being lowered or directly changed into other non-Rusheng tones. In the previous literature, the tonal systems of Songzi dialects were defined under the Five-point Scale. However, with the new findings of this research, there is a challenge in the need for a new theory for the definition of tones. Key words：Rusheng, Songzi, Falsetto, Southwestern Mandarin, Language Evolution, Tonal flip-flo