Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation

Transformer-based speech self-supervised learning (SSL) models, such as HuBERT, show surprising performance in various speech processing tasks. However, huge number of parameters in speech SSL models necessitate the compression to a more compact model for wider usage in academia or small companies. In this study, we suggest to reuse attention maps across the Transformer layers, so as to remove key and query parameters while retaining the number of layers. Furthermore, we propose a novel masking distillation strategy to improve the student model's speech representation quality. We extend the distillation loss to utilize both masked and unmasked speech frames to fully leverage the teacher model's high-quality representation. Our universal compression strategy yields the student model that achieves phoneme error rate (PER) of 7.72% and word error rate (WER) of 9.96% on the SUPERB benchmark.Comment: Interspeech 2023. Code URL: https://github.com/sungnyun/ARMHuBER

The Risk of Type 2 Diabetes Mellitus according to Changes in Obesity Status in Late Middle-Aged Adults: A Nationwide Cohort Study of Korea

Background Although obesity is a well-known risk factor of type 2 diabetes mellitus (T2DM), there is scant data on discriminating the contribution of previous obesity and recent weight gain on developing T2DM. Methods We analyzed the Korean National Health Insurance Service-Health Screening Cohort data from 2002 to 2015 where Korean residents underwent biennial health checkups. Participants were classified into four groups according to their obesity status (body mass index [BMI] ≥25 kg/m2) before and after turning 50 years old: maintaining normal (MN), becoming obese (BO), becoming normal (BN), and maintaining obese (MO). Cox proportional hazards regression model was used to estimate the risk of T2DM factoring in the covariates age, sex, BMI, presence of impaired fasting glucose or hypertension, family history of diabetes, and smoking status. Results A total of 118,438 participants (mean age, 52.5±1.1 years; men, 45.2%) were prospectively evaluated for incident T2DM. A total of 7,339 (6.2%) participants were diagnosed with T2DM during a follow-up period of 4.8±2.6 years. Incidence rates of T2DM per 1,000 person-year were 9.20 in MN, 14.81 in BO, 14.42 in BN, 21.38 in MO. After factoring in covariates, participants in the groups BN (adjusted hazard ratio [aHR], 1.15; 95% confidence interval [CI], 1.04 to 1.27) and MO (aHR, 1.14; 95% CI, 1.06 to 1.24) were at increased risk of developing T2DM compared to MN, whereas BO (hazard ratio, 1.06; 95% CI, 0.96 to 1.17) was not. Conclusion Having been obese before 50 years old increased the risk of developing T2DM in the future, but becoming obese after 50 did not. Therefore, it is important to maintain normal weight from early adulthood to prevent future metabolic perturbations

Anti-allergic and anti-inflammatory effects of butanol extract from Arctium Lappa L

Background: Atopic dermatitis is a chronic, allergic inflammatory skin disease that is accompanied by markedly increased levels of inflammatory cells, including eosinophils, mast cells, and T cells. Arctium lappa L. is a traditional medicine in Asia. This study examined whether a butanol extract of A. lappa (ALBE) had previously unreported anti-allergic or anti-inflammatory effects.Methods: This study examined the effect of ALBE on the release of ??-hexosaminidase in antigen-stimulated-RBL-2H3 cells. We also evaluated the ConA-induced expression of IL-4, IL-5, mitogen-activated protein kinases (MAPKs), and nuclear factor (NF)-??B using RT-PCR, Western blotting, and ELISA in mouse splenocytes after ALBE treatment.Results: We observed significant inhibition of ??-hexosaminidase release in RBL-2H3 cells and suppressed mRNA expression and protein secretion of IL-4 and IL-5 induced by ConA-treated primary murine splenocytes after ALBE treatment. Additionally, ALBE (100 ??g/mL) suppressed not only the transcriptional activation of NF-??B, but also the phosphorylation of MAPKs in ConA-treated primary splenocytes.Conclusions: These results suggest that ALBE inhibits the expression of IL-4 and IL-5 by downregulating MAPKs and NF-??B activation in ConA-treated splenocytes and supports the hypothesis that ALBE may have beneficial effects in the treatment of allergic diseases, including atopic dermatitis. ?? 2011 Sohn et al; licensee BioMed Central Ltd

Ugly or Pretty: The Effects of Aesthetics and Exercise Involvement on Consumers’ Evaluations of Healthy Functional Foods

PURPOSE This study explores the impact of visual shape and exercise involvement on consumers' evaluation and happiness with healthy functional foods, specifically protein cookies. METHODS Using a one-factor design with two levels (pretty vs. ugly shape) and one measured variable (exercise involvement), we uncover interesting insights. RESULTS Consumers highly engaged in exercise show a greater purchase intention for ugly-shaped healthy functional foods compared to pretty-shaped ones. Conversely, consumers with low exercise involvement express a higher purchase intention and happiness when it comes to pretty-shaped healthy functional foods compared to their ugly counterparts. CONCLUSIONS These findings contribute to the sports management and sports science literature by shedding light on how visual shape influences the evaluation of healthy functional foods by sport consumers. Furthermore, this research offers valuable practical implications for designing the shape of such foods to cater to the preferences of sports enthusiasts

Molecular Subgroup Analysis of Clinical Outcomes in a Phase 3 Study of Gemcitabine and Oxaliplatin with or without Erlotinib in Advanced Biliary Tract Cancer

AbstractBACKGROUND: We previously reported that the addition of erlotinib to gemcitabine and oxaliplatin (GEMOX) resulted in greater antitumor activity and might be a treatment option for patients with biliary tract cancers (BTCs). Molecular subgroup analysis of treatment outcomes in patients who had specimens available for analysis was undertaken. METHODS: Epidermal growth factor receptor (EGFR), KRAS, and PIK3CA mutations were evaluated using peptide nucleic acid–locked nucleic acid polymerase chain reaction clamp reactions. Survival and response rates (RRs) were analyzed according to the mutational status. Sixty-four patients (48.1%) were available for mutational analysis in the chemotherapy alone group and 61 (45.1%) in the chemotherapy plus erlotinib group. RESULTS: 1.6% (2/116) harbored an EGFR mutation (2 patients; exon 20), 9.6% (12/121) harbored a KRAS mutation (12 patients; exon 2), and 9.6% (12/118) harbored a PIK3CA mutation (10 patients, exon 9 and 2 patients, exon 20). The addition of erlotinib to GEMOX in patients with KRAS wild-type disease (n = 109) resulted in significant improvements in overall response compared with GEMOX alone (30.2% vs 12.5%, P = .024). In 95 patients with both wild-type KRAS and PIK3CA, there was evidence of a benefit associated with the addition of erlotinib to GEMOX with respect to RR as compared with GEMOX alone (P = .04). CONCLUSION: This study demonstrates that KRAS mutational status might be considered a predictive biomarker for the response to erlotinib in BTCs. Additionally, the mutation status of PIK3CA may be a determinant for adding erlotinib to chemotherapy in KRAS wild-type BTCs

High-Throughput Genotyping in Metastatic Esophageal Squamous Cell Carcinoma Identifies Phosphoinositide-3-Kinase and BRAF Mutations

Background: Given the high incidence of metastatic esophageal squamous cell carcinoma, especially in Asia, we screened for the presence of somatic mutations using OncoMap platform with the aim of defining subsets of patients who may be potential candidate for targeted therapy. Methods and Materials We analyzed 87 tissue specimens obtained from 80 patients who were pathologically confirmed with esophageal squamous cell carcinoma and received 5-fluoropyrimidine/platinum-based chemotherapy. OncoMap 4.0, a mass-spectrometry based assay, was used to interrogate 471 oncogenic mutations in 41 commonly mutated genes. Tumor specimens were prepared from primary cancer sites in 70 patients and from metastatic sites in 17 patients. In order to test the concordance between primary and metastatic sites from the patient for mutations, we analyzed 7 paired (primary-metastatic) specimens. All specimens were formalin-fixed paraffin embedded tissues and tumor content was >70%. Results: In total, we have detected 20 hotspot mutations out of 80 patients screened. The most frequent mutation was PIK3CA mutation (four E545K, five H1047R and one H1047L) (N = 10, 11.5%) followed by MLH1 V384D (N = 7, 8.0%), TP53 (R306, R175H and R273C) (N = 3, 3.5%), BRAF V600E (N = 1, 1.2%), CTNNB1 D32N (N = 1, 1.2%), and EGFR P733L (N = 1, 1.2%). Distributions of somatic mutations were not different according to anatomic sites of esophageal cancer (cervical/upper, mid, lower). In addition, there was no difference in frequency of mutations between primary-metastasis paired samples. Conclusions: Our study led to the detection of potentially druggable mutations in esophageal SCC which may guide novel therapies in small subsets of esophageal cancer patients

Controlling the Magnetic Anisotropy of the van der Waals Ferromagnet Fe3GeTe2 through Hole Doping.

Identifying material parameters affecting properties of ferromagnets is key to optimized materials that are better suited for spintronics. Magnetic anisotropy is of particular importance in van der Waals magnets, since it not only influences magnetic and spin transport properties, but also is essential to stabilizing magnetic order in the two-dimensional limit. Here, we report that hole doping effectively modulates the magnetic anisotropy of a van der Waals ferromagnet and explore the physical origin of this effect. Fe3-xGeTe2 nanoflakes show a significant suppression of the magnetic anisotropy with hole doping. Electronic structure measurements and calculations reveal that the chemical potential shift associated with hole doping is responsible for the reduced magnetic anisotropy by decreasing the energy gain from the spin-orbit induced band splitting. Our findings provide an understanding of the intricate connection between electronic structures and magnetic properties in two-dimensional magnets and propose a method to engineer magnetic properties through doping

Quaternary structures of Vac8 differentially regulate the Cvt and PMN pathways.

Armadillo (ARM) repeat proteins constitute a large protein family with diverse and fundamental functions in all organisms, and armadillo repeat domains share high structural similarity. However, exactly how these structurally similar proteins can mediate diverse functions remains a long-standing question. Vac8 (vacuole related 8) is a multifunctional protein that plays pivotal roles in various autophagic pathways, including piecemeal microautophagy of the nucleus (PMN) and cytoplasm-to-vacuole targeting (Cvt) pathways in the budding yeast Saccharomyces cerevisiae. Vac8 comprises an H1 helix at the N terminus, followed by 12 armadillo repeats. Herein, we report the crystal structure of Vac8 bound to Atg13, a key component of autophagic machinery. The 70-angstrom extended loop of Atg13 binds to the ARM domain of Vac8 in an antiparallel manner. Structural, biochemical, and in vivo experiments demonstrated that the H1 helix of Vac8 intramolecularly associates with the first ARM and regulates its self-association, which is crucial for Cvt and PMN pathways. The structure of H1 helix-deleted Vac8 complexed with Atg13 reveals that Vac8[Delta 19-33]-Atg13 forms a heterotetramer and adopts an extended superhelical structure exclusively employed in the Cvt pathway. Most importantly, comparison of Vac8-Nvj1 and Vac8-Atg13 provides a molecular understanding of how a single ARM domain protein adopts different quaternary structures depending on its associated proteins to differentially regulate 2 closely related but distinct cellular pathways