f-Divergence Minimization for Sequence-Level Knowledge Distillation

Knowledge distillation (KD) is the process of transferring knowledge from a large model to a small one. It has gained increasing attention in the natural language processing community, driven by the demands of compressing ever-growing language models. In this work, we propose an f-DISTILL framework, which formulates sequence-level knowledge distillation as minimizing a generalized f-divergence function. We propose four distilling variants under our framework and show that existing SeqKD and ENGINE approaches are approximations of our f-DISTILL methods. We further derive step-wise decomposition for our f-DISTILL, reducing intractable sequence-level divergence to word-level losses that can be computed in a tractable manner. Experiments across four datasets show that our methods outperform existing KD approaches, and that our symmetric distilling losses can better force the student to learn from the teacher distribution.Comment: Accepted by ACL 202

Analysis of electrophysiological activation of the uterus during human labor contractions

This cohort study uses electromyometrial imaging to examine the underlying electrophysiological origins of human labor at the myometrium level

Magnetic resonance imaging of the supra-cervical fetal membrane detects an increased risk of prelabor rupture of membranes

OBJECTIVE: In 10% of term deliveries and 40% of preterm deliveries, the fetal membrane (FM) ruptures before labor. However, the ability to predict these cases of premature rupture of membranes (PROM) and preterm premature rupture of membranes (PPROM) is very limited. In this paper, our objective was to determine whether a prediction method based on T2 weighted magnetic resonance imaging (MRI) of the supra-cervical FM could predict PROM and PPROM. METHODS: This prospective cohort study enrolled 77 women between the 28th and 37th weeks of gestation. Two indicators of fetal membrane defects, including prolapsed depth \u3e5 mm and signal abnormalities, are investigated for our prediction. Fisher\u27s exact test was used to determine whether prolapsed depth \u3e5 mm and/or signal abnormalities were associated with PROM and PPROM. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated for prolapsed depth \u3e5 mm, signal abnormalities, and the combination of prolapsed depth \u3e5 mm and signal abnormalities. RESULT: Among 12 women with PROM (5 preterm and 7 term, prior to labor onset), 9 had membrane prolapse \u3e5 mm and 5 had FM signal abnormalities. Among 65 women with rupture of membranes at term, 2 had membrane prolapse \u3e5 mm and 1 had signal abnormalities. By Fisher\u27s exact test both indicators, membrane prolapse \u3e5 mm and signal abnormalities, were associated with PROM (P\u3c0.001, P\u3c0.001) and PPROM (P=0.001, P\u3c0.001). Additionally, membrane prolapse \u3e5 mm, signal abnormalities, and the combination of the two indicators all demonstrated high specificity for predicting PROM (96.9%, 98.5%, and 100%, respectively) and PPROM (90.3%, 97.2%, and 100%, respectively). CONCLUSION: MRI can distinguish the supra-cervical fetal membran

In vivo assessment of supra-cervical fetal membrane by MRI 3D CISS: A preliminary study

In approximately 8% of term births and 33% of pre-term births, the fetal membrane (FM) ruptures before delivery

Noninvasive electromyometrial imaging of human uterine maturation during term labor

Electromyometrial imaging (EMMI) was recently developed to image the three-dimensional (3D) uterine electrical activation during contractions noninvasively and accurately in sheep. Herein we describe the development and application of a human EMMI system to image and evaluate 3D uterine electrical activation patterns at high spatial and temporal resolution during human term labor. We demonstrate the successful integration of the human EMMI system during subjects\u27 clinical visits to generate noninvasively the uterine surface electrical potential maps, electrograms, and activation sequence through an inverse solution using up to 192 electrodes distributed around the abdomen surface. Quantitative indices, including the uterine activation curve, are developed and defined to characterize uterine surface contraction patterns. We thus show that the human EMMI system can provide detailed 3D images and quantification of uterine contractions as well as novel insights into the role of human uterine maturation during labor progression