DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling

Pre-trained models like BERT (Devlin et al., 2018) have dominated NLP / IR applications such as single sentence classification, text pair classification, and question answering. However, deploying these models in real systems is highly non-trivial due to their exorbitant computational costs. A common remedy to this is knowledge distillation (Hinton et al., 2015), leading to faster inference. However -- as we show here -- existing works are not optimized for dealing with pairs (or tuples) of texts. Consequently, they are either not scalable or demonstrate subpar performance. In this work, we propose DiPair -- a novel framework for distilling fast and accurate models on text pair tasks. Coupled with an end-to-end training strategy, DiPair is both highly scalable and offers improved quality-speed tradeoffs. Empirical studies conducted on both academic and real-world e-commerce benchmarks demonstrate the efficacy of the proposed approach with speedups of over 350x and minimal quality drop relative to the cross-attention teacher BERT model.Comment: 13 pages. Accepted to Findings of EMNLP 202

Finding Best Semantic Relatedness Functions For Schema Matchers

Data management is one of todays interesting problem to solve. Data sets grow rapidly and techniques which process them should grow as well. Mining proper knowledge from vast data source became challenging task. To achieve reliable, accurate and integrate definition data sources. for this purpose all different data structure definitions (schema) should be matched together

Comparison of Ultrasonographic Findings between Patients with Tethered Cord Syndrome and Healthy Children

Background: Tethered cord syndrome (TCS) is a type of occult spinal dysraphism, which necessitates early detection as an essential component of patient management in reducing complications. This study aimed to compare the findings of spinal cord ultrasonography between TCS patients and healthy individuals. Methods: The current study is a case-control study of patients who were admitted to the Akbar and Ghaem Hospitals (Mashhad, Iran) in 2019. The study population comprised 30 children with TCS aged under two years old, and the control group included 34 healthy peers of the same age. The maximum distance of the spinal cord from the posterior canal wall was measured in millimeters using ultrasonography. Demographic and sonographic findings of each participant were recorded in checklists, which were then entered into SPSS software. P values less than 0.05 were considered statistically significant.Results: The study included 30 children with TCS and 34 healthy individuals with a mean age of 7.67±6.39 months. TCS patients had a significantly shorter maximum distance of the spinal cord from the posterior wall of the spinal canal than the control group (1.75±0.62 mm vs. 2.79±0.76, P<0.001). After performing corrective surgery, the TCS patients indicated significant improvement in this interval (1.57±0.54 mm to 2.95±0.49 mm, respectively, P=0.001).Conclusion: In comparison to children without TCS, the spinal cord was substantially closer to the posterior canal wall in TCS patients. However, these outcomes were improved significantly in patients after surgery