Structural Features for Predicting the Linguistic Quality of Text: Applications to Machine Translation, Automatic Summarization and Human-Authored Text

Sentence structure is considered to be an important component of the overall linguistic quality of text. Yet few empirical studies have sought to characterize how and to what extent structural features determine fluency and linguistic quality. We report the results of experiments on the predictive power of syntactic phrasing statistics and other structural features for these aspects of text. Manual assessments of sentence fluency for machine translation evaluation and text quality for summarization evaluation are used as gold-standard. We find that many structural features related to phrase length are weakly but significantly correlated with fluency and classifiers based on the entire suite of structural features can achieve high accuracy in pairwise comparison of sentence fluency and in distinguishing machine translations from human translations. We also test the hypothesis that the learned models capture general fluency properties applicable to human-authored text. The results from our experiments do not support the hypothesis. At the same time structural features and models based on them prove to be robust for automatic evaluation of the linguistic quality of multi-document summaries

Predicting the Fluency of Text with Shallow Structural Features: Case Studies of Machine Tanslation and Human-Written Text

Sentence fluency is an important component of overall text readability but few studies in natural language processing have sought to understand the factors that define it. We report the results of an initial study into the predictive power of surface syntactic statistics for the task; we use fluency assessments done for the purpose of evaluating machine translation. We find that these features are weakly but significantly correlated with fluency. Machine and human translations can be distinguished with accuracy over 80%. The performance of pairwise comparison of fluency is also very high—over 90% for a multi-layer perceptron classifier. We also test the hypothesis that the learned models capture general fluency properties applicable to human-written text. The results do not support this hypothesis: prediction accuracy on the new data is only 57%. This finding suggests that developing a dedicated, task-independent corpus of fluency judgments will be beneficial for further investigations of the problem

경추 측면 X선 영상을 기반으로 한 갑상선 수술 환자에서 어려운 후두경 예측 딥러닝 모델

학위논문(석사) -- 서울대학교대학원 : 의과대학 의학과, 2022. 8. 정철우.예상하지 못한 어려운 후두경은 심각한 기도관련 합병증과 연관되어 있다. 본 연구는 후향적으로 수집된 갑상선 수술을 받은 총 14,135명 환자의 경추 측면 X선을 통해 어려운 후두경 (Cormack-Lehane 등급 3-4)를 예측하는 딥러닝 모델을 개발 및 검증하였다. 개발 모델의 성능은 기존의 6개의 딥러닝 모델과 비교하였다. 개발 모델에서 어려운 후두경 예측의 민감도는 95.6%, 특이도 91.2%를 나타냈다. Area Under ROC curve의 경우 개발 모델에서 0.972(0.955~0.988), 기존 모델의 경우 각각 VGG-Net: 0.842, ResNet: 0.841, Xception: 0.863, ResNext: 0.825, DenseNet: 0.889, SENet: 0.875를 나타냈다. 어려운 후두경과 관련된 해부학적 특징을 설명하기 위해 클래스 활성화 맵(Class Activation Map)을 사용하였다. 클래스 활성화 맵에서 설골, 인두 및 경추 주변이 강조되었다. 본 연구를 통해 개발된 딥러닝 모델은 경추 측면 X선 영상을 이용한 어려운 후두경 예측에 높은 성능을 보였다.An unanticipated difficult laryngoscopy is associated with serious airway-related complications. We here developed and validated a deep learning-based model that predicts a difficult laryngoscopy (Cormack–Lehane grade 3–4) from a cervical spine lateral X-ray using data from 14,135 patients undergoing thyroid surgery. The performance of our model was compared with six representative deep learning architectures. A class activation map was created to elucidate the anatomical features associated with difficult laryngoscopy. Our model showed 95.6% sensitivity and 91.2% specificity for predicting difficult laryngoscopy. The area under the receiver operating characteristic curve of our model was 0.972 (0.955‒0.988), which was higher than that of other models (VGG-Net: 0.842, ResNet: 0.841, Xception: 0.863, ResNext: 0.825, DenseNet: 0.889, and SENet: 0.875, all P < 0.001). The class activation map demonstrated clear differences around the hyoid bone, pharynx, and cervical spine. The model showed excellent performance for predicting difficult laryngoscopy using a cervical spine lateral X-ray image.1. Introduction 1 2. Materials and Methods 2 2.1 Inclusion and Exclusion Criteria 2 2.2 Anesthesia Management 2 2.3 Data Collection and Preprocessing 2 2.4 Model Building 3 2.5 Model Validation 4 2.6 Sensitivity Analysis 4 2.7 Statistical Analysis 4 3. Results 6 3.1 Dataset Construction 6 3.2 Performance of the Models 6 3.3 Sensitivity Analysis 6 4. Discussion 8 5. Conclusions 11 References 23 Abstract 26 Tables 12 [Table 1] 12 [Table 2] 13 [Table 3] 14 Figures 15 [Figure 1] 15 [Figure 2] 16 [Figure 3] 17 [Figure 4] 18 [Figure 5] 19 Supplementary Materials 20 [Supplementary Materials] 20석

Improving the in silico assessment of pathogenicity for compensated variants

Understanding the functional sequelae of amino-acid replacements is of fundamental importance in medical genetics. Perhaps, the most intuitive way to assess the potential pathogenicity of a given human missense variant is by measuring the degree of evolutionary conservation of the substituted amino-acid residue, a feature that generally serves as a good proxy metric for the functional/structural importance of that residue. However, the presence of putatively compensated variants as the wild-type alleles in orthologous proteins of other mammalian species not only challenges this classical view of amino-acid essentiality but also precludes the accurate evaluation of the functional impact of this type of missense variant using currently available bioinformatic prediction tools. Compensated variants constitute at least 4% of all known missense variants causing human-inherited disease and hence represent an important potential source of error in that they are likely to be disproportionately misclassified as benign variants. The consequent under-reporting of compensated variants is exacerbated in the context of next-generation sequencing where their inappropriate exclusion constitutes an unfortunate natural consequence of the filtering and prioritization of the very large number of variants generated. Here we demonstrate the reduced performance of currently available pathogenicity prediction tools when applied to compensated variants and propose an alternative machine-learning approach to assess likely pathogenicity for this particular type of variant

Concurrent Validity of the Child Behavior Checklist DSM-Oriented Scales: Correspondence with DSM Diagnoses and Comparison to Syndrome Scales

This study used receiver operating characteristic (ROC) methodology and discriminative analyses to examine the correspondence of the Child Behavior Checklist (CBCL) rationally-derived DSM-oriented scales and empirically-derived syndrome scales with clinical diagnoses in a clinic-referred sample of children and adolescents (N = 476). Although results demonstrated that the CBCL Anxiety, Affective, Attention Deficit/Hyperactivity, Oppositional and Conduct Problems DSM-oriented scales corresponded significantly with related clinical diagnoses derived from parent-based structured interviews, these DSM-oriented scales did not evidence significantly greater correspondence with clinical diagnoses than the syndrome scales in all cases but one. The DSM-oriented Anxiety Problems scale was the only scale that evidenced significantly greater correspondence with diagnoses above its syndrome scale counterpart —the Anxious/Depressed scale. The recently developed and rationally-derived DSM-oriented scales thus generally do not add incremental clinical utility above that already afforded by the syndrome scales with respect to corresponding with diagnoses. Implications of these findings are discussed