Assessing ASR Model Quality on Disordered Speech using BERTScore

Word Error Rate (WER) is the primary metric used to assess automatic speech recognition (ASR) model quality. It has been shown that ASR models tend to have much higher WER on speakers with speech impairments than typical English speakers. It is hard to determine if models can be be useful at such high error rates. This study investigates the use of BERTScore, an evaluation metric for text generation, to provide a more informative measure of ASR model quality and usefulness. Both BERTScore and WER were compared to prediction errors manually annotated by Speech Language Pathologists for error type and assessment. BERTScore was found to be more correlated with human assessment of error type and assessment. BERTScore was specifically more robust to orthographic changes (contraction and normalization errors) where meaning was preserved. Furthermore, BERTScore was a better fit of error assessment than WER, as measured using an ordinal logistic regression and the Akaike's Information Criterion (AIC). Overall, our findings suggest that BERTScore can complement WER when assessing ASR model performance from a practical perspective, especially for accessibility applications where models are useful even at lower accuracy than for typical speech.Comment: Accepted to Interspeech 2022 Workshop on Speech for Social Goo

Our Space: Being a Responsible Citizen of the Digital World

Our Space is a set of curricular materials designed to encourage high school students to reflect on the ethical dimensions of their participation in new media environments. Through role-playing activities and reflective exercises, students are asked to consider the ethical responsibilities of other people, and whether and how they behave ethically themselves online. These issues are raised in relation to five core themes that are highly relevant online: identity, privacy, authorship and ownership, credibility, and participation.Our Space was co-developed by The Good Play Project and Project New Media Literacies (established at MIT and now housed at University of Southern California's Annenberg School for Communications and Journalism). The Our Space collaboration grew out of a shared interest in fostering ethical thinking and conduct among young people when exercising new media skills

Consensus-Based Recommendations on the Prevention of Squamous Cell Carcinoma in Solid Organ Transplant Recipients:A Delphi Consensus Statement

IMPORTANCE There is a paucity of evidence to guide physicians regarding prevention strategies for cutaneous squamous cell carcinoma (CSCC) in solid organ transplant recipients (SOTRs). OBJECTIVE To examine the development and results of a Delphi process initiated to identify consensus-based medical management recommendations for prevention of CSCC in SOTRs. EVIDENCE REVIEW Dermatologists with more than 5 years' experience treating SOTRs were invited to participate. A novel actinic damage and skin cancer index (AD-SCI), consisting of 6 ordinal stages corresponding to an increasing burden of actinic damage and CSCC, was used to guide survey design. Three sequential web-based surveys were administered from January 1, 2019, to December 31, 2020. Pursuant to Delphi principles, respondents thoroughly reviewed all peer responses between rounds. Supplemental questions were also asked to better understand panelists' rationale for their responses. FINDINGS The Delphi panel comprised 48 dermatologists. Respondents represented 13 countries, with 27 (56%) from the US. Twenty-nine respondents (60%) were Mohs surgeons. Consensus was reached with 80% or higher concordance among respondents when presented with a statement, question, or management strategy pertaining to prevention of CSCC in SOTRs. A near-consensus category of 70% to less than 80% concordance was also defined. The AD-SCI stage-based recommendations were established if consensus or near-consensus was achieved. The panel was able to make recommendations for 5 of 6 AD-SCI stages. Key recommendations include the following: cryotherapy for scattered actinic keratosis (AK); field therapy for AK when grouped in 1 anatomical area, unless AKs are thick in which case field therapy and cryotherapy were recommended; combination lesion directed and field therapy with fluorouracil for field cancerized skin; and initiation of acitretin therapy and discussion of immunosuppression reduction or modification for patients who develop multiple skin cancers at a high rate (10 CSCCs per year) or develop high-risk CSCC (defined by a tumor with approximately >= 20% risk of nodal metastasis). No consensus recommendation was achieved for SOTRs with a first low risk CSCC. CONCLUSIONS AND RELEVANCE Physicians may consider implementation of panel recommendations for prevention of CSCC in SOTRs while awaiting high-level-of-evidence data. Additional clinical trials are needed in areas where consensus was not reached