Application of Crowdsourcing and Machine Learning to Predict Sentiments in Textual Student Feedback in Large Computer Science Classes

With the increasing enrollment numbers into popular computer science courses, there is a need to bridge the similarly increasing feedback gap between individual students and course instructors. One way to address this challenge is for instructors to collect feedback from students in form of textual reviews or unit-of-study reflections – however, manually reading these reviews is time-consuming, and self-reported Likert scale responses are noisy. Rule-based approaches to sentiment analysis such as VADER (Valence Aware Dictionary and sEntiment Reasoner) have been used to capture the sentiments conveyed in textual feedback, they however fail to capture contextual differences as many words have different sentiments in different contexts. In this work, I investigated the use of supervised machine learning approaches and compared their performance in predicting the sentiment in student feedback collected in large computer science classes with the lexicon-based approach VADER. I found that machine learning models trained solely on student self-reported sentiment ratings were only comparable with a balanced accuracy of 73.8% versus 73% (VADER). However, a hybrid approach using the VADER score as a feature and training using the student self-ratings performed better than VADER alone. Using better quality labels collected through a crowdsourcing experiment led to the best machine learning model performance

Distributional Latent Variable Models with an Application in Active Cognitive Testing

Cognitive modeling commonly relies on asking participants to complete a battery of varied tests in order to estimate attention, working memory, and other latent variables. In many cases, these tests result in highly variable observation models. A near-ubiquitous approach is to repeat many observations for each test, resulting in a distribution over the outcomes from each test given to each subject. In this paper, we explore the usage of latent variable modeling to enable learning across many correlated variables simultaneously. We extend latent variable models (LVMs) to the setting where observed data for each subject are a series of observations from many different distributions, rather than simple vectors to be reconstructed. By embedding test battery results for individuals in a latent space that is trained jointly across a population, we are able to leverage correlations both between tests for a single participant and between multiple participants. We then propose an active learning framework that leverages this model to conduct more efficient cognitive test batteries. We validate our approach by demonstrating with real-time data acquisition that it performs comparably to conventional methods in making item-level predictions with fewer test items.Comment: 9 pages, 6 figure

Histo-Blood Group Antigen Null Phenotypes Associated With a Decreased Risk of Clinical Rotavirus Vaccine Failure Among Children <2 Years of Age Participating in the Vaccine Impact on Diarrhea in Africa (VIDA) Study in Kenya, Mali, and the Gambia

Background: Previously studied risk factors for rotavirus vaccine failure have not fully explained reduced rotavirus vaccine effectiveness in low-income settings. We assessed the relationship between histo-blood group antigen (HBGA) phenotypes and clinical rotavirus vaccine failure among children 4. Conclusions: Our study demonstrated a significant relationship between null HBGA phenotypes and decreased rotavirus vaccine failure in a population with P[8] as the most common infecting genotype. Further studies are needed in populations with a large burden of P[6] rotavirus diarrhea to understand the role of host genetics in reduced rotavirus vaccine effectiveness

Characteristics of Salmonella recovered from stools of children enrolled in the Global Enteric Multicenter Study

Background: The Global Enteric Multicenter Study (GEMS) determined the etiologic agents of moderate-To-severe diarrhea (MSD) in children under 5 years old in Africa and Asia. Here, we describe the prevalence and antimicrobial susceptibility of nontyphoidal Salmonella (NTS) serovars in GEMS and examine the phylogenetics of Salmonella Typhimurium ST313 isolates. Methods: Salmonella isolated from children with MSD or diarrhea-free controls were identified by classical clinical microbiology and serotyped using antisera and/or whole-genome sequence data. We evaluated antimicrobial susceptibility using the Kirby-Bauer disk-diffusion method. Salmonella Typhimurium sequence types were determined using multi-locus sequence typing, and whole-genome sequencing was performed to assess the phylogeny of ST313. Results: Of 370 Salmonella-positive individuals, 190 (51.4%) were MSD cases and 180 (48.6%) were diarrhea-free controls. The most frequent Salmonella serovars identified were Salmonella Typhimurium, serogroup O:8 (C2-C3), serogroup O:6,7 (C1), Salmonella Paratyphi B Java, and serogroup O:4 (B). The prevalence of NTS was low but similar across sites, regardless of age, and was similar among both cases and controls except in Kenya, where Salmonella Typhimurium was more commonly associated with cases than controls. Phylogenetic analysis showed that these Salmonella Typhimurium isolates, all ST313, were highly genetically related to isolates from controls. Generally, Salmonella isolates from Asia were resistant to ciprofloxacin and ceftriaxone, but African isolates were susceptible to these antibiotics. Conclusions: Our data confirm that NTS is prevalent, albeit at low levels, in Africa and South Asia. Our findings provide further evidence that multidrug-resistant Salmonella Typhimurium ST313 can be carried asymptomatically by humans in sub-Saharan Africa

Practical Sentiment Analysis for Education: The Power of Student Crowdsourcing

Sentiment analysis provides a promising tool to automatically assess the emotions voiced in written student feedback such as periodically collected unit-of-study reflections. The commonly used dictionary-based approaches are limited to major languages and fail to capture contextual differences. Pretrained large language models have been shown to be biased and online versions raise privacy concerns. Hence, we resort to traditional supervised machine learning (ML) approaches which are designed to overcome these issues by learning from domain-specific labeled data. However, these labels are hard to come by -- in our case manually annotating student feedback is prone to bias and time-consuming, especially in high-enrollment courses. In this work, we investigate the use of student crowdsourced labels for supervised sentiment analysis for education. Specifically, we compare crowdsourced and student self-reported labels with human expert annotations and use them in various ML approaches to evaluate the performance on predicting emotions of written student feedback collected from large computer science classes. We find that the random forest model trained with student-crowdsourced labels tremendously improves the identification of reflections with negative sentiment. In addition to our quantitative study, we describe our crowdsourcing experiment which was intentionally designed to be an educational activity in an introduction to data science course

Mobile Phone Technology for Preventing HIV and Related Youth Health Problems, Sexual Health, Mental Health, and Substance Use Problems in Southwest Uganda (Youth Health SMS): Protocol for a Pilot Randomized Controlled Trial

BackgroundEast and Southern Africa have the highest HIV incidence and prevalence in the world, with adolescents and young adults being at the greatest risk. Despite effective combination prevention tools, including the recently available pre-exposure prophylaxis (PrEP), HIV incidence among adolescents and young adults in Uganda remains high, and PrEP use remains low. Mental health and substance use (behavioral health) play a role in sexual behavior and decision-making, contributing to an increase in the risk for acquiring HIV. Interventions that target multiple HIV risk factors, including sexual and mental health and problematic substance use, are crucial to ending the HIV epidemic. Yet few interventions addressing HIV related health disparities and comorbidities among adolescents and young adults in East and Southern Africa currently exist. ObjectiveThis study aims to evaluate the acceptability and feasibility of Kirabo, an SMS text message intervention informed by the information, motivation, and behavior model and to be disseminated through secondary schools. The study will gather preliminary estimates of Kirabo’s effectiveness in increasing HIV testing and linking users to mental health counselors. MethodsWe identified Mobile 4 Reproductive Health for adaptation using the assessment, decision, administration, production, topical experts, integration, training, testing (ADAPT-ITT) framework. Mobile 4 Reproductive Health is an evidence-based automated 2-way SMS text messaging and interactive voice response platform that offers sexual and reproductive health information and links users to HIV clinics in East Africa. Through ADAPT-ITT we refined our approach and created Kirabo, an SMS text message–based intervention for linking adolescents and young adults to health services, including HIV testing and mental health counseling. We will conduct a 2-arm randomized controlled trial in Masaka, Uganda. Adolescents (N=200) will be recruited from local schools. Baseline sociodemographic characteristics, HIV test history, and behavioral health symptoms will be assessed. We will evaluate acceptability and feasibility using surveys, interviews, and mobile phone data. The preliminary efficacy of Kirabo in increasing HIV testing and linking users to mental health counselors will be evaluated immediately after the intervention and at the 3-month follow-up. We will also assess the intervention’s impact on self-efficacy in testing for HIV, adopting PrEP, and contacting a mental health counselor. ResultsIntervention adaptation began in 2019. A pretest was conducted in 2021. The randomized controlled trial, including usability and feasibility assessments and effectiveness measurements, commenced in August 2023. ConclusionsKirabo is a tool that assists in the efforts to end the HIV epidemic by targeting the health disparities and comorbidities among adolescents in Uganda. The intervention includes local HIV clinic information, PrEP information, and behavioral health screening, with referrals as needed. Increasing access to prevention strategies and mitigating factors that make adolescents and young adults susceptible to HIV acquisition can contribute to global efforts to end the HIV epidemic. Trial RegistrationClinicalTrials.gov NCT05130151; https://clinicaltrials.gov/study/NCT05130151 International Registered Report Identifier (IRRID)DERR1-10.2196/4935