Personalized Stopping Rules in Bayesian Adaptive Mastery Assessment

We propose a new model to assess the mastery level of a given skill efficiently. The model, called Bayesian Adaptive Mastery Assessment (BAMA), uses information on the accuracy and the response time of the answers given and infers the mastery at every step of the assessment. BAMA balances the length of the assessment and the certainty of the mastery inference by employing a Bayesian decision-theoretic framework adapted to each student. All these properties contribute to a novel approach in assessment models for intelligent learning systems. The purpose of this research is to explore the properties of BAMA and evaluate its performance concerning the number of questions administered and the accuracy of the final mastery estimates across different students. We simulate student performances and establish that the model converges with low variance and high efficiency leading to shorter assessment duration for all students. Considering the experimental results, we expect our approach to avoid the issue of over-practicing and under-practicing and facilitate the development of Learning Analytics tools to support the tutors in the evaluation of learning effects and instructional decision making.Comment: 12 page

Tracking patterns in self-regulated learning using students’ self-reports and online trace data

For decades, self-report instruments – which rely heavily on students’ perceptions and beliefs – have been the dominant way of measuring motivation and strategy use. An event-based measure based on online trace data arguably has the potential to remove analytical restrictions of self-report measures. The purpose of this study is therefore to triangulate constructs suggested in theory and measured using self-reported data with revealed online traces of learning behaviour. The results show that online trace data of learning behaviour are complementary to self-reports, as they explained a unique proportion of variance in student academic performance and reveal that self-reports explain more variance in online learning behaviour of prior weeks than variance in learning behaviour in succeeding weeks. Student motivation is, however, to a lesser extent captured with online trace data, likely because of its covert nature. In that respect, it is of importance to recognize the crucial role of self-reports in capturing student learning holistically. This manuscript is ‘frontline’ in the sense that event-based measurement methodologies using online trace data are relatively unexplored. The comparison with self-report data made in this manuscript sheds new light on the added value of innovative and traditional methods of measuring motivation and strategy use

Guided digital health intervention for depression in Lebanon: randomised trial

Background Most people with mental disorders in communities exposed to adversity in low-income and middle-income countries (LMICs) do not receive effective care. Digital mental health interventions are scalable when digital access is adequate, and can be safely delivered during the COVID-19 pandemic. Objective To examine the effects of a new WHO-guided digital mental health intervention, Step-by-Step, supported by a non-specialist helper in Lebanon, in the context of concurring economic, humanitarian and political crises, a large industrial disaster and the COVID-19 pandemic. Methods We conducted a single-blind, two-arm pragmatic randomised trial, comparing guided Step-by-Step with enhanced care as usual (ECAU) among people suffering from depression and impaired functioning. Primary outcomes were depression (Patient Health Questionnaire 9 (PHQ-9)) and impaired functioning (WHO Disability Assessment Schedule-12 (WHODAS)) at post-treatment. Findings 680 people with depression (PHQ-9>10) and impaired functioning (WHODAS>16) were randomised to Step-by-Step or ECAU. Intention-to-treat analyses showed effects on depression (standardised mean differences, SMD: 0.71; 95% CI: 0.45 to 0.97), impaired functioning (SMD: 0.43; 95% CI: 0.21 to 0.65), post-traumatic stress (SMD: 0.53; 95% CI: 0.27 to 0.79), anxiety (SMD: 0.74; 95% CI: 0.49 to 0.99), subjective well-being (SMD: 0.37; 95% CI: 0.12 to 0.62) and self-identified personal problems (SMD: 0.56; 95% CI 0.29 to 0.83). Significant effects on all outcomes were retained at 3-month follow-up. Conclusions Guided digital mental health interventions can be effective in the treatment of depression in communities exposed to adversities in LMICs, although some uncertainty remains because of high attrition

Guided digital health intervention for depression in Lebanon: randomised trial

Background Most people with mental disorders in communities exposed to adversity in low-income and middle-income countries (LMICs) do not receive effective care. Digital mental health interventions are scalable when digital access is adequate, and can be safely delivered during the COVID-19 pandemic. Objective To examine the effects of a new WHO-guided digital mental health intervention, Step-by-Step, supported by a non-specialist helper in Lebanon, in the context of concurring economic, humanitarian and political crises, a large industrial disaster and the COVID-19 pandemic. Methods We conducted a single-blind, two-arm pragmatic randomised trial, comparing guided Step-by-Step with enhanced care as usual (ECAU) among people suffering from depression and impaired functioning. Primary outcomes were depression (Patient Health Questionnaire 9 (PHQ-9)) and impaired functioning (WHO Disability Assessment Schedule-12 (WHODAS)) at post-treatment. Findings 680 people with depression (PHQ-9>10) and impaired functioning (WHODAS>16) were randomised to Step-by-Step or ECAU. Intention-to-treat analyses showed effects on depression (standardised mean differences, SMD: 0.71; 95% CI: 0.45 to 0.97), impaired functioning (SMD: 0.43; 95% CI: 0.21 to 0.65), post-traumatic stress (SMD: 0.53; 95% CI: 0.27 to 0.79), anxiety (SMD: 0.74; 95% CI: 0.49 to 0.99), subjective well-being (SMD: 0.37; 95% CI: 0.12 to 0.62) and self-identified personal problems (SMD: 0.56; 95% CI 0.29 to 0.83). Significant effects on all outcomes were retained at 3-month follow-up. Conclusions Guided digital mental health interventions can be effective in the treatment of depression in communities exposed to adversities in LMICs, although some uncertainty remains because of high attrition. Clinical implications Guided digital mental health interventions should be considered for implementation in LMICs. Trial registration number ClinicalTrials.gov NCT03720769

Step-by-step: Feasibility randomised controlled trial of a mobile-based intervention for depression among populations affected by adversity in Lebanon

Background: E-mental health interventions may help to bridge the mental health treatment gap. Evidence on their effectiveness is compelling in high-income countries. Not enough evidence has been generated on their use with communities affected by adversity in low- and middle-income countries. The World Health Organization (WHO), the National Mental Health Programme (NMMP) at Ministry of Public Health (MoPH) in Lebanon and other partners have adapted a WHO intervention called Step-by-Step for use with Lebanese and displaced people living in Lebanon. Step-by-Step is a minimally guided, internet-based intervention for adults with depression. In this study, a feasibility randomised controlled trial (RCT) and a qualitative process evaluation were conducted to explore the feasibility and the acceptability of the research methods, and the intervention, in preparation for two fully powered trials to assess the effectiveness and cost-effectiveness of Step-by-Step in Lebanon. Method: Participants were recruited through social media. Inclusion criteria were: being able to understand and speak Arabic or English; access to an internet connected device; aged over 18; living in Lebanon; scores above cut-off on the Patient Health Questionnaire and the WHO Disability Assessment Schedule 2.0. Participants were randomly assigned to the intervention or enhanced care as usual. They completed post-assessments eight weeks after baseline, and follow-up assessments another three months later. Primary outcomes were depression and level of functioning, secondary outcomes were anxiety, post-traumatic stress, and well-being. Qualitative interviews were conducted to evaluate the feasibility and acceptability of the research procedures and the intervention. Results: A total of N = 138 participants, including 33 Syrians, were recruited and randomised into two equal groups. The dropout rate was higher in the control group (73% post- and 82% follow-up assessment) than in the intervention group (63% post- and 72% follow-up assessment). The intervention was perceived as relevant, acceptable and beneficial to those who completed it. Suggestions were made to further adapt the content and to make the intervention more engaging. Statistical analyses were conducted despite the small sample size. Complete cases analysis showed a statistically significant symptom reduction in depression, anxiety, disability, and post-traumatic stress, and statistically significant improvement in well-being and functioning. Intention-to-treat analysis revealed non-significant effects. Conclusion: The research design, methods and procedures are feasible and acceptable in the context of Lebanon and can be applied in the RCTs. Preliminary findings suggest that Step-by-Step may be effective in reducing symptoms of depression and anxiety and improving functioning and well-being

Exploring the meaning of unresolved loss and trauma in more than 1,000 Adult Attachment Interviews

Unresolved states of mind regarding experiences of loss/abuse (U/d) are identified through lapses in the monitoring of reasoning, discourse, and behavior surrounding loss/abuse in response to the Adult Attachment Interview. Although the coding system for U/d has been widely used for decades, the individual indicators of unresolved loss/abuse have not been validated independently of the development sample. This study examined the psychometric validity of U/d, using individual participant data from 1,009 parent-child dyads across 13 studies. A latent class analysis showed that subsets of commonly occurring U/d indicators could differentiate interviewees with or without unresolved loss/abuse. Predictive models suggested a psychometric model of U/d consisting of a combination of these common indicators, with disbelief and psychologically confused statements regarding loss being especially important indicators of U/d. This model weakly predicted infant disorganized attachment. Multilevel regression analysis showed no significant association between ratings of unresolved other trauma and infant disorganized attachment, over and above ratings of unresolved loss/abuse. Altogether, these findings suggest that the coding system of U/d may have been overfitted to the initial development sample. Directions for further articulation and optimization of U/d are provided

Relative Private School Effectiveness in the Netherlands: A Reexamination of PISA 2006 and 2009 data

AbstractAn ongoing question is whether private (religious) schools provide better education than public schools. This study re- addresses this issue, using PISA 2006 and 2009 data for the Netherlands and three different methodologies. Overall, there is no consistent pattern. Results based on ordinary least squares and propensity score matching suggest private school attendance is positively associated with mathematics achievement, but only for PISA 2006. Instead, the results generated by an instrumental variable approach are very unstable. A thorough understanding of selection processes in Dutch education, and better data, seem necessary for future empirical work on this matter