A Markov Approach for Increasing Precision in the Assessment of Data-Intensive Behavioral Interventions

Health interventions using real-time sensing technology are characterized by intensive longitudinal data, which has the potential to enable nuanced evaluations of individuals’ responses to treatment. Existing analytic tools were not developed to capitalize on this opportunity as they typically focus on first-order findings such as changes in the level and/or slope of outcome variables over different intervention phases. This paper introduces an exploratory, Markov-based empirical transition method that offers a more comprehensive assessment of behavioral responses when intensive longitudinal data are available. The procedure projects a univariate time-series into discrete states and empirically determines the probability of transitioning from one state to another. State transition probabilities are summarized separately in phase-specific transition matrices. Comparing transition matrices illuminates intricate, quantifiable differences in behavior between intervention phases. Statistical significance is estimated via bootstrapping techniques. This paper introduces the methodology via three case studies from a secondhand smoke reduction trial utilizing real-time air particle sensors. Analysis enabled the identification of complex phenomena such as avoidance and escape behavior in response to punitive contingencies for tobacco use. Additionally, the largest changes in behavior dynamics were associated with the introduction of behavioral feedback. The Markov approach‘s ability to elucidate subtle behavioral details has not typically been feasible with standard methodologies, mainly due to historical limitations associated with infrequent repeated measures. These results suggest that the evaluation of intervention effects in data-intensive single-case designs can be enhanced, providing rich information that can ultimately be used to develop interventions uniquely tailored to specific individuals

Economic Evaluation of a Personalized Nutrition Plan Based on Omic Sciences Versus a General Nutrition Plan in Adults with Overweight and Obesity:A Modeling Study Based on Trial Data in Denmark

Background: Since there is no diet that is perfect for everyone, personalized nutrition approaches are gaining popularity to achieve goals such as the prevention of obesity-related diseases. However, appropriate choices about funding and encouraging personalized nutrition approaches should be based on sufficient evidence of their effectiveness and cost-effectiveness. In this study, we assessed whether a newly developed personalized plan (PP) could be cost-effective relative to a non-personalized plan in Denmark. Methods: Results of a 10-week randomized controlled trial were combined with a validated obesity economic model to estimate lifetime cost-effectiveness. In the trial, the intervention group (PP) received personalized home-delivered meals based on metabolic biomarkers and personalized behavioral change messages. In the control group these meals and messages were not personalized. Effects were measured in body mass index (BMI) and quality of life (EQ-5D-5L). Costs [euros (€), 2020] were considered from a societal perspective. Lifetime cost-effectiveness was assessed using a multi-state Markov model. Univariate, probabilistic sensitivity, and scenario analyses were performed. Results: In the trial, no significant differences were found in the effectiveness of PP compared with control, but wide confidence intervals (CIs) were seen [e.g., BMI (−0.07, 95% CI −0.51, 0.38)]. Lifetime estimates showed that PP increased costs (€520,102 versus €518,366, difference: €1736) and quality-adjusted life years (QALYs) (15.117 versus 15.106, difference: 0.011); the incremental cost-utility ratio (ICUR) was therefore high (€158,798 to gain one QALY). However, a 20% decrease in intervention costs would reduce the ICUR (€23,668 per QALY gained) below an unofficial gross domestic product (GDP)-based willingness-to-pay threshold (€47,817 per QALY gained). Conclusion: On the basis of the willingness-to-pay threshold and the non-significant differences in short-term effectiveness, PP may not be cost-effective. However, scaling up the intervention would reduce the intervention costs. Future studies should be larger and/or longer to reduce uncertainty about short-term effectiveness. Trial Registration Number: ClinicalTrials.gov registry (NCT04590989).</p

Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Sickness absence among patients with chronic pain in Swedish specialist healthcare

Background: Chronic pain beyond three months is a global public health problem. Every third adult suffers from a chronic pain condition, resulting in a socioeconomic burden that corresponds to 3-10% of gross domestic product in western economies. This burden can be largely attributed to absenteeism-related productivity loss where a few highly impaired individuals are the most resource-intensive. Simultaneously, a detailed overview of sickness absence (SA) associated with chronic pain is complicated by incongruent classification due to conflicting perspectives on the condition as either a symptom or a disease in its own right. Aim: Based on a well-defined chronic pain population in the Swedish specialist healthcare, this thesis primarily aims to provide a SA overview, to explore the possibility of SA prevention, and to evaluate interdisciplinary treatment (IDT) as a SA intervention. A secondary objective was to assess the psychometric properties of three questionnaires that measure the core domains of the chronic pain experience. Methods: The aims were addressed in three register-based studies using microdata from five Swedish national registers. Study I used sequence analysis to describe SA in 44,241 patients over a 7-year period and subsequently developed a machine learning-based model to predict chronic pain-related SA in the final two years. Study II emulated a target trial to compare the total SA duration over a 5-year period for 25,613 patients that were either included in an IDT program or in other/no interventions. Study III analyzed the properties of the Short Form-36 Health Survey (SF-36), the EuroQol 5-Dimensions instrument (EQ-5D), and the Hospital Anxiety and Depression Scale (HADS) within the item response theory-framework. Results: SA increased from 17% to 48% over the five years before specialist healthcare entry to then decrease to 38% over the final two years. With information on eight predictors, it was possible to discriminate between patients that would have low or high SA in the coming two years with 80% accuracy. SA trends were similar for patients in IDT programs and other/no interventions, albeit the IDT patients had 67 (95% CI: 48, 87) more SA days over the complete 5-year period. Finally, the psychometric evaluation revealed that SF-36 adequately captured physical and mental health, while HADS was suitable as a measure of overall emotional distress, and EQ-5D had insufficient precision for any meaningful application. Conclusion: Our findings are most useful to guide policy and research. SA in the studied patients remained high over the entire observation period. Decision support tools could prove valuable in identifying patients at risk of high SA earlier in the healthcare chain in order to direct preventative measures. We found no support for IDT decreasing SA more than other/no interventions, but it is possible that this was a consequence of our methodology. Further studies of the IDT effects are needed, but uncontrolled designs that attribute SA change over time to IDT are inappropriate for this purpose, as the SA peak observed around specialist healthcare entry is likely to be driven by the referral procedure. Finally, SF-36 and HADS are psychometrically sound measures of the chronic pain experience core domains

On the Parametrization of Epidemiologic Models: Lessons from Modelling COVID-19 Epidemic

Numerous prediction models of SARS-CoV-2 pandemic were proposed in the past. Unknown parameters of these models are often estimated based on observational data. However, lag in case-reporting, changing testing policy or incompleteness of data lead to biased estimates. Moreover, parametrization is time-dependent due to changing age-structures, emerging virus variants, non-pharmaceutical interventions, and vaccination programs. To cover these aspects, we propose a principled approach to parametrize a SIR-type epidemiologic model by embedding it as a hidden layer into an input-output non-linear dynamical system (IO-NLDS). Observable data are coupled to hidden states of the model by appropriate data models considering possible biases of the data. This includes data issues such as known delays or biases in reporting. We estimate model parameters including their time-dependence by a Bayesian knowledge synthesis process considering parameter ranges derived from external studies as prior information. We applied this approach on a specific SIR-type model and data of Germany and Saxony demonstrating good prediction performances. Our approach can estimate and compare the relative effectiveness of non-pharmaceutical interventions and provide scenarios of the future course of the epidemic under specified conditions. It can be translated to other data sets, i.e., other countries and other SIR-type models

Use of automated coding methods to assess motivational behaviour in education

Teachers’ motivational behaviour is related to important student outcomes. Assessing teachers’ motivational behaviour has been helpful to improve teaching quality and enhance student outcomes. However, researchers in educational psychology have relied on self-report or observer ratings. These methods face limitations on accurately and reliably assessing teachers’ motivational behaviour; thus restricting the pace and scale of conducting research. One potential method to overcome these restrictions is automated coding methods. These methods are capable of analysing behaviour at a large scale with less time and at low costs. In this thesis, I conducted three studies to examine the applications of an automated coding method to assess teacher motivational behaviours. First, I systematically reviewed the applications of automated coding methods used to analyse helping professionals’ interpersonal interactions using their verbal behaviour. The findings showed that automated coding methods were used in psychotherapy to predict the codes of a well-developed behavioural coding measure, in medical settings to predict conversation patterns or topics, and in education to predict simple concepts, such as the number of open/closed questions or class activity type (e.g., group work or teacher lecturing). In certain circumstances, these models achieved near human level performance. However, few studies adhered to best-practice machine learning guidelines. Second, I developed a dictionary of teachers’ motivational phrases and used it to automatically assess teachers’ motivating and de-motivating behaviours. Results showed that the dictionary ratings of teacher need support achieved a strong correlation with observer ratings of need support (rfull dictionary = .73). Third, I developed a classification of teachers’ motivational behaviour that would enable more advanced automated coding of teacher behaviours at each utterance level. In this study, I created a classification that includes 57 teacher motivating and de-motivating behaviours that are consistent with self-determination theory. Automatically assessing teachers’ motivational behaviour with automatic coding methods can provide accurate, fast pace, and large scale analysis of teacher motivational behaviour. This could allow for immediate feedback and also development of theoretical frameworks. The findings in this thesis can lead to the improvement of student motivation and other consequent student outcomes