The Exponentially Weighted Moving Average Procedure for Detecting Changes in Intensive Longitudinal Data in Psychological Research in Real-Time:A Tutorial Showcasing Potential Applications

Affect, behavior, and severity of psychopathological symptoms do not remain static throughout the life of an individual, but rather they change over time. Since the rise of the smartphone, longitudinal data can be obtained at higher frequencies than ever before, providing new opportunities for investigating these person-specific changes in real-time. Since 2019, researchers have started using the exponentially weighted moving average (EWMA) procedure, as a statistically sound method to reach this goal. Real-time, person-specific change detection could allow (a) researchers to adapt assessment intensity and strategy when a change occurs to obtain the most useful data at the most useful time and (b) clinicians to provide care to patients during periods in which this is most needed. The current paper provides a tutorial on how to use the EWMA procedure in psychology, as well as demonstrates its added value in a range of potential applications.</p

The Data Representativeness Criterion: Predicting the Performance of Supervised Classification Based on Data Set Similarity

In a broad range of fields it may be desirable to reuse a supervised classification algorithm and apply it to a new data set. However, generalization of such an algorithm and thus achieving a similar classification performance is only possible when the training data used to build the algorithm is similar to new unseen data one wishes to apply it to. It is often unknown in advance how an algorithm will perform on new unseen data, being a crucial reason for not deploying an algorithm at all. Therefore, tools are needed to measure the similarity of data sets. In this paper, we propose the Data Representativeness Criterion (DRC) to determine how representative a training data set is of a new unseen data set. We present a proof of principle, to see whether the DRC can quantify the similarity of data sets and whether the DRC relates to the performance of a supervised classification algorithm. We compared a number of magnetic resonance imaging (MRI) data sets, ranging from subtle to severe difference is acquisition parameters. Results indicate that, based on the similarity of data sets, the DRC is able to give an indication as to when the performance of a supervised classifier decreases. The strictness of the DRC can be set by the user, depending on what one considers to be an acceptable underperformance.Comment: 12 pages, 6 figure

The Exponentially Weighted Moving Average Procedure for Detecting Changes in Intensive Longitudinal Data in Psychological Research in Real-Time:A Tutorial Showcasing Potential Applications

Affect, behavior, and severity of psychopathological symptoms do not remain static throughout the life of an individual, but rather they change over time. Since the rise of the smartphone, longitudinal data can be obtained at higher frequencies than ever before, providing new opportunities for investigating these person-specific changes in real-time. Since 2019, researchers have started using the exponentially weighted moving average (EWMA) procedure, as a statistically sound method to reach this goal. Real-time, person-specific change detection could allow (a) researchers to adapt assessment intensity and strategy when a change occurs to obtain the most useful data at the most useful time and (b) clinicians to provide care to patients during periods in which this is most needed. The current paper provides a tutorial on how to use the EWMA procedure in psychology, as well as demonstrates its added value in a range of potential applications.</p

Detecting Mean Changes in Experience Sampling Data in Real Time:A Comparison of Univariate and Multivariate Statistical Process Control Methods

Detecting early warning signals of developing mood disorders in continuously collected affective experience sampling (ESM) data would pave the way for timely intervention and prevention of a mood disorder from occurring or to mitigate its severity. However, there is an urgent need for online statistical methods tailored to the specifics of ESM data. Statistical process control (SPC) procedures, originally developed for monitoring industrial processes, seem promising tools. However, affective ESM data violate major assumptions of the SPC procedures: The observations are not independent across time, often skewed distributed, and characterized by missingness. Therefore, evaluating SPC performance on simulated data with typical ESM features is a crucial step. In this article, we didactically introduce six univariate and multivariate SPC procedures: Shewhart, Hotelling’s T2, EWMA, MEWMA, CUSUM and MCUSUM. Their behavior is illustrated on publicly available affective ESM data of a patient that relapsed into depression. To deal with the missingness, autocorrelation, and skewness in these data, we compute and monitor the day averages rather than the individual measurement occasions. Moreover, we apply all procedures on simulated data with typical affective ESM features, and evaluate their performance at detecting small to moderate mean changes. The simulation results indicate that the (M)EWMA and (M)CUSUM procedures clearly outperform the Shewhart and Hotelling’s T2 procedures and support using day averages rather than the original data. Based on these results, we provide some recommendations for optimizing SPC performance when monitoring ESM data as well as a wide range of directions for future research.</p

Detecting Mean Changes in Experience Sampling Data in Real Time:A Comparison of Univariate and Multivariate Statistical Process Control Methods

Detecting early warning signals of developing mood disorders in continuously collected affective experience sampling (ESM) data would pave the way for timely intervention and prevention of a mood disorder from occurring or to mitigate its severity. However, there is an urgent need for online statistical methods tailored to the specifics of ESM data. Statistical process control (SPC) procedures, originally developed for monitoring industrial processes, seem promising tools. However, affective ESM data violate major assumptions of the SPC procedures: The observations are not independent across time, often skewed distributed, and characterized by missingness. Therefore, evaluating SPC performance on simulated data with typical ESM features is a crucial step. In this article, we didactically introduce six univariate and multivariate SPC procedures: Shewhart, Hotelling’s T2, EWMA, MEWMA, CUSUM and MCUSUM. Their behavior is illustrated on publicly available affective ESM data of a patient that relapsed into depression. To deal with the missingness, autocorrelation, and skewness in these data, we compute and monitor the day averages rather than the individual measurement occasions. Moreover, we apply all procedures on simulated data with typical affective ESM features, and evaluate their performance at detecting small to moderate mean changes. The simulation results indicate that the (M)EWMA and (M)CUSUM procedures clearly outperform the Shewhart and Hotelling’s T2 procedures and support using day averages rather than the original data. Based on these results, we provide some recommendations for optimizing SPC performance when monitoring ESM data as well as a wide range of directions for future research.</p

Introducing change point detection analyses in relationship research : an investigation of couples’ emotion dynamics

Many relationship theories assume some form of interdependence between relationship partners. Partners are thought to continuously influence each other and to be influenced by each other over time. These influences are not expected to be constant, but dynamic (sometimes partners influence each other a lot, and sometimes they do not influence each other). To investigate such changes in interpersonal dynamics, we showcase the value of using a change point detection approach, which can be used to monitor virtually any preferred quantification of interpersonal dynamics across time. Concretely, we introduce the KCP-RS method, which scans times series for changes in user-specified statistics, in interpersonal emotion dynamic research. We used KCP-RS to investigate changes in 96 couples' emotional experiences during two 10-minute conversations, which were meant to elicit a negative and a positive interaction context. Based on participants' continuous reports of the valence of their emotional experience, we looked for changes in three statistical measures, aiming to capture emotional similarity between partners (i.e., does their valence fluctuate together). Specifically, we investigated the occurrence, frequency, and direction of change in partners' linear correlations, instantaneous derivative matching (IMD), and signal matching (SM). While correlation changes were only observed in 2% of the couples, IDM changes were detected for about one third of the couples (34%), and SM changes were detected in about half of them (49%). Most couples demonstrated one change point, and the direction of the change differed depending on the specific emotional similarity measure. In a first validation of this method, we demonstrated how such change points can pinpoint to subtle but meaningful dynamic processes in couples. We end by discussing the added value of change point detection analyses for relationship research and interpersonal research in general

Real-time Detection of Mean and Variance Changes in Experience Sampling Data: A Comparison of Existing and Novel Statistical Process Control Approaches

Retrospective analyses of experience sampling (ESM) data have shown that changes in mean and variance levels may serve as early warning signs of an imminent depression. Detecting such early warning signs prospectively would pave the way for timely intervention and prevention. The exponentially weighted moving average (EWMA) procedure seems a promising method to scan ESM data for the presence of mean changes in real-time. Based on simulation and empirical studies, computing and monitoring day averages using EWMA works particularly well. We therefore expand this idea to the detection of variance changes and propose to use EWMA to prospectively scan for mean changes in day variability statistics (i.e., s2, s, ln(s)). When both mean and variance changes are of interest, the multivariate extension of the EWMA procedure (MEWMA) can be applied to both the day averages and a day statistic of variability. We evaluate these novel approaches to detecting variance changes by comparing them to EWMA-type procedures that have been specifically developed to detect a combination of mean and variance changes in the raw data: EWMA-S2 and EWMA-ln(S2). We ran a simulation study to examine the performance of the two types of approaches in detecting mean, variance or both types of changes. The results indicate that monitoring day statistics using (M)EWMA works well and outperforms the EWMA-S2 and EWMA-ln(S2) procedures. Based on the results, we provide recommendations on which statistic of variability to monitor based on the type of change (i.e., variance increase or decrease) one expects