Assessing and accounting for measurement in intensive longitudinal studies:Current Practices, Considerations, and Avenues for Improvement

Purpose: Intensive longitudinal studies, in which participants complete questionnaires multiple times a day over an extended period, are increasingly popular in the social sciences in general and quality-of-life research in particular. The intensive longitudinal methods allow for studying the dynamics of constructs (e.g., how much patient-reported outcomes vary across time). These methods promise higher ecological validity and lower recall bias than traditional methods that question participants only once, since the high frequency means that participants complete questionnaires in their everyday lives and do not have to retrospectively report about a large time interval. However, to ensure the validity of the results obtained from analyzing the intensive longitudinal data (ILD), greater awareness and understanding of appropriate measurement practices are needed. Method: We surveyed 42 researchers experienced with ILD regarding their measurement practices and reasons for suboptimal practices. Results: Results showed that researchers typically do not use measures validated specifically for ILD. Participants assessing the psychometric properties and invariance of measures in their current studies was even less common, as was accounting for these properties when analyzing dynamics. This was mainly because participants did not have the necessary knowledge to conduct these assessments or were unaware of their importance for drawing valid inferences. Open science practices, in contrast, appear reasonably well ingrained in ILD studies. Conclusion: Measurement practices in ILD still need improvement in some key areas; we provide recommendations in order to create a solid foundation for measuring and analyzing psychological constructs.</p

Assessing and accounting for measurement in intensive longitudinal studies:Current Practices, Considerations, and Avenues for Improvement

Purpose: Intensive longitudinal studies, in which participants complete questionnaires multiple times a day over an extended period, are increasingly popular in the social sciences in general and quality-of-life research in particular. The intensive longitudinal methods allow for studying the dynamics of constructs (e.g., how much patient-reported outcomes vary across time). These methods promise higher ecological validity and lower recall bias than traditional methods that question participants only once, since the high frequency means that participants complete questionnaires in their everyday lives and do not have to retrospectively report about a large time interval. However, to ensure the validity of the results obtained from analyzing the intensive longitudinal data (ILD), greater awareness and understanding of appropriate measurement practices are needed. Method: We surveyed 42 researchers experienced with ILD regarding their measurement practices and reasons for suboptimal practices. Results: Results showed that researchers typically do not use measures validated specifically for ILD. Participants assessing the psychometric properties and invariance of measures in their current studies was even less common, as was accounting for these properties when analyzing dynamics. This was mainly because participants did not have the necessary knowledge to conduct these assessments or were unaware of their importance for drawing valid inferences. Open science practices, in contrast, appear reasonably well ingrained in ILD studies. Conclusion: Measurement practices in ILD still need improvement in some key areas; we provide recommendations in order to create a solid foundation for measuring and analyzing psychological constructs.</p

Assessing Community Progress on the Blueprint to End Homelessness

In 2002, the Indianapolis Housing Task Force published the Blueprint to End Homelessness, an ambitious 10-year strategy to end homelessness in Indianapolis by 2012. The Blueprint called for regular reports and evaluation of progress toward the Blueprint’s goals. The Coalition for Homelessness Intervention and Prevention (CHIP), charged with moving the Blueprint forward, has completed its own annual Community Progress Reports for 2009, 2010, and 2011. This report does not seek to replicate or evaluate these or any of the many previous reports CHIP has facilitated. We take what is presented in the previous reports as accurate and eminently useful. The annual Community Progress Reports, in particular, already serve as good evaluations of progress toward the Blueprint goals. Instead, this report seeks to identify issues not yet covered, areas where data have not been collected, areas where data collection could be improved, or areas where existing data have not yet been analyzed for the purpose of assessing Blueprint goals. We have gathered and analyzed new qualitative and quantitative data from CHIP, stakeholders, the homeless, and other sources to provide additional measures of progress toward achieving the various goals stipulated in the Blueprint and to establish new measures for future assessment. Besides qualitative interviews with samples of stakeholders and homeless, we collected census data on affordable housing for Marion County, the U.S., and four other comparison counties. We conducted a Strengths, Weaknesses, Opportunities, and Threats (SWOT) Analysis of CHIP’s annual Community Progress Reports. CHIP also provided nine years’ worth of client data from the Homeless Management Information Systems (HMIS). Finally, we collected progress reports from other jurisdictions implementing ten-year/community plans and looked at those. The overarching goal of the Blueprint has not been achieved. Homelessness has not been eliminated and will not be eliminated by the 2012 date established in the Blueprint. Progress has been and continues to be made in many areas, though. It is hoped this report will help the community as it moves forward with creating a new strategic plan

When, how and for whom changes in engagement happen:A transition analysis of instructional variables

The pace of our knowledge on online engagement has not been at par with our need to understand the temporal dynamics of online engagement, the transitions between engagement states, and the factors that influence a student being persistently engaged, transitioning to disengagement, or catching up and transitioning to an engaged state. Our study addresses such a gap and investigates how engagement evolves or changes over time, using a person-centered approach to identify for whom the changes happen and when. We take advantage of a novel and innovative multistate Markov model to identify what variables influence such transitions and with what magnitude, i.e., to answer the why. We use a large data set of 1428 enrollments in six courses (238 students). The findings show that online engagement changes differently —across students— and at different magnitudes —according to different instructional variables and previous engagement states. Cognitively engaging instructions helped cognitively engaged students stay engaged while negatively affecting disengaged students. Lectures —a resource that requires less mental energy— helped improve disengaged students. Such differential effects point to the different ways interventions can be applied to different groups, and how different groups may be supported. A balanced, carefully tailored approach is needed to design, intervene, or support students' engagement that takes into account the diversity of engagement states as well as the varied response magnitudes that intervention may incur across diverse students’ profiles

Investigating dynamics in attentive and inattentive responding together with their contextual correlates using a novel mixture IRT model for intensive longitudinal data

In ecological momentary assessment (EMA), respondents answer brief questionnaires about their current behaviors or experiences several times per day across multiple days. The frequent measurement enables a thorough grasp of the dynamics inherent in psychological traits, but it also increases respondent burden. To lower this burden, respondents may engage in careless and insufficient effort responding (C/IER) and leave data contaminated with responses that do not reflect what researchers want to measure. We introduce a novel approach to investigate C/IER in EMA data. Our approach combines a confirmatory mixture item response theory model separating C/IER from attentive behavior with latent Markov factor analysis. This allows for (1) gauging the occurrence of C/IER and (2) studying transitions among states of different response behaviors as well as their contextual correlates. The approach can be implemented using standard R packages. In an empirical application, we showcase the efficacy of this approach in both pinpointing C/IER instances in EMA and gaining insights into their underlying causes. In a simulation study investigating robustness against unaccounted changes in measurement models underlying attentive responses, the approach proved robust against heterogeneity in loading patterns but not against heterogeneity in the factor structure. Extensions to accommodate the latter are discussed.<br/

Upper Class Enclave Identity: A Case Study of the Golden Hill Community

Urban enclaves play an important role in sociological theory and in overall community development. This project looks at the exclusivity of a particular enclave in the Indianapolis area, the Golden Hill Community, and through observation and interviews, examines the makeup of this isolated community. Considered by many to be an exclusive upper-class neighborhood, this research looks closely at the social interaction of residents with each other, as well as the outside community, in order to determine its strength and significance as an urban enclave. This paper suggests that Golden Hill, contrary to other upper class urban enclaves, exhibits a type of shared history among its residents not found in other similar neighborhoods. This finding, coupled with other geographical and historical factors, have helped contribute to the overall stability of the neighborhood throughout the past one hundred years

How to explore within‑person and between‑person measurement model differences in intensive longitudinal data with the R package lmfa

Intensive longitudinal data (ILD) have become popular for studying within-person dynamics in psychological constructs (or between-person differences therein). Before investigating the dynamics, it is crucial to examine whether the measurement model (MM) is the same across subjects and time and, thus, whether the measured constructs have the same meaning. If the MM differs (e.g., because of changes in item interpretation or response styles), observations cannot be validly compared. Exploring differences in the MM for ILD can be done with latent Markov factor analysis (LMFA), which classifies observations based on the underlying MM (for many subjects and time points simultaneously) and thus shows which observations are comparable. However, the complexity of the method or the fact that no open-source software for LMFA existed until now may have hindered researchers from applying the method in practice. In this article, we provide a step-by-step tutorial for the new user-friendly software package lmfa, which allows researchers to easily perform the analysis LMFA in the freely available software R to investigate MM differences in their own ILD

Continuous-time Latent Markov Factor Analysis for exploring measurement model changes across time

Drawing valid inferences about daily or long-term dynamics of psychological constructs (e.g., depression) requires the measurement model (indicating which constructs are measured by which items) to be invariant within persons over time. However, it might be affected by time- or situation-specific artifacts (e.g., response styles) or substantive changes in item interpretation. To efficiently evaluate longitudinal measurement invariance, and violations thereof, we proposed Latent Markov factor analysis (LMFA), which clusters observations based on their measurement model into separate states, indicating which measures are validly comparable. LMFA is, however, tailored to “discretetime” data, where measurement intervals are equal, which is often not the case in longitudinal data. In this paper, we extend LMFA to accommodate unequally spaced intervals. The so-called “continuous-time” (CT) approach considers the measurements as snapshots of continuously evolving processes. A simulation study compares CT-LMFA parameter estimation to its discrete-time counterpart and a depression data application shows the advantages of CT-LMFA