Two-Method Planned Missing Designs for Longitudinal Research

We examine longitudinal extensions of the two-method measurement design, which uses planned missingness to optimize cost-efficiency and validity of hard-to-measure constructs. These designs use a combination of two measures: a “gold standard” that is highly valid but expensive to administer, and an inexpensive (e.g., survey-based) measure that contains systematic measurement bias (e.g., response bias). Using simulated data on four measurement occasions, we compared the cost-efficiency and validity of longitudinal designs where the gold standard is measured at one or more measurement occasions. We manipulated the nature of the response bias over time (constant, increasing, fluctuating), the factorial structure of the response bias over time, and the constraints placed on the latent variable model. Our results showed that parameter bias is lowest when the gold standard is measured on at least two occasions. When a multifactorial structure was used to model response bias over time, it is necessary to have the “gold standard” measures included at every time point, in which case most of the parameters showed low bias. Almost all parameters in all conditions displayed high relative efficiency, suggesting that the 2-method design is an effective way to reduce costs and improve both power and accuracy in longitudinal research

Exploring Student, Family, and School Predictors of Self-Determination Using NLTS2 Data

This study conducted secondary analysis of data from the National Longitudinal Transition Study-2 (NLTS2) to examine the degree to which student, family, and school constructs predicted self-determination outcomes. Multi-group structural equation modeling was used to examine predictive relationships between 5 student, 4 family, and 7 school constructs developed from NLTS2 data and self-determination outcomes (autonomy, psychological empowerment, and self-realization) across disability groups. The pattern of predictive relationship between the constructs and self-determination outcomes across disability groups was complex. Only one construct—self-concept—showed a positive predictive relationship with all three self-determination constructs across most disability groups. Implications of the complex pattern of findings for research and practice are discussed

Multilevel Examination of Facility Characteristics, Social Integration, and Health for Older Adults Living in Nursing Homes

Objectives. Testing a model based on past research and theory, this study assessed relationships between facility characteristics (i.e., culture change efforts, social workers) and residents’ social networks and social support across nursing homes and examined relationships between multiple aspects of social integration (i.e., social networks, social capital, social engagement, social support) and mental and functional health for older adults in nursing homes. Methods. Data were collected at nursing homes using a planned missing data design with random sampling techniques. Data collection occurred at the individual-level through in-person structured interviews with older adult nursing home residents (N = 140) and at the facility-level (N = 30) with nursing home staff. Results. The best fitting multilevel structural equation model indicated that the culture change sub-scale for relationships significantly predicted differences in residents’ social networks. Additionally, social networks had a positive indirect relationship with mental and functional health among residents primarily via social engagement. Social capital had a positive direct relationship with both health outcomes. Discussion. To predict better social integration and mental and functional health outcomes for nursing homes residents, study findings support prioritizing that close relationships exist among staff, residents, and the community as well as increased resident social engagement and social trust

On the Joys of Missing Data

We provide conceptual introductions to missingness mechanisms—missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR)—and state-of-the-art methods of handling missing data—full-information maximum likelihood (FIML) and multiple imputation (MI)—followed by a discussion of planned missing designs: multiform questionnaire protocols, two-method measurement models, and wave-missing longitudinal designs. We reviewed 80 articles of empirical studies published in the 2012 issues of the Journal of Pediatric Psychology to present a picture of how adequately missing data are currently handled in this field. To illustrate the benefits of utilizing MI or FIML and incorporating planned missingness into study designs, we provide example analyses of empirical data gathered using a three-form planned missing design

What's the Matter with Kansas?: The Development and Confirmation of the Evolutionary Attitudes and Literacy Survey (EALS)

This is the publisher's version, also available electronically from http://link.springer.com/article/10.1007%2Fs12052-010-0294-1.The present survey was designed to assess predominant regional belief systems and the roles these beliefs play in science understanding and attitudes, and curricular effectiveness in colleges and universities. To this end, we created a wide variety of theory-driven subscales (lower order factors) reflecting, for example, exposure to evolutionary material, young earth creationist beliefs, moral and social objections, political ideology, endorsement of intelligent design fallacies, knowledge (and distrust) of the scientific enterprise, and attitudes of evolutionary theory’s relevance in several domains (e.g., sciences and humanities). We also included potentially important demographic variables (e.g., rural upbringing, family size). Finally, we assessed openness to experience, a key facet of personality. Hierarchical Structural Equation Modeling (SEM) analysis showed the 16 constructs to have a theoretically meaningful and quantitatively coherent higher order factor structure. In this large Kansas sample of university students, creationist reasoning and conservative orientation (political and religious) were negatively associated with exposure to evolutionary theory, knowledge about it, and positive attitudes toward its relevance. At the same time, exposure to the theory was positively associated with knowledge and positive attitudes. Importantly, though most Kansas-specific demographic variables (e.g., rural origins) were largely unrelated to outcomes of interest in this university-based sample, the personality factor openness to experience appears to be highly relevant for several higher order factors (e.g., exposure, knowledge and relevance, and creationist reasoning). We close with implications for educators and the next steps in survey development

A cross-cultural comparison of the self-determination construct in Italian and American adolescents

The functional theory of self-determination (fSDT) defines and operationalises self-determination within a human agentic context. It emerged from research on adolescents with disabilities, however has been increasingly applied to youth without disabilities. While comparability has been evaluated in youth with and without disabilities, it has not been explored across cultures. The purpose of this study was to explore the cross-cultural comparability of the fSDT in a sample of Italian and American adolescents. We were specifically interested in examining the universal aspects of the self-determination construct, as well as specific differences in the operationalisation of self-determination across cultures. The findings tentatively suggest that the construct of self-determination is comparable across Italian and American adolescents; however, there are specific differences in the measurement and operationalisation of self-determination across cultures. Directions for future research are discussed

Planned Missing Data Designs & Small Sample Size: How Small is Too Small?

Utilizing planned missing data (PMD) designs (ex. 3-form surveys) enables researchers to ask participants fewer questions during the data collection process. An important question, however, is just how few participants are needed to effectively employ planned missing data designs in research studies. This paper explores this question by using simulated three-form planned missing data to assess analytic model convergence, parameter estimate bias, standard error bias, mean squared error (MSE), and relative efficiency (RE).Three models were examined: a one-time point, cross-sectional model with 3 constructs; a two-time point model with 3 constructs at each time point; and a three-time point, mediation model with 3 constructs over three time points. Both full-information maximum likelihood (FIML) and multiple imputation (MI) were used to handle the missing data. Models were found to meet convergence rate and acceptable bias criteria with FIML at smaller sample sizes than with MI

The impact of the Self-Determined Learning Model of Instruction on student self-determination

This is the publisher's version, also found here: http://cec.metapress.com/content/c5n78j60w945vk2x/?p=9b7820f03e7c4a37be218efe08d17483&pi=0Promoting self-determination has become a best practice in special education. There remains, however, a paucity of causal evidence for interventions to promote self-determination. This article presents the results of a group-randomized, modified equivalent control group design study of the efficacy of the Self Determined Learning Model of Instruction (SDLMI, Wehmeyer, Palmer, Agran, Mithaug, & Martin, 2000) to promote self-determination. The authors used data on self-determination using multiple measures collected with 312 high school students with cognitive disabilities in both a control and a treatment group to examine the relationship between the SDLMI and self-determination. After determining strong measurement invariance for each latent construct, they found significant differences in latent means across measurement occasions and differential effects attributable to the SDLMI. This was true across disability category, though there was variance across disability populations