4,411 research outputs found
Bridging Parametric and Nonparametric Methods in Cognitive Diagnosis
A number of parametric and nonparametric methods for estimating cognitive
diagnosis models (CDMs) have been developed and applied in a wide range of
contexts. However, in the literature, a wide chasm exists between these two
families of methods, and their relationship to each other is not well
understood. In this paper, we propose a unified estimation framework to bridge
the divide between parametric and nonparametric methods in cognitive diagnosis
to better understand their relationship. We also develop iterative joint
estimation algorithms and establish consistency properties within the proposed
framework. Lastly, we present comprehensive simulation results to compare
different methods, and provide practical recommendations on the appropriate use
of the proposed framework in various CDM contexts
Nonparametric diagnostic classification analysis for testlet based tests
Diagnostic classification Diagnostic Classification Models (DCMs) are multidimensional confirmatory latent class models that can classify individuals into different classes based on their attribute mastery profiles. While DCMs represent the more prevalent parametric approach to diagnostic classification analysis, the Hamming distance method, a newly developed nonparametric diagnostic classification method, is quite promising in that it does not require fitting a statistical model and is less demanding on sample size. However, both parametric and nonparametric approach have assumptions of local item independency, which is often violated by testlet based tests. This study proposed a conditional-correlation based nonparametric approach to assess testlet effect and a set of testlet Hamming distance methods to account for the testlet effects in classification analyses. Simulation studies were conducted to evaluate the proposed methods. In the conditional-correlation approach, the testlet effects were computed as the average item-pair correlations within the same testlet by conditioning on attribute profiles. The inverse of the testlet effect was then used in testlet Hamming distance method to weight the Hamming distances for that particular testlet. Simulation studies were conducted to evaluate the proposed methods in conditions with varying sample size, testlet effect size, testlet size, balance of testlet size, and balance of testlet effect size. Although the conditional-correlation based approach often underestimated true testlet effect sizes, it was still able to detect the relative size of different testlet effects. The developed testlet Hamming distance methods seem to be an improvement over the estimation methods that ignore testlet effects because they provided slightly higher classification accuracy where large testlet effects were present. In addition, Hamming distance method and maximum likelihood estimation are robust to local item dependency caused by low to moderate testlet effects. Recommendations for practitioners and study limitations were provided
cdcatR: An R package for cognitive diagnostic computerized adaptive testing
Cognitive diagnosis models (CDMs) are confirmatory latent class models that provide
fine-grained information about skills and cognitive processes. These models have gained attention in
the last few years because of their usefulness in educational and psychological settings. Recently,
numerous developments have been made to allow for the implementation of cognitive diagnosis
computerized adaptive testing (CD-CAT). Despite methodological advances, CD-CAT applications
are still scarce. To facilitate research and the emergence of empirical applications in this area, we
have developed the cdcatR package for R software. The purpose of this document is to illustrate the
different functions included in this package. The package includes functionalities for data generation,
model selection based on relative fit information, implementation of several item selection rules
(including item exposure control), and CD-CAT performance evaluation in terms of classification
accuracy, item exposure, and test length. In conclusion, an R package is made available to researchers
and practitioners that allows for an easy implementation of CD-CAT in both simulation and applied
studies. Ultimately, this is expected to facilitate the development of empirical applications in this areaThis research was funded by Ministerio de Ciencia e Innovación, grant number PSI2017-
85022-P, and Cátedra de Modelos y Aplicaciones Psicométricas (Instituto de Ingeniería del
Conocimiento and Autonomous University of Madrid
Some theoretical and applied developments to support cognitive learning and adaptive testing
Cognitive diagnostic Modeling (CDM) and Computerized Adaptive Testing (CAT) are useful tools to measure subjects' latent abilities from two different aspects. CDM plays a very important role in the fine-grained assessment, where the primary purpose is to accurately classify subjects according to the skills or attributes they possess, while CAT is a useful tool for coarse-grained assessment, which provides a single number to indicate the student's overall ability. This thesis discusses and solves several theoretical and applied issues related to these two areas.
The first problem we investigate related to a nonparametric classifier in Cognitive Diagnosis. Latent Class models for cognitive diagnosis have been developed to classify examinees into one of the 2K attribute profiles arising from a K-dimensional vector of binary skill indicators. These models recognize that response patterns tend to deviate from the ideal responses that would arise if skills and items generated item responses through a purely deterministic conjunctive process. An alternative to employing these latent class models is to minimize the distance between observed item response patterns and ideal response patterns, in a nonparametric fashion that utilizes no stochastic terms for these deviations. Theorems are presented that show the consistency of this approach, when the true model is one of several common latent class models for cognitive diagnosis. Consistency of classification is independent of sample size, because no model parameters need to be estimated. Simultaneous consistency for a large group of subjects can also be shown given some conditions on how sample size and test length grow with one another.
The second issue we consider is still within CDM framework, however our focus is about the model misspecification. The maximum likelihood classification rule is a standard method to classify examinee attribute profiles in cognitive diagnosis models. Its asymptotic behavior is well understood when the model is assumed to be correct, but has not been explored in the case of misspecified latent class models. We investigate the consequences of using a simple model when the true model is different. In general, when a CDM is misspecified as a conjunctive model, the MLE for attribute profiles is not necessarily consistent. A sufficient condition for the MLE to be a consistent estimator under a misspecified DINA model is found. The true model can be any conjunctive models or even a compensatory model. Two examples are provided to show the consistency and inconsistency of the MLE under a misspecified DINA model. A Robust DINA MLE technique is proposed to overcome the inconsistency issue, and theorems are presented to show that it is a consistent estimator for attribute profile as long as the true model is a conjunctive model. Simulation results indicate that when the true model is a conjunctive model, the Robust DINA MLE and the DINA MLE based on the simulated item parameters can result in relatively good classification results even when the test length is short. These findings demonstrate that simple models can be fitted without severely affecting classification accuracy in some cases.
The last one discusses and solves a controversial issue related to CAT. In Computerized Adaptive Testing (CAT), items are selected in real time and are adjusted to the test-taker's ability. A long debated question related to CAT is that they do not allow test-takers to review and revise their responses. The last chapter of this thesis presents a CAT design that preserves the efficiency of a conventional CAT, but allows test takers to revise their previous answers at any time during the test, and the only imposed restriction is on the number of revisions to the same item. The proposed method relies on a polytomous Item Response Theory model that is used to describe the first response to each item, as well as any subsequent revisions to it. The test-taker's ability is updated on-line with the maximizer of a partial likelihood function. I have established the strong consistency and asymptotic normality of the final ability estimator under minimal conditions on the test-taker's revision behavior. Simulation results also indicated this proposed design can reduce measurement error and is robust against several well-known test-taking strategies
The effects of mixture-induced local dependence on diagnostic classification
Diagnostic Classification Models (DCMs) have been extensively researched in recent psychometric literature for providing mastery skill profiles for diagnostic feedback (Henson, Templin, & Willse, 2009). DCMs are multidimensional confirmatory latent class models (LCMs) where latent classes represent skill mastery profiles and latent attributes are categorical (mastery or non-mastery). DCMs make a central assumption that once mastery profiles are accounted for that items are independent, referred to as local independence (LI). Construct irrelevant variance (e.g., differential item functioning (DIF), speededness, test wiseness, item-to-skill misspecification) or underrepresentation (extra dimensionality, inappropriate definitional grain-size of defined skills) could introduce systematic within-class variation which would violate LI. Using connections of LCMs with mixture IRT models, this study explores the effects of introducing systematic within-class variation on diagnostic classification. The log-linear cognitive diagnosis model (LCDM) is extended to include continuous abilities, akin to a multidimensional item response theory (MIRT) model with underling mixtures due to skill mastery/nonmastery. Data were then simulated for different ability variances related to distribution overlap conditions. Multiple DCMs are then fit using the LCDM framework in a simulation study. Impact on classification and local dependence detection are summarized. It was found that as mixture overlap increased due to companion ability variance that diagnostic classification in DCMs greatly suffered, but can be detected by Yen’s Q3. The relationship of the degree of inaccuracy and effect sizes based on ability variance and group separation is delineated. Recommendations for practitioners are given along with areas for future study
Potential Alzheimer\u27s Disease Plasma Biomarkers
In this series of studies, we examined the potential of a variety of blood-based plasma biomarkers for the identification of Alzheimer\u27s disease (AD) progression and cognitive decline. With the end goal of studying these biomarkers via mixture modeling, we began with a literature review of the methodology. An examination of the biomarkers with demographics and other health factors found evidence of minimal risk of confounding along the causal pathway from biomarkers to cognitive performance. Further study examined the usefulness of linear combinations of biomarkers, achieved via partial least squares (PLS) analysis, as predictors of various cognitive assessment scores and clinical cognitive diagnosis. The identified biomarker linear combinations were not effective at predicting cognitive outcomes. The final study of our biomarkers utilized mixture modeling through the extension of group-based trajectory modeling (GBTM). We modeled five biomarkers, covering a range of functions within the body, to identify distinct trajectories over time. Final models showed statistically significant differences in baseline risk factors and cognitive assessments between developmental trajectories of the biomarker outcomes. This course of study has added valuable information to the field of plasma biomarker research in relation to Alzheimer’s disease and cognitive decline
Recommended from our members
Application of Deep Learning to Brain Connectivity Classification in Large MRI Datasets
The use of machine learning for whole-brain classification of magnetic resonance imaging (MRI) data is of clear interest, both for understanding phenotypic differences in brain structure and function and for diagnostic applications. Developments of deep learning models in the past decade have revolutionized photographic image and speech recognition, bringing promise to do the same to other fields of science. However, there are many practical and theoretical challenges in the translation of such methods to the unique context of MRIs of the brain. This thesis presents a theoretical underpinning for whole-brain classification of extremely large datasets of multi-site MRIs, including machine learning model architecture, dataset curation methods, machine learning visualization methods, encoding of MRI data, and feature extraction. To replicate large sample sizes typically applied to deep learning models, a dataset of over 50,000 functional and structural MRIs was amassed from nine different databases, and the undertaken analyses were conducted on three covariates commonly found across these collections: sex, resting state/task, and autism spectrum disorder. I find that deep learning is not only a method that has promise for clinical application in the future, but also a powerful statistical tool for analyzing complex, nonlinear relationships in brain data where conventional statistics may fail. However, results are also dependent on factors such as dataset imbalances, confounding factors such as motion and head size, selected methods of encoding MRI data, variability of machine learning models and selected methods of visualizing the machine learning results. In this thesis, I present the following methodological innovations: (1) a method of balancing datasets as a means of regressing out measurable confounding factors; (2) a means of removing spatial biases from deep learning visualization methods; (3) methods of encoding functional and structural datasets as connectivity matrices; (4) the use of ensemble models and convolutional neural network architectures to improve classification accuracy and consistency; (5) adaptation of deep learning visualization methods to study brain connections utilized in the classification process. Additionally, I discuss interpretations, limitations, and future directions of this research.Gates Cambridge Scholarshi
Student Perceptions of Biology Teachers\u27 Interpersonal Teaching Behaviors and Student Achievement
Inadequate student-teacher interactions in undergraduate courses have been linked to poor student performance. Researchers have noted that students\u27 perceptions of student-teacher relationships may be an important factor related to student performance. The administration of a Mid-Atlantic community college prioritized increasing undergraduate biology student performance. The purpose of this quantitative study was to examine the relationship between students\u27 biology achievement and their perceptions of interpersonal teaching behaviors and student-teacher interactions in introductory biology courses. Leary\u27s theory on interpersonal communication and the systems communication theory of Watzlawick, Beavin, and Jackson served as the theoretical foundation. The Wubbel\u27s Likert-scale questionnaire on student-teacher interactions was administered to 318 undergraduate biology students. Non-parametric Spearman\u27s rank correlations revealed a significant direct correlation between students\u27 grades and their perceptions of teachers\u27 interpersonal teaching behaviors. The relationship between student achievement and students\u27 perceptions of student-teacher interactions prompted the recommendation for additional study on the importance of student-teacher interactions in undergraduate programs. A recommendation for local practice included faculty development on strategies for improving student-teacher interactions. The study\u27s implications for positive social change include increased understanding for administrators and instructors on the importance of teacher-student interactions at the community college level
Does age moderate self-pain enmeshment in chronic pain patients?
Research has demonstrated that chronic pain can compromise identity by becoming enmeshed and centralised with pain. Pain-identity enmeshment and pain-identity centrality are associated with greater affective distress and poorer chronic pain adjustment. However, the literature infers differences between older and younger individuals in terms of pain adjustment, whereby older adults perceive pain as concomitant of aging and experience this as less biographically disruptive and perceive themselves to be younger than their chronological age, which is associated with greater psychological wellbeing. Research has yet to explore the relationship between perceived age and pain-identity enmeshment and adjustment in chronic pain. The purpose of this research was to investigate age in relation to pain-identity enmeshment and centrality and to examine the predictive value of age in pain adjustment.
90 patients with osteoarthritis (OA) and chronic pain were recruited from a musculoskeletal service. Participants completed standardised measures of pain intensity and perceived control (VAS), pain severity and interference (BPI), acceptance (CPAQ), identity (CES, Possible Selves Interviews), affective distress (HADS), and catastrophising (PCS) and provided information regarding their perceived age. Statistical analysis included; correlation, chi square, analysis of variance and linear regression to investigate potential age differences.
Chronological age evidenced few significant relationships with variables of pain adjustment and identity. Perceived age evidenced significant relationships with all variables of adjustment and identity, however, did not statistically predict chronic pain adjustment. However, hoped-for proximity and centrality significantly predicted chronic pain adjustment. The CES demonstrated significant relatedness to enmeshment, although effect sizes were small. Therefore, it appears possible that an individual may experience pain becoming central to their identity yet remain un-enmeshed with pain.
These findings indicate the necessity to assess hoped-for proximity and centrality in chronic pain populations across all age groups. This research indicates the potential for incorrectly perceiving expectedness and adjustment ease in old age. The implications of these findings are explored, in conjunction with the limitations of this research and potential areas for further research
- …