Single case experimental designs and their statistics

-

In God We Trust, All Others Bring Data: A Bayesian Approach to Standard Setting

Standard setting is an inherent part of pass/fail decisions in assessment. Although various standard setting methods are available, they all have their limitations and no method provides a golden solution to all our standard setting headaches. Some methods require potentially labor-intensive standard setting panels of judges who have specific knowledge. Other methods require student cohorts of ‘sufficient’ size. However, small cohorts are quite prevalent in medical programs across the globe, and standard setting panels are not always feasible due to logistic or financial constraints or may result in inadequate judgments due to bias or a lack of specific knowledge. This manuscript presents a new standard setting method, which is based on the Bayesian principle of updating our knowledge or beliefs about a phenomenon of interest with incoming data, uses information that is not considered in methods already available and can be applied to both small and larger cohorts regardless of whether standard setting panels are available. As demonstrated in this manuscript through a worked example, the new method is easy to implement and requires only a minimum of calculations which can be done in zero-cost, user-friendly Open Source software. Options for future research comparing different standard setting methods are discussed

Assessment programs and their components : a network approach

Exams and other assessments in health science education are not random events; rather, they are part of a bigger assessment program that is constructively aligned with the intended learning outcomes at different stages of a health science curriculum. Depending on topical and temporal distance, assessments in the program are correlated with each other to a more or lesser extent. Although correlation does not equate causation, once we come to understand the correlational structure of an assessment program, we can use that information to make predictions of future performance, to consider early intervention for students who are otherwise likely to drop out, and to inform revisions in either assessment or teaching. This article demonstrates how the correlational structure of an assessment program can be represented in terms of a network, in which the assessments constitute our nodes and the degree of connectedness between any two nodes can be represented as a thicker or thinner line connecting these two nodes, depending on whether the correlation between the two assessments at hand is stronger or weaker. Implications for educational practice and further research are discussed

THE SUBJECT OF STATISTICS IN NATURAL SCIENCE CURRICULA: A CASE STUDY

Statistics is considered to be an indispensable part of a wide range of curricula across the globe, natural science curricula included. Teachers and curriculum developers are typically confronted with four questions with regard to the role and position of statistics in a curriculum: (1) how to integrate statistics in the curriculum; (2) which topics to cover and in what detail; (3) how much time to allocate to statistics in a curriculum; and (4) how to organize a course and which study materials to select. This paper addresses these four questions through a case study: four curricula at Charles University, Prague, Czech Republic, are compared in terms of how they address these four questions. Placing this comparison in a framework of cognitive load theory and two decades of research inspired by this theory, this paper concludes with a number of guidelines for addressing the aforementioned four questions when designing a curriculum

Revisiting cognitive load theory : second thoughts and unaddressed questions

In cognitive load theory (CLT), learning is the development of cognitive schemas in a long-term memory with no known limits and can happen only if our limited working memory can process new information presented and the amount of information that does not contribute to learning is low. According to this theory, learning is optimal when instructional support is decreased going from worked examples via completion problem to autonomous problem solving and learners do not benefit from practicing retrieval with complex content. However, studies on productive failure and retrieval practice have provided clear evidence against these two guidelines. In this article, issues with CLT and research inspired by this theory, which remain largely ignored among cognitive load theorists but have likely contributed to these contradictory findings, are discussed. This article concludes that these issues should make us question the usefulness of CLT in health science education, medical education and other complex domains, and presents recommendations for both educational practice and future research on the matter

Statistics for N = 1 : A Non-Parametric Bayesian Approach

Research in education is often associated with comparing group averages and linear relations in sufficiently large samples and evidence-based practice is about using the outcomes of that research in the practice of education. However, there are questions that are important for the practice of education that cannot really be addressed by comparisons of group averages and linear relations, no matter how large the samples. Besides, different types of constraints including logistic, financial, and ethical ones may make larger-sample research unfeasible or at least questionable. What has remained less known in many fields is that there are study designs and statistical methods for research involving small samples or even individuals that allow us to address questions of importance for the practice of education. This article discusses one type of such situations and provides a simple coherent statistical approach that provides point and interval estimates of differences of interest regardless of the type of the outcome variable and that is of use in other types of studies involving large samples, small samples, and single individuals

Síntomas durante el uso de simulación con realidad virtual altamente inmersiva para el aprendizaje en salud

We read with great interest the article by González et al (1) supporting the use of highlyimmersive virtual reality (VR)simulationfor anatomy learning. The results of their systematicreview highlighted a positive reception of the VR experienceby students, with an increase inenjoyment and attention, and at least the same effectiveness for learning when compared totraditional cadaveric methods. They also highlighted VR technology canbecome a long-terminvestment that reduces costs.These findings are in line with the results of other systematic reviewsindicating that immersive VR for education was easy to use, facilitated learning of content (2), andincreased cognitive and psychomotor performance

We need more replication research – A case for test-retest reliability

Following debates in psychology on the importance of replication research, we have also started to see pleas for a more prominent role for replication research in medical education. To enable replication research, it is of paramount importance to carefully study the reliability of the instruments we use. Cronbach’s alpha has been the most widely used estimator of reliability in the field of medical education, notably as some kind of quality label of test or questionnaire scores based on multiple items or of the reliability of assessment across exam stations. However, as this narrative review outlines, Cronbach’s alpha or alternative reliability statistics may complement but not replace psychometric methods such as factor analysis. Moreover, multiple-item measurements should be preferred above single-item measurements, and when using single-item measurements, coefficients as Cronbach’s alpha should not be interpreted as indicators of the reliability of a single item when that item is administered after fundamentally different activities, such as learning tasks that differ in content. Finally, if we want to follow up on recent pleas for more replication research, we have to start studying the test-retest reliability of the instruments we use

Вариант теории упрочнения, учитывающий зависимость параметров уравнений состояния от напряжения и температуры

Предложена методика конкретизации определяющих соотношений теории упрочнения, учи­тывающей поврежденность материала. Предполагается, что параметры уравнения ползу­чести и эволюционного соотношения поврежденности являются функциями напряжения и температуры. Эффективность подхода проиллюстрирована при описании кривых ползу­чести сталей 20X13 и ЭП44 в достаточно широком диапазоне изменения напряжений.Запропоновано методику конкретизації визначальних співвідношень теорії зміцнення, яка враховує пошкоджуваність матеріалу. Припускається, що параметри рівняння повзучості й еволюційного співвідношення пошкоджу­ваності є функціями напруження і температури. Ефективність підходу про­ілюстровано при опису кривих повзучості сталей 20X13 і ЕП44 в достатньо широкому діапазоні зміни напружень.We propose a technique for concretization of the governing equations of the strain-hardening theory with account of the material damage. Parameters of the creep equation and damage evo­lution dependence are assumed to be functions of stress and temperature. The proposed ap­proach efficiency is demonstrated by descrip­tion of creep curves of steels 20Kh13 and EP44 in a broad range of stress variation