Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses

Articles published in several prominent educational journals were examined to investigate the use of data-analysis tools by researchers in four research paradigms: between-subjects univariate designs, between-subjects multivariate designs, repeated measures designs, and covariance designs. In addition to examining specific details pertaining to the research design (e.g., sample size, group size equality/inequality) and methods employed for data analysis, we also catalogued whether: (a) validity assumptions were examined, (b) effect size indices were reported, (c) sample sizes were selected based on power considerations, and (d) appropriate textbooks and/or articles were cited to communicate the nature of the analyses that were performed. Our analyses imply that researchers rarely verify that validity assumptions are satisfied and accordingly typically use analyses that are nonrobust to assumption violations. In addition, researchers rarely report effect size statistics, nor do they routinely perform power analyses to determine sample size requirements. We offer many recommendations to rectify these shortcomings.Social Sciences and Humanities Research Counci

The effect of skewness and kurtosis on the robustness of linear mixed models

This study analyzes the robustness of the linear mixed model (LMM) with the Kenward–Roger (KR) procedure to violations of normality and sphericity when used in split-plot designs with small sample sizes. Specifically, it explores the independent effect of skewness and kurtosis on KR robustness for the values of skewness and kurtosis coefficients that are most frequently found in psychological and educational research data. To this end, a Monte Carlo simulation study was designed, considering a split-plot design with three levels of the between-subjects grouping factor and four levels of the within-subjects factor. Robustness is assessed in terms of the probability of type I error. The results showed that (1) the robustness of the KR procedure does not differ as a function of the violation or satisfaction of the sphericity assumption when small samples are used; (2) the LMM with KR can be a good option for analyzing total sample sizes of 45 or larger when their distributions are normal, slightly or moderately skewed, and with different degrees of kurtosis violation; (3) the effect of skewness on the robustness of the LMM with KR is greater than the corresponding effect of kurtosis for common values; and (4) when data are not normal and the total sample size is 30, the procedure is not robust. Alternative analyses should be performed when the total sample size is 30

Transcriptomic analysis of the late stages of grapevine (Vitis vinifera cv. Cabernet Sauvignon) berry ripening reveals significant induction of ethylene signaling and flavor pathways in the skin

Background: Grapevine berry, a nonclimacteric fruit, has three developmental stages; the last one is when berrycolor and sugar increase. Flavors derived from terpenoid and fatty acid metabolism develop at the very end of thisripening stage. The transcriptomic response of pulp and skin of Cabernet Sauvignon berries in the late stages ofripening between 22 and 37 \ub0Brix was assessed using whole-genome micorarrays.Results: The transcript abundance of approximately 18,000 genes changed with \ub0Brix and tissue type. There were alarge number of changes in many gene ontology (GO) categories involving metabolism, signaling and abioticstress. GO categories reflecting tissue differences were overrepresented in photosynthesis, isoprenoid metabolismand pigment biosynthesis. Detailed analysis of the interaction of the skin and pulp with \ub0Brix revealed that therewere statistically significantly higher abundances of transcripts changing with \ub0Brix in the skin that were involved inethylene signaling, isoprenoid and fatty acid metabolism. Many transcripts were peaking around known optimalfruit stages for flavor production. The transcript abundance of approximately two-thirds of the AP2/ERF superfamilyof transcription factors changed during these developmental stages. The transcript abundance of a unique clade ofERF6-type transcription factors had the largest changes in the skin and clustered with genes involved in ethylene,senescence, and fruit flavor production including ACC oxidase, terpene synthases, and lipoxygenases. The transcriptabundance of important transcription factors involved in fruit ripening was also higher in the skin.Conclusions: A detailed analysis of the transcriptome dynamics during late stages of ripening of grapevine berriesrevealed that these berries went through massive transcriptional changes in gene ontology categories involvingchemical signaling and metabolism in both the pulp and skin, particularly in the skin. Changes in the transcriptabundance of genes involved in the ethylene signaling pathway of this nonclimacteric fruit were statisticallysignificant in the late stages of ripening when the production of transcripts for important flavor and aroma compoundswere at their highest. Ethylene transcription factors known to play a role in leaf senescence also appear to play a role infruit senescence. Ethylene may play a bigger role than previously thought in this non-climacteric fruit

Medical students can learn the basic application, analytic, evaluative, and psychomotor skills of critical care medicine

Objective: To determine whether fourth-year medical students can learn the basic analytic, evaluative, and psychomotor skills needed to initially manage a critically ill patient. Design: Student learning was evaluated using a performance examination, the objective structured clinical examination (OSCE). Students were randomly assigned to one of two clinical scenarios before the elective. After the elective, students completed the other scenario, using a crossover design. Setting: Five surgical intensive care units in a tertiary care university teaching hospital. Participants: Forty fourth-year medical students enrolled in the critical care medicine (CCM) elective. Interventions: All students evaluated a live 'simulated critically ill' patient, requested physiologic data from a nurse, ordered laboratory tests, received data in real time, and intervened as they deemed appropriate. Measurements and Main Results: Student performance of specific behavioral objectives was evaluated at five stations. They were expected to a) assess airway, breathing, and circulation in appropriate sequence; b) prepare a manikin for intubation, obtain an acceptable airway on the manikin, demonstrate bag-mouth ventilation, and perform acceptable laryngoscopy and intubation; c) provide appropriate mechanical ventilator settings; d) manage hypotension; and e) request and interpret pulmonary artery data and initiate appropriate therapy. OSCEs were videotaped and reviewed by two faculty members masked to time of examination A checklist of key behaviors was used to evaluate performance The primary outcome measure was the difference in examination score before and after the rotation. Secondary outcomes included the difference in scores at each rotation. The mean preelective score was 57.0% ± 8.3% compared with 85.9% ± 7.4% (p < .0001) after the elective. Significant improvement was demonstrated at each station except station I. Conclusion: Fourth-year medical students without a CCM elective do not possess the basic cognitive and psychomotor skills necessary to initially manage critically ill patients. After an appropriate 1-month CCM elective, students' thinking and application skills required to initially manage critically ill patients improved markedly, as demonstrated by an OSCE using a live simulated 'patient' and manikin

Influence of perceived difficulty of cases on student osteopaths’ diagnostic reasoning: a cross sectional study

Abstract Background Diagnostic reasoning refers to the cognitive processes by which clinicians formulate diagnoses. Despite the implications for patient safety and professional identity, research on diagnostic reasoning in osteopathy remains largely theoretical. The aim of this study was to investigate the influence of perceived task difficulty on the diagnostic reasoning of students osteopaths. Methods Using a single-blinded, cross sectional study design, sixteen final year pre-registration osteopathy students diagnosed two standardized cases under two context conditions (complex versus control). Context difficulty was manipulated via verbal manipulation and case order was randomized and counterbalanced across subjects to ensure that each case was diagnosed evenly under both conditions (i.e. half of the subjects performed either case A or B first). After diagnosis, participants were presented with items (literal, inferred and filler) designed to represent analytical and non-analytical reasoning. Response time and error rate for each item were measured. A repeated measures analysis of variance (concept type x context) was performed to identify differences across conditions and make inferences on diagnostic reasoning. Results Participants made significantly more errors when judging literal concepts and took significantly less time to recognize filler concepts in the complex context. No significant difference in ability to judge inferred concepts across contexts was found. Conclusions Although speculative and preliminary, our findings suggest the perception of complexity led to an increased reliance on analytical reasoning at the detriment of non-analytical reasoning. To reduce the associated cognitive load, osteopathic educational institutions could consider developing the intuitive diagnostic capabilities of pre-registration students. Postgraduate mentorship opportunities could be considered to enhance the diagnostic reasoning of professional osteopaths, particularly recent graduates. Further research exploring the influence of expertise is required to enhance the validity of this study