Utilizing Generalizability Theory to Investigate the Reliability of Grades Assigned to Undergraduate Research Papers

Background:  Educational researchers have long espoused the virtues of writing with regard to student cognitive skills. However, research on the reliability of the grades assigned to written papers reveals a high degree of contradiction, with some researchers concluding that the grades assigned are very reliable whereas others suggesting that they are so unreliable that random assignment of grades would have been almost as helpful.   Purpose: The primary purpose of the study was to investigate the reliability of grades assigned to written reports. The secondary purpose was to illustrate the use of Generalizability Theory, specifically the fully-crossed two-facet model, for computing interrater reliability coefficients.   Setting: The participants for this study were 29 undergraduate students enrolled in an introductory-level course on Political Behavior in Spring 2011 at a Midwest university.   Intervention: Not applicable. Research Design: Students were randomly assigned to one of nine groups. Two-facet fully crossed G-study and D-study designs were used wherein two raters graded four assignments for 9 student groups—72 evaluations in total. The universe of admissible observations was deemed to be random for both raters and assignments, whereas the universe of generalization was deemed to be mixed (random for two raters but fixed for four assignments).   Data Collection and Analysis: The semester-long project was assigned to groups consisting of an annotated bibliography, survey development, sampling design, and analysis and final report. Four grading rubrics were developed and utilized to evaluate the quality of each written report. Two-facet generalizability analyses were conducted to assess interrater reliability using software developed by one of the authors.   Findings: This study found a very high interrater reliability coefficient (0.929) for only two raters who received no training in how to use the four grading rubrics. Keywords: grading; reliability; Generalizability Theory; writing  &nbsp

New Preshower detector for DIRAC Experiment

The Preshower (PSh) detector is a component of the DIRAC setup. It is designed to improve rejection efficiency of e-e+ pairs background in the {\pi}{\pi} and K{\pi} pair measurement. To increase the overall efficiency, a new two-layer structure scintillator Preshower detector has been realized in the region where the Nitrogen Cherenkov detector has been shortened to introduce new detectors. The new Preshower-Cherenkov combination ensures the electron rejection efficiency better than 99.9% in momentum region 1-7 GeV/c.Comment: to be published in NIM

How Good are Our Measures? Investigating the Appropriate Use of Factor Analysis for Survey Instruments

Background: Evaluation work frequently utilizes factor analysis to establish the dimensionality, reliability, and stability of surveys. However, survey data is typically ordinal, violating the assumptions of most statistical methods, and thus is often factor-analyzed inappropriately. Purpose: This study illustrates the salient analytical decisions for factor-analyzing ordinal survey data appropriately and demonstrates the repercussions of inappropriate analyses. Setting: The data used for this study are drawn from an evaluation of the efficacy of a drama-based approach to teaching Shakespeare in elementary and middle school.  Intervention: Not applicable. Research Design: Survey research. Data Collection and Analysis: Four factor analytic methods were compared: a traditional exploratory factor analysis (EFA), a full-information EFA, and two EFAs within the confirmatory factor analysis framework (E/CFA) conducted according to the Jöreskog method and the Gugiu method. Findings: Methods appropriate for ordinal data produce better models, the E/CFAs outperform the EFAs, and the Gugiu method demonstrates greater model interpretability and stability than the Jöreskog method. These results suggest that the Gugiu E/CFA may be the preferable factor analytic method for use with ordinal data. Practical applications of these findings are discussed. Keywords: factor analysis; ordinal data; E/CFA; survey research

Quantitative Methods for Estimating the Reliability of Qualitative Data

Background: Measurement is a indispensable aspect of conducting both quantitative and qualitative research and evaluation. With respect to qualitative research, measurement typically occurs during the coding process.   Purpose: This paper presents quantitative methods for determining the reliability of conclusions from qualitative data sources. Although some qualitative researchers disagree agree with such application, a link between the qualitative and quantitative fields is successfully established through data collection and coding procedures.   Setting: Not applicable.   Intervention: Not applicable. Research Design: Case study.   Data Collection and Analysis: Narrative data were collected from a random sample of 528 undergraduate students and 28 professors.   Findings: The calculation of the kappa statistic, weighted kappa statistic, ANOVA Binary Intraclass Correlation, and Kuder-Richardson 20 is illustrated through a fictitious example. Formulae are presented so that the researcher can calculate these estimators without the use of sophisticated statistical software.   Keywords: qualitative coding; qualitative methodology; reliability coefficient

High-quality chronic care delivery improves experiences of chronically ill patients receiving care

__Abstract__ Objective. Investigate whether high-quality chronic care delivery improved the experiences of patients. Design. This study had a longitudinal design. Setting and Participants. We surveyed professionals and patients in 17 disease management programs targeting patients with cardiovascular diseases, chronic obstructive pulmonary disease, heart failure, stroke, comorbidity and eatin

Experimental investigation of the effect of ionization on the 51V(p,n)51Cr reaction

The investigation of the effects of average atomic ionization on nuclear reactions is of prime importance for nuclear astrophysics. No direct experimental measurement using a plasma target has been done yet. In this regard, we measured for the first time the neutron production of a (p,n) reaction in different states of ionization. The studied nuclear reaction was 51V(p,n)51Cr. We measured a significantly lower neutron production than expected when the target was ionized, even when taking into account existing electron screening theory or the effect of the stopping power in the target on the injected proton beam. This experiment is a first step in the process to characterize the influence of ionization at astrophysically relevant energies.Comment: 20 pages, 10 figures, submitted to EP

First πK\pi K atom lifetime and πK\pi K scattering length measurements

The results of a search for hydrogen-like atoms consisting of $\pi^{\mp}K^{\pm}$ mesons are presented. Evidence for $\pi K$ atom production by 24 GeV/c protons from CERN PS interacting with a nickel target has been seen in terms of characteristic $\pi K$ pairs from their breakup in the same target ($178 \pm 49$) and from Coulomb final state interaction ($653 \pm 42$). Using these results the analysis yields a first value for the $\pi K$ atom lifetime of $\tau=(2.5_{-1.8}^{+3.0})$ fs and a first model-independent measurement of the S-wave isospin-odd $\pi K$ scattering length $\left|a_0^-\right|=\frac{1}{3}\left|a_{1/2}-a_{3/2}\right|= \left(0.11_{-0.04}^{+0.09} \right)M_{\pi}^{-1}$ ($a_I$ for isospin $I$).Comment: 14 pages, 8 figure