Informing Writing: The Benefits of Formative Assessment

Examines whether classroom-based formative writing assessment - designed to provide students with feedback and modified instruction as needed - improves student writing and how teachers can improve such assessment. Suggests best practices

State of the art review : language testing and assessment (part two).

In Part 1 of this two-part review article (Alderson & Banerjee, 2001), we first addressed issues of washback, ethics, politics and standards. After a discussion of trends in testing on a national level and in testing for specific purposes, we surveyed developments in computer-based testing and then finally examined self-assessment, alternative assessment and the assessment of young learners. In this second part, we begin by discussing recent theories of construct validity and the theories of language use that help define the constructs that we wish to measure through language tests. The main sections of the second part concentrate on summarising recent research into the constructs themselves, in turn addressing reading, listening, grammatical and lexical abilities, speaking and writing. Finally we discuss a number of outstanding issues in the field

Computer-Based Assessment of Non-Cognitive Attributes of Occupational Therapy Students: A Pilot Evaluation

Purpose: Holistic admissions tools have been considered for use to diversify student admissions. The ability to screen non-cognitive attributes of applicants is an important element of holistic admissions. The objective of this study was to establish instrument reliability, inter-rater reliability, validity, item discrimination, item difficulty, and bias of the Computerized Assessment of Non-Cognitive Attributes of Health Care Professionals (CANA-HP). Methods: A novel methodology was developed to screen non-cognitive attributes of health profession applicants. Using a cross-sectional design, a convenience sample of students invited for interviews into a Mid-western occupational therapy program were recruited for participation. The 37 participants who consented, completed a demographic survey followed by the 12 question CANA-HP. Results: Open-ended questions had adequate internal reliability, discrimination, and difficulty. Multiple choice questions had low reliability and discrimination. No correlation was found between the CANA-HP and standardized cognitive assessments, except non-science GPA which was significantly correlated to the total open-ended (essay) scores and total overall score. Conclusions and Recommendations: The CANA-HP was not biased toward individuals from varied demographic backgrounds. Predictive validity of this tool and non-cognitive measurements of success are still needed. Occupational therapists in educational settings could consider adding open-ended ethical questions to the application process when interviews are not feasible. Classes in the non-sciences and other non-cognitive markers of success may also help identify students with ‘grit’ and critical reasoning which are important to practicing clinician

WV Writes and Westest 2 online writing: the impact of writing prompts on student writing proficiency

This study’s purpose was to determine the effects of students practicing writing using practice writing prompts prior to completing the summative state writing assessment. It adds data to the body of knowledge related to the use of practicing writing using practice prompts prior to students taking a high stakes state-level writing assessment. The type of research design used for this study was a quantitative, post hoc, 2 x 2 ANOVA. The data were obtained from the WESTEST 2 Online Writing composite scores and the five analytic trait scores that comprise the WESTEST 2 Online Writing scores. The study had a population of 6,459 11th grade students enrolled in West Virginia public schools. These students had all taken the WESTEST 2 Online Writing as 11th graders in the spring of 2013, and in preparation for the year-end, state level writing assessment completed either Writing Roadmap 2.0 prompts or WESTEST 2 practice prompts. Using random sampling, 190 students who wrote essays using WESTEST 2 practice prompts and 190 students who wrote essays using Writing Roadmap prompts were selected from the student population. This gave a total of 380 students in the sample size. Findings revealed that no significant effects were found when using one type of writing prompt over another on composite writing scores or on the five analytic writing scores. However, significance was demonstrated (p. 000) with the scores of females being greater than male students. Results gave stakeholders evidence that students who had a generic writing prompt versus a mirror image of the high stakes writing assessment scored no better or worse than the other. The new assessment vendors, states, counties, schools, and teachers will all benefit from these study findings as new assessment systems are adopted based on Common Core writing standards across the nation. The results are critical in supporting the discrepancy that stands between females and males and their writing proficiencies. This study can support efforts that focus on addressing the inequalities and ensuring that the discrepancy is removed and male students become better writers

Scaling Undergraduate Scientific Writing via Prominent Feature Analysis

Prominent Feature Analysis (PFA) is a reliable and valid writing assessment tool, derived from the writing it is used to assess. PFA, used to assess on-demand expository essays in Grades 3-12, uncovers positive and negative characteristics of a sample. To extend PFA to a new academic level and genre, I assessed scientific writing of 208 undergraduates, identifying 35 linguistic and 20 scientific prominent features. An essay could earn up to 28 positive (24 linguistic and four scientific), and up to 27 negative marks (11 linguistic and 16 scientific). The minimum prominent features number in a paper was 3, the maximum was 25 (M = 12.45, SD = 3.88). The highest positive and negative prominent features numbers noted were 17 (M = 4.11, SD = 3.96), and 16 (M = 8.34, SD = 3.25) respectively. Rasch analysis revealed a good data-model fit, with item separation of 5.81 (.97 reliability). The estimated feature difficulty of items spanned over 10 logits; common errors were easier to avoid than “good writing” characteristics to exhibit. Significant correlations among linguistic, but not between linguistic and scientific features, suggest writing proficiency does not assure excellence in scientific writing in novices. Ten linguistic features significantly strongly and moderately inter-correlated with each other, appearing to represent writing proficiency. Student GPA correlated significantly with the raw prominent features scores (r = .37; p \u3c .01), and negatively with the sum of negative linguistic features (r = -.40, p \u3c .01), providing support for scale’s validity, and suggesting that good students are better at avoiding common writing errors than less able learners. Additionally, PFA scores positively significantly correlated with composite ACT scores. To investigate PFA’s ability to track change in writing over time, I compared 2 sets of prominent features scores of 25 students. In comparison with earlier essays, later (longer) essays exhibited significantly more positive, and more negative features. Prominent features scores did not correlate significantly between the sets. This suggests, that while PFA is a valid and appropriate tool for analysis of undergraduate scientific writing, it was not suitable for tracking change in writing ability in this small sample

Performances: Assessment Resource Kit

The assessment of some kinds of learning requires the observation of student performances. Examples include playing a musical instrument, dancing, using scientific apparatus, operating equipment, performing gymnastics routines, diving, problem-solving in a group, applying ball-handling skills, and participating in small group discussions. \u27Performance assessment\u27 is the assessment of students as they engage in an activity. It is the on-the-spot evaluation of performance, behaviour or interaction. This issue on the assessment of performances in the Assessment Resource Kit (ARK) magazine series contains five articles: 1. What is performance assessment? 2. Informal classroom observations; 3. Classroom performance assessment \u27events\u27; 4. When comparability matters; 5. Designing performance assessments.https://research.acer.edu.au/ark/1003/thumbnail.jp

A Study Of An Attempt To Improve The Reliability Of Teachers\u27 Holistic Scores Of Elementary Writing Through In-house Profess

This study evaluated the effectiveness of a school-based training that attempted to improve the reliability of holistic scores teachers assigned to the writings of elementary school students. Seventeen teachers at one suburban elementary school located in the Southeastern United States participated in three training sessions that allowed for scoring practice and group discussions. The trainers, or presenters, were faculty-experts. A comparison of scores the participants assigned to students\u27 writings before and after the training was conducted. The analyses included t-tests that compared the participants\u27 mean scores to the scores assigned by raters from the state, a within-group analysis of reliability as measured by Cronbach\u27s Alpha, and percentage agreement analyses. The results suggested that the in-house training activities promoted higher inter-rater reliability of scores assigned to students\u27 writings by the teachers in this study. This study also compared teachers identified as being highly confident writers with teachers who reported low levels of self-confidence related to writing. Prior to the training, the highly confident teachers\u27 scores tended to be lower than the state scores and the scores assigned by their less confident peers. During group discussions, however, the high-confidence group was just as likely to change their scores to a higher level as to a lower level, and by the end of the training, both groups demonstrated more consistent score patterns

The standard measurement in online learning: a rubric as a focus on teaching-learning practices to move up quality education

The main objective of this paper is to facilitate the 21st century’s quality assurance for learning and distance learning to measure learning outcomes and enable students to achieve skills and knowledge in global transformative education. Technologies in education have provided a common platform for transforming education into online learning practices throughout the global COVID-19 affliction. Accordingly, the transformation of modern technologies in education has opened up many challenges and reliability issues in teaching and learning processes. The paper highlights the rethinking of rubric perceptions, uses, and design in learning, including online learning, in an interactive way to promote quality education. The research methodology involved a systematic review of the literature and data analysis synthesized using both qualitative and quantitative techniques. Rubrics, as an influential assessment instrument in learning, have been applied along with standardizing the quality of students in higher education for almost three decades globally. This study is significant in focusing on the rubric, the practical instrument, which is helpful for great accomplishments and challenging skills, and creativity in tertiary education. Well-designed and carefully designed rubrics and e-rubrics help students become more rational judges of the quality of their work by identifying the essential criteria and targeting the goals and learning outcomes of education. Rubrics enhance students\u27 competency to communicate their ideas effectively, which correlates with the standard of sustainable quality education. Further examination of the rubric is needed to identify the challenges of reliability and authenticity in digital education

Estimating intra-rater reliability on an oral english proficiency test from a Bilingual Education Program

Este estudio tiene como objetivo presentar los resultados de una investigación la cual pretendía estimar el nivel de confiabilidad intra-evaluador en un examen de suficiencia oral en inglés, y determinar los diferentes factores internos y externos que afectan la consistencia del evaluador. Los participantes involucrados en el desarrollo de este estudio fueron dos profesores encargados de evaluar la sección de habla de un examen de suficiencia administrado en la Licenciatura en Bilingüismo con énfasis en inglés. Se calculó un coeficiente de correlación con el fin de establecer la consistencia de los evaluadores mientras que un protocolo verbal retrospectivo fue llevado a cabo para recopilar información acerca de los factores que influyen en la confiabilidad del evaluador. Los resultados sugieren que hay un alto nivel de confiabilidad intra-evaluador en el examen de suficiencia en cuanto el coeficiente de correlación arrojó valores superiores a .80. No obstante, aspectos relacionados con la falta de adhesión a los criterios de la rúbrica, la relación evaluador-estudiante, las condiciones físicas, y la presión y responsabilidad del evaluador para dar una nota precisa fueron identificados como factores que afectan la consistencia del evaluador. Finalmente, se proporcionaron algunas implicaciones procedentes de esta investigación