A FOCUS ON CONTENT: THE USE OF RUBRICS IN PEER REVIEW TO GUIDE STUDENTS AND INSTRUCTORS

Students who are solving open-ended problems would benefit from formative assessment, i.e., from receiving helpful feedback and from having an instructor who is informed about their level of performance. Open-ended problems challenge existing assessment techniques. For example, such problems may have reasonable alternative solutions, or conflicting objectives. Analyses of open-ended problems are often presented as free-form text since they require arguments and justifications for one solution over others, and students may differ in how they frame the problems according to their knowledge, beliefs and attitudes.This dissertation investigates how peer review may be used for formative assessment. Computer-Supported Peer Review in Education, a technology whose use is growing, has been shown to provide accurate summative assessment of student work, and peer feedback can indeed be helpful to students. A peer review process depends on the rubric that students use to assess and give feedback to each other. However, it is unclear how a rubric should be structured to produce feedback that is helpful to the student and at the same time to yield information that could be summarized for the instructor.The dissertation reports a study in which students wrote individual analyses of an open-ended legal problem, and then exchanged feedback using Comrade, a web application for peer review. The study compared two conditions: some students used a rubric that was relevant to legal argument in general (the domain-relevant rubric), while others used a rubric that addressed the conceptual issues embedded in the open-ended problem (the problem-specific rubric).While both rubric types yield peer ratings of student work that approximate the instructor's scores, feedback elicited by the domain-relevant rubric was redundant across its dimensions. On the contrary, peer ratings elicited by the problem-specific rubric distinguished among its dimensions. Hierarchical Bayesian models showed that ratings from both rubrics can be fit by pooling information across students, but only problem-specific ratings are fit better given information about distinct rubric dimensions

Peer-review in practice: eight years of Aropä

Articles discussing and analysing student peer-review activities proliferate the educational literature, typically describing one or more class exercises where students provide feedback on each other’s work. These papers usually focus on a peer-review activity designed as a scholarly study, and make conclusions about its success or otherwise. There is not one standard model for ‘peer-review’, and information on the many different assessment designs used is distributed over an increasing number of publications and websites. This paper provides a meta-review of peer-review activities as they are implemented in practice, using configuration data from over a thousand assignments conducted using an online peer-review system during an eight-year period. We present data on the wide variety of assignment designs and the parameters that comprise them, their rubrics, and comparisons between subject areas. Information on the norms and range of all decisions to be made will encourage instructors (both new to and experienced in conducting peer-review activities) to reflect on and justify the choices they make

Rubrics and Revision: What are the Effects of 3\u3csup\u3erd\u3c/sup\u3e Graders Using Rubrics to Self-Assess or Peer-Assess Drafts of Writing?

The purpose of this quasi-experimental research thesis was to determine the effects of rubric-referenced peer-revision and self-assessment on the writing drafts of 3rd grade students. A convenience sample of students in existing classrooms engaged in two persuasive writing assignments. The first assignment established a baseline score for comparison purposes. During the second assignment, a peer-revision group and self-assessment group received different interventions that focused on revision guided by a rubric. A third control group did not receive an intervention. Student opinions toward the usefulness of the treatments were also gathered through a questionnaire that was delivered after the writing assignments were complete. The utilization of rubrics to assist peers in revision had a statistically significant, positive effect on student scores during the second persuasive writing assignment. The treatment of rubric-referenced self-assessment did not have an overall positive effect on student scores during the second assignment. The control group’s scores decreased slightly on the second assignment. Almost every student in the peer-revision group thought the treatment was beneficial for student writing. In comparison, a little more than half the students in the self-assessment group considered the treatment to be useful in helping them achieve higher scores or become more proficient writers

Informing Writing: The Benefits of Formative Assessment

Examines whether classroom-based formative writing assessment - designed to provide students with feedback and modified instruction as needed - improves student writing and how teachers can improve such assessment. Suggests best practices

Examining Formative Critique In The High School Visual Arts Classroom

Formative assessment techniques are integral to high school visual arts teachers’ curricula, but are not clearly delineated by state and national organizations. Additionally, formative assessment in the high school visual arts classroom, defined in this study as formative critique, had not been examined as extensively, and most research investigated either high school core content practices or critique procedures in higher education settings. This qualitative phenomenological study sought to understand commonalities in the implementation of formative critique by high school visual arts teachers in central Kentucky. In this study, six veteran high school visual arts teachers were interviewed, using open-ended questions and a conversational approach. All interviews were conducted via video conference and digital materials used by participants were collected for analysis. This study found that despite a lack of pedagogical guidance on the use of formative critique from art education organizations, participants’ approaches were remarkably similar. Participants favored an individualized, conversational approach, and used questions to guide student work. Teachers in this study built relationships with their students, creating a supportive classroom atmosphere which lead to positive experiences in visual arts courses. According to participants, formative critique led to student growth and the production of more advanced artwork. Interviewees indicated that students were more willing to take risks and put forth effort when formative critique was used to build constructive environments. Participants indicated that formative critique is used for positive daily interactions with students and focuses on artistic processes. Additionally, summative assessment could be viewed in a formative context under certain circumstances. These findings could be instructive for policy implementation when designing visual arts standards for high school classrooms and could be used to guide administrators’ assessment of teacher practice. Finally, limitations and suggestions for further research are presented

Assessment for Experiential Learning

Chan’s book explores the challenges in assessing experiential learning, deepens our understanding, and inspires readers to think critically about the purpose of assessment in experiential learning. Experiential learning has been studied and proven to be effective for student learning, particularly for the development of holistic competencies (i.e. 21st century skills, soft skills, transferable skills) considered essential for individuals to succeed in the increasingly global and technology-infused 21st century society. Universities around the world are now actively organising experiential learning activities or programmes for students to gain enriching and diversified learning experiences, however the assessment of these programmes tends to be limited, unclear, and contested. Assessment plays a central role in education policies and students’ approach to learning. But do educators know how to assess less traditional learning such as service learning, entrepreneurship, cross-discipline or cross-cultural projects, internships and student exchanges? While the current assessment landscape is replete with assessments that measure knowledge of core content areas such as mathematics, law, languages, science and social studies, there is a lack of assessments and research that focus on holistic competencies. How do we assess students’ ability to think critically, problem solve, adapt, self-manage and collaborate? Central to the discussion in this book, is the reason students are assessed and how they should be assessed to bring out their best learning outcomes.　Offering a collection of best assessment practice employed by teachers around the world, this volume brings together both theoretical and empirical research that underpins assessment; and perceptions of different stakeholders – understanding of assessment in experiential learning from students, teachers, and policymakers. The idea of assessment literacy also plays an important role in experiential learning, for example, reflection is often used in assessing students in experiential learning but how reflection literate are educators, are they aware of the ethical dilemmas that arise in assessing students? These questions are discussed in detail. The volume also introduces a quality assurance programme to recognise student development within experiential learning programmes. The book will be particularly informative to academic developers, teachers, students and community partners who struggle with the development and assessment for experiential learning, those who plan to apply for funding in experiential learning, and policymakers and senior managements seeking evidence and advice on fine-tuning curricular, assessment designs and quality assurance

A Rubric to Evaluate and Enhance Requirements Elicitation Interviewing Skills

Eliciting effective requirements is vital for successful Information Systems development and implementation. Interviews with stakeholders and users are an important part of the requirements elicitation process. Thus, teaching students how to better perform requirements elicitation interviews is a critical task for information systems faculty. However, prior to this research, a common tool or rubric to evaluate the effectiveness of requirements elicitation interviews was not found in the literature. The purpose of this research was to develop a rubric that can be used to both evaluate (provide summative measures) and enhance (via formative training techniques) the requirements elicitation interviewing skills of information systems students. The results of this research provide both quantitative and qualitative evidence that the rubric developed and described in this paper substantially improved the ability of our students to conduct requirements elicitation interviews. Along with detailing the various methodologies we used, this paper provides practical pedagogical suggestions and lessons learned along with covering possible future avenues of research in this area