Development and Uses of Upper-division Conceptual Assessment

The use of validated conceptual assessments alongside more standard course exams has become standard practice for the introductory courses in many physics departments. These assessments provide a more standard measure of certain learning goals, allowing for comparisons of student learning across instructors, semesters, and institutions. Researchers at the University of Colorado Boulder have developed several similar assessments designed to target the more advanced physics content of upper-division classical mechanics, electrostatics, quantum mechanics, and electrodynamics. Here, we synthesize the existing research on our upper-division assessments and discuss some of the barriers and challenges associated with developing, validating, and implementing these assessments as well as some of the strategies we have used to overcome these barriers.Comment: 12 pages, 5 figures, submitted to the Phys. Rev. ST - PER Focused collection on Upper-division PE

Constructing and Applying Rubrics in College-Level EFL Writing Assessment in China

Assessment is a critical component in the teaching of writing and plays an important role in discovering and helping to address students’ writing difficulties. Therefore, it is essential for teachers to approach writing assessment in a reliable and valid way. Previous studies showed that assessment rubrics, used as a standard to describe performance evaluation, can help teachers effectively assess student writing. The reliability and validity of the use of rubrics for helping teachers in assessing writing fairly and improving students’ writing ability has received much research attention; however, less attention has been paid to teacher training in the area of rubric construction and use. This study examined the use of and attitudes towards rubrics in college level EFL writing classes in China and the effects of teacher training in the design and application of rubrics. The data for this study were drawn from Chinese college-level English teachers as well as first- and second-year undergraduate students from a medium-sized university in China, targeting the following: (1) investigating the current writing assessment method adopted in college EFL classes in a university in China and the current situation of teacher training, examined through pre-study questionnaires, (2) conducting teacher training on rubric construction and development and examining the implementation of training through two student writing assignments, (3) examining the students’ and teachers’ attitudes towards rubric use and teachers’ attitudes towards training through post-study questionnaires. Analysis of the pre-study questionnaires indicated that, in at least the target Chinese institution, rubrics were not widely used by the college-level English teachers, and their primary focus in writing assessment was on grammatical and vocabulary errors. In addition, it appeared that in this context, the English teachers did not receive adequate training concerning the use of rubrics for writing assessment. Data collected during teacher training and application showed that after the implementation of teacher-designed rubrics as assessment tools, students’ overall and subentry writing scoring improved, though no causal relationships can be claimed. Analysis of the post-study questionnaires indicated that both students and teachers held positive attitudes towards rubrics, which they perceived as being helpful in improving writing proficiency and gaining confidence in writing. In addition, teachers also had a positive attitude towards the training. This thesis differs from previous research by combining teacher training and rubric implementation, with the focus both on teachers and students rather than only on one side. Results support the effectiveness of both rubrics in assessment and teacher training in rubric construction and use

When Your Plate is Already Full: Efficient and Meaningful Outcomes Assessment for Busy Law Schools

The American Bar Association (ABA) accreditation standards involving outcome-based assessment are a game changer for legal education. The standards reaffirm the importance of providing students with formative feedback throughout their course of study to assess and improve student learning. The standards also require law schools to evaluate their effectiveness, and to do so from the perspective of student performance within the institution’s program of study. The relevant question is no longer what are law schools teaching their students, but instead, what are students learning from law schools in terms of the knowledge, skills, and values that are essential for those entering the legal profession. In other words, law schools must shift their assessment focus from one centered around inputs to one based on student outputs. Compliance with the ABA’s assessment mandate comes at a time when law school resources are spread thinner than ever. Indeed, faculty already work with plates that are full with students, scholarship, and service. Thus, while not all in the legal academy are on board with the ABA’s approach to outcomes assessment or to outcomes assessment generally, as busy educators, we should all at least agree that the requisite response should be efficient, given that resources are limited, and meaningful, such that the work done can benefit our learners. To do so, law schools should begin at their own tables set with full plates, so to speak, taking stock of what institutions and their faculty are already doing in terms of assessment. And it is important to think broadly here, as faculty may be surprised to learn how many of their colleagues are already doing relevant work. While law schools may already be inclined to begin from within, this Article outlines concrete strategies they can use when working with existing faculty expertise and resources to respond to the ABA’s assessment mandate in a meaningful way for students, and with the goal of maximizing efficiency and gaining broad buy in. While prior scholarship has outlined best practices for outcomes assessment and even shared examples of how to engage in the process in the law school setting, this Article is unique in its depth and breadth of coverage by setting out a detailed case study that illustrates the process of developing an authentic assessment tool and beginning the process for adapting that tool to respond to both the individual student assessment and law school assessment required by the ABA

A robust methodology for automated essay grading

None of the available automated essay grading systems can be used to grade essays according to the National Assessment Program – Literacy and Numeracy (NAPLAN) analytic scoring rubric used in Australia. This thesis is a humble effort to address this limitation. The objective of this thesis is to develop a robust methodology for automatically grading essays based on the NAPLAN rubric by using heuristics and rules based on English language and neural network modelling

TMFD 144-Intro to Design Theory and Criticism: A Peer Review of Teaching Benchmark Portfolio

In this benchmark teaching portfolio, I examine TMFD 144: Intro to Design Theory and Criticism. It is offered every semester. The course focuses on visual literacy in the art and design fields. The overarching course goal is to teach students to actively “see” and interpret the visual world. Students should also learn to communicate clearly and effectively about the visual aspects of art and design as well as their interpretation. This course is a required foundation course for all TMFD majors and minors. It is also an ACE 2 course that enrolls students from across the university. The course is taught entirely online through Canvas. For the portfolio I examined trends in scores on an assignment that students revised three times over the semester. Students wrote 2 short essays on 2 different paintings. The assignment is intended to demonstrate student competency and the internalization of course materials. The findings indicated an average 4% improvement over the semester but when drilling down there was high variability in students scores in both the midterm and final essays as indicated by high standard deviations. The assignment has potential to be an excellent assessment but it needs modified to accommodate the diverse knowledge and skill set that an ACE course includes. I make suggestions to adjust the criteria, provide more structure in the assignment and to more clearly connect for students, the assignment to course goals and weekly workloads

Action Research in Bridging the Gap Between Theory and Practice: A Study of the Evaluation and Reporting of Student Achievement

The theory and practice of evaluating student academic performance has been a source of concern in academia for over a century. The challenge of successfully implementing assessment practices that reflect the true measure of a student’s academic achievement, and that accurately and effectively communicate the student’s level of mastery to stakeholders, has not been met according to measurement specialists. This study was designed to examine the implementation process of research-based classroom assessment practices that both accurately measure the academic achievement of students and effectively communicate the students’ level of mastery. Interviews were conducted to examine the practicality of the assessment practices and whether the evidences gathered from these practices support performance grades that accurately articulate student achievement. The study showed that recommendations from measurement specialists are practical assessments for the classroom, accurately measure the academic achievement of students, and effectively communicate the students’ level of mastery. However, training in pre-service teacher programs that continue to be supported by in-service professional development is critical to the successful implementation of the recommendations, and to bridging the gap between theory and practice

Student outcomes and learning environments at the tertiary level in New Zealand: the develpment of an assessment framework

This thesis reports the findings of a case study into the outcomes and learning environments of students studying for a Diploma of Technology at a New Zealand Tertiary Education Institution. The primary focus of the study was to develop an assessment framework that would facilitate simultaneous assessment of the local Institute achievement-based Diploma of Technology and the competency-based assessment of a National Diploma that the local diploma subsumed. From this primary focus on assessment, the study provided the opportunity to undertake a grounded theory study of literature that impacted on the learning environment, supported the necessity to rationalise the student learning outcomes from both an academic and work skill perspective, and presented a suitable situation in which to take a fresh look at the method of grading and marking students’ assessment material.The grounded theory study grew out of the need to condense the vast amount of literature that was gathered in the process of searching for background material to use in the building of a foundation on which to construct a dual assessment model. Although no literature was found that specifically dealt with the simultaneous dual assessment, a large amount of material was found that related to various aspects of the learning environment. Through the process of a grounded theory study, this material was condensed into categories of data that in turn were used to develop a theoretical model of an ‘ideal’ learning environment. Into this model was also added the results of a questionnaire based research study into the perceived need for diploma graduates to have a range of employability skills. This study involved a range of employers who considered themselves likely to employ a diploma graduate. Because the learning outcomes for the two diplomas covered essentially the same material yet the actual wording of the outcomes were substantially quite different, there was a need to rationalise the sets of learning outcomes for each diploma. This process led to a common set of outcomes that in turn were used as a focus for students’ learning and assessment.Once these common outcomes had been identified, a rubric based marking/scoring system was developed so that both students and teacher could quickly grade students’ assessment material and then convert that grade into a mark. The use of the grade facilitated the assessment of achievement against a unit standard and the resultant marks satisfied the need for an achievement mark. The results and findings from the various studies were then translated into a working model that was used for two courses over one semester. Various other research methodologies were then used in order to provide some evaluation of the working model.The thesis does present some of the difficulties facing tertiary teachers in an environment that is becoming more and more of a production line business rather than a service to provide learning opportunities for students. However it also presents solid evidence that teachers can take measures to prove themselves through study and initiative and provide those focussed learning environments where students can attain the outcomes necessary for a successful career in tomorrow’s world