Making research-based education more successful: Improving critical thinking and engagement through well-directed peer assessment

Universities increasingly recognize the need to train students using research-based education, using their discipline knowledge within group practical activities and to develop their critical thinking and teamwork skills to prepare them for their careers after graduation. With that in mind, students carry out substantial research-based projects many of which are in groups. These research-based projects may take the form of short labs, longer projects within a module during term time, or intensive one or two-week long full-time projects. In these cases, students may work together in disciplinary or multidisciplinary teams. In addition, some MEng students have a group project in the 3rd or 4th year of their degree that traditionally accounts for an equivalent of 2 taught modules. Despite the well-accepted educational benefits of getting students to work in research-based activities and in teams, some issues can detract from the student experience, i.e. (i) critical thinking skills are needed but difficult to obtain; and (ii) dissatisfaction with the assessment of group work. This paper presents work aimed at overcoming these two issues. Acquiring critical thinking is challenging and requires practice. Academic staff should implement long-term approaches to facilitate it. Introducing students to the critical analysis of someone else’s work early on in their degree programme is an excellent way of developing critical skills. We have incorporated this via peer assessment activities (e.g. of a report, a set of calculations, etc.) that initiates students in reviewing and constructively criticizing peers’ work. This stretches them because assessing a piece of work can be harder than completing the work itself, requiring a deeper understanding of the material and of alternative approaches. However, there are problems with traditional peer assessment which include (i) student disengagement leading to provide poor feedback to their peers, and (ii) students lacking confidence in their peers’ marking skills, and therefore the mark obtained. We have developed and successfully run for the past 3 years a new method of peer assessment (360 degree peer assessment) that addresses these main two issues, providing a better experience for students, and a successful tool for academics to foster and support the students’ critical thinking development. In the simplest way of assessing group work, the project deliverable (e.g. a report, a prototype, a video) is assessed and all members of the team would receive the same mark regardless of their individual effort. This leads to various problems: (i) dysfunctional behaviour and uneven participation, with some students not contributing their share; and (ii) frustration of high-performing students who do not see their work as being recompensed. Often, the mark will include an individual component, but it is either based on a separate piece of work (not practical to mark neither encouraging students into the group spirit), or they are set by the tutor based on some criteria considering the attitude of the individual in the group with just partial information. Alternatively, various practitioners have started to include an element of individual peer assessed contribution (IPAC) to team work. With this approach, each student in the group receives a mark based both on the overall “group mark” but also on the individual’s personal contribution towards the final product. This contribution is assessed directly by their peers, who are more aware of each team member’s contribution, and encourages self-reflection. However, the IPAC factor needs to be carefully assessed and applied. Following some initial work on the field, Pilar Garcia-Souto set up the IPAC Consortium whose ultimate goal is to “Identify a method for peer assessment of individual contribution in group work, develop or obtain an appropriate tool to implement it, and disseminate these across UCL and beyond; showing how to make the practice successful and efficient.” This consortium is currently formed by 40 members of staff from over 20 departments across UCL, and includes teaching staff in a range of fields (biomedical engineering, mechanical engineering, electrical engineering, physics, management, archaeology, architecture, culture, etc.), educational researchers (e.g. the Centre for Advanced Teaching and Learning, and the Institute of Education), and support staff (e.g. from the Digital Education and e-learning environment department). In this paper we will talk of our achievements so far and make recommendations for practitioners. In summary, this paper explores how a well-thought peer assessment method can aid students to develop critical thinking skills and allow academics to address group work assessment concerns, such that Research-based Education is more successful. Our approach is scalable and should appeal to anyone interested on incorporating or updating research-based education activities, regardless if you are designing a small activity within a module or a full programme of studies

Trajectories of charged particles trapped in Earth's magnetic field

I outline the theory of relativistic charged-particle motion in the magnetosphere in a way suitable for undergraduate courses. I discuss particle and guiding center motion, derive the three adiabatic invariants associated with them, and present particle trajectories in a dipolar field. I provide twelve computational exercises that can be used as classroom assignments or for self-study. Two of the exercises, drift-shell bifurcation and Speiser orbits, are adapted from active magnetospheric research. The Python code provided in the supplement can be used to replicate the trajectories and can be easily extended for different field geometries.Comment: 10 pages, 7 figures. Submitted to American Journal of Physic

Co-ordinating assessment across a programme

Assessment within a degree programme is critical for providing summative grades and formative feedback on specific pieces of work. Incorporating different forms of assessment into a programme provides students with opportunities to develop a wide range of skills beyond core disciplinary knowledge. Examples include research-based assessment, outward facing assessments aimed at different audiences, and authentic assessment linked to professional workplace practice. The opportunity to develop relevant professional skills is particularly important in an accredited engineering programme. The modular approach to programme development, prevalent in the UK, where different modules are often developed autonomously and assessed independently can make it difficult to introduce broader, creative assessment practices and can lead to heavy reliance on one method. For example, all module organisers might feel that their module is best assessed through written reports. Even if this is the most appropriate assessment mechanism for each individual module, we postulate that over the programme as a whole, students might learn more if they are required to submit a range of different types of outputs. By spreading this assessment portfolio across modules, we can develop and test a wider range of skills even while reducing the total assessment load. We will give examples from a programme that uses a combination of traditional assessments, authentic workplace-like assessments, research-based assessments, and assessment for different audiences. The paper examines the individual module compromises which may need to be made if assessment is to be seen holistically, to create programme-wide balance to maximise student development

360° peer assessment: improving reliability and engagement

A new more rigorous method for peer-assessment was designed and trialled, aiming to improve the student’s learning experience at UCL and IOE. The “360 degree peer assessment” process involves peers firstly marking and giving feedback to pieces of work and secondly being assessed by the recipients of the feedback on the quality of their marking. The peer marking is anonymous. Tutors monitor the process, but only need to intervene and moderate marks if the recipient disputes the mark and feedback, or if there is disparity of marks for one given piece of work. This approach was trialled with two undergraduate Biomedical engineering modules over four assignments and with one postgraduate education module. The benefits we found for this method include formalising the process for dealing with disputes in peer assessment and improving reliability. Undergraduate students also reported dedicating more time to peer assessment, they learnt more about the assessment process and were motivated to read feedback. The postgraduate students were motivated to undertake peer assessment but remained concerned about reliability and they did not appreciate that teacher moderation would occur when needed. The system is still under development. We aim to develop this methodology and use it increasingly in other modules and disciplines, explore for which types of coursework this approach is most suitable, and assess impact on student and staff workload. The authors want to thank to the IOE/UCL Strategic Partnership Teaching and Learning Fund and the Medical Physics and Biomedical Engineering Department UCL who partially funded this project

Moderated peer assessment of individual contribution to group work

UCL Engineering trains students to use engineering knowledge within extended group practical activities to prepare them for their careers after graduation. However, despite the substantial educational benefits of getting students to work in teams, providing individual assessment can be challenging. Students frequently express dissatisfaction if all members of a team are given the same mark regardless of the individual effort. Here, we aim to promote student engagement and improve student experience during group work by giving each student an individual mark. The individual mark results from multiplying the overall “group mark” by a personal contribution factor. This personal contribution is assessed directly by peers, who are aware of each team member’s contribution, encouraging self-reflection, and moderated by tutors when necessary. This practice has been well received by students in other universities. We are working with a student committee to identify and evaluate various methods and e-learning systems that would aid us to run this practice efficiently even for large numbers of students. This includes rules to flag cases requiring moderation. This project, partially funded by ELDG 2015, fits with our aim of increasing students’ satisfaction and engagement with assessment. We have combined it with our ‘360 degrees peer assessment method’, which we presented at last year’s conference, to provide a reliable and individual peer assessment of group work. We provide a novel approach to group assessment which encourages self-reflection and is intended to improve the learning experience and student satisfaction during group work, in line with UCL 2034

360 degree peer assessment to train engineering students in giving good quality feedback

Engineering degrees need to incorporate activities to develop the students' skills and confidence in constructing quality feedback, and ability to critically analyse someone else's work. These skills are highly linked with what industry expects from graduates, and implicit requirements to gain accreditation from UK professional bodies such as The Institution of Engineering and Technology. This paper reports how a novel method of peer assessment called 360 degrees peer assessment (360PA) was used to train students to give good and insightful feedback to a piece of work, while addressing some of the traditional peer assessment limitations. 360PA was successfully applied to a variety of typical engineering assignments (technical reports, research dissertations, presentations and mathematical problems). Students and staff's quantitative and qualitative feedback were collected. Our experience suggests that the incorporation of various 360PA assignments during the degree is beneficial. Staff praise the method, students feel that 360PA has better prepared them to construct feedback (score 4.0/5), and the quality of the feedback provided by the students is consistently high (∼85 ± 5%). Recommendations for practice are given. Our approach is scalable and should appeal to anyone interested in improving students' engagement with their feedback, or in helping students to develop such critical skills, regardless of class size

Microscopic Model of Charge Carrier Transfer in Complex Media

We present a microscopic model of a charge carrier transfer under an action of a constant electric field in a complex medium. Generalizing previous theoretical approaches, we model the dynamical environment hindering the carrier motion by dynamic percolation, i.e., as a medium comprising particles which move randomly on a simple cubic lattice, constrained by hard-core exclusion, and may spontaneously annihilate and re-appear at some prescribed rates. We determine analytically the density profiles of the "environment" particles, as seen from the stationary moving charge carrier, and calculate its terminal velocity as the function of the applied field and other system parameters. We realize that for sufficiently small external fields the force exerted on the carrier by the "environment" particles shows a viscous-like behavior and define an analog of the Stokes formula for such dynamic percolative environments. The corresponding friction coefficient is also derived.Comment: appearing in Chem. Phys. Special Issue on Molecular Charge Transfer in Condensed Media - from Physics and Chemistry to Biology and Nano-Engineering, edited by A.Kornyshev (Imperial College London), M.Newton (Brookhaven Natl Lab) and J.Ulstrup (Technical University of Denmark

Improving students' critical thinking and communication skills

University engineering faculties, professional engineering institutions and industry increasingly recognize that higher education should support students to develop key professional skills such as critical thinking and communication skills. This paper examines three activities aimed at teaching these skills, i.e. practical open-ended group activities; discussion with experts or as experts; and peer assessment. These methods were assessed in terms of student and staff opinions, but also practicality. Our research indicates that it is beneficial to integrate and balance these three types of activities within engineering degrees as they complement each other. Our findings and conclusions can be applicable to any engineering degree, whether the aim is to incorporate the teaching of these skills in a small activity within a module or a full programme of studies

Identification of patients who will not achieve seizure remission within 5 years on AEDs

Objective: To identify people with epilepsy who will not achieve a 12-month seizure remission within 5 vyears of starting treatment. Methods: The Standard and New Antiepileptic Drug (SANAD) study is the largest prospective study in patients with epilepsy to date. We applied a recently developed multivariable approach to the SANAD dataset that takes into account not only baseline covariates describing a patient’s history before diagnosis but also follow-up data as predictor variables. Results: Changes in number of seizures and treatment history were the most informative timedependent predictors and were associated with history of neurologic insult, epilepsy type, age at start of treatment, sex, and having a first-degree relative with epilepsy. Our model classified 95% of patients. Of those classified, 95% of patients observed not to achieve remission at 5 years were correctly classified (95% confidence interval [CI] 89.5%–100%), with 51% identified by 3 years and 90% within 4 years of follow-up. Ninety-seven percent (95% CI 93.3%–98.8%) of patients observed to achieve a remission within 5 years were correctly classified. Of those predicted not to achieve remission, 76% (95% CI 58.5%–88.2%) truly did not achieve remission (positive predictive value). The predictive model achieved similar accuracy levels via external validation in 2 independent United Kingdom–based datasets. Conclusion: Our approach generates up-to-date predictions of the patient’s risk of not achieving seizure remission whenever new clinical information becomes available that could influence patient counseling and management decisions