Assessing the quality of peer feedback in an online peer learning community

A goal of higher education is the development of students’ evaluative judgment, that is, the capacity to make judgements of the quality of work done by oneself and others.  Over-reliance on judgements of teachers, for example by focusing on grades, can inhibit students developing this ability. Peer review is one approach that can develop evaluative judgement but an issue that may prevent its implementation is the perceived quality of peer feedback. We used an online platform, PeerWise, in a second year genetics course to investigate the quality of peer feedback and whether evaluative judgment was evident. PeerWise enables students to write, comment on, answer and rate multiple choice questions. We found it necessary to allocate some marks as an incentive for participation but 5% of the course grade resulted in high engagement with minimal intervention from the convenor. This set-up allowed students to make and share judgements anonymously in a low-stakes context. Maximum marks were awarded for writing four accurate questions, covering at least three of the four content modules, and submitting four meaningful comments. This resulted in 959 questions and 1529 comments from 248 students. A qualitative analysis of all comments led to the identification of five criteria used to assess questions: knowledge, clarity, complexity, engagement and explanation. Students demonstrated the ability to make complex and reasonable judgements about question quality invoking multiple criteria. They made constructive suggestions for improvement and reflected on their own learning, consistent with the development of evaluative judgement.&nbsp

A Feasibility Study of a Co-adapted Mindfulness-based Intervention for Adolescents aged 13-16 in Guwahati, India.

With mental health problems including suicide for adolescent in India steadily increasing, the need for preventive action is urgent. There are major knowledge gaps about what works, at scale, to prevent poor mental health and improve adolescent wellbeing, in low resource settings. One approach is school-based prevention programmes, built on evidence-based practices such as mindfulness. Mindfulness-based interventions (MBIs) have been introduced in schools globally, including in India, as a promising way to improve adolescent mental health and wellbeing. Most MBIs for adolescents have been developed in Western settings. The World Health Organisation (WHO) recommends that where research and development capacity is limited, evidence-based programmes developed in one context or country could be adapted for delivery in another. This doctoral work engaged local stakeholders to adapt, and feasibility test an MBI originally developed for adolescents in the UK. The study aimed to discover if the MBI could be adapted into a suitable and acceptable school-based prevention intervention for adolescents (13-16y) in Assam. A partnership was established with MIND India, an Institute of Positive Mental Health & Research. The doctorate delivered three sequential studies, based on the exploration (Study 1), preparation (Study 2), and implementation (Study 3) phases for intervention development and testing. Study 1 was a consultation with n=14 in-country experts on adolescent mental health needs in India and on the co-adaptation, design and feasibility testing of the intervention. Study 2 co-adapted an MBI and other intervention components with n=19 adolescent end users (aged 13-18y). Study 3 determined the feasibility of the co-adapted MBI (n= 39 adolescents) to inform whether progression to a controlled trial of effectiveness was warranted. The interventions’ primary outcome was wellbeing, and a number of secondary outcomes were tested. A theory of change and logic model were produced. Study 1 findings endorsed a school-based universal prevention approach, the use of mindfulness and engaging with adolescent young people in co-adaptation research. Study 2 indicated the strong appetite that young people in Guwahati had for school wellbeing programmes and their knowledge on what is needed to support the wellbeing of people their age. Findings indicated an openness to mindfulness, and that solutions to cope with difficult times was a priority. Collecting data about mental health was also acceptable to them and their guardians. Acceptance of all core components of the proposed MBI was established meaning that only surface changes were required to assure culturally appropriate language and activities. Study 3 demonstrated that the co-adapted MBI met feasibility thresholds with 77% of participants attending at least 5 out of 8 sessions, and 95% of participants who completed the post-intervention evaluation recommending the programme to their peers. The study indicated that progression to an effectiveness trial is warranted and highlights the advantages of continued partnership with adolescents in India to promote acceptable community prevention approaches for wellbeing. Overall, the thesis has contributed new knowledge to meaningful participation in wellbeing programme development by adolescents in India and has aided the prevention agenda by establishing the feasibility of a community-based approach. Findings extend knowledge about mindfulness approaches with adolescents in India, including via school-based delivery

Examination of the interaction between parental military-status and race among non-Hispanic Black and non-Hispanic White adolescents with overweight/obesity

OBJECTIVES: Adolescent military-dependents experience distinct risk and protective factors, which may necessitate additional clinical considerations. In civilian youth, overweight/obesity is associated with eating, internalizing, and externalizing difficulties, with some studies reporting more difficulties among non-Hispanic White (vs. non-Hispanic Black) youth. It is unknown if these disparities exist among adolescent military-dependents, or between civilian and military-dependent youth. METHODS: Non-Hispanic Black (187 civilian, 38 military-dependent) and non-Hispanic White (205 civilian, 84 military-dependent) adolescents with overweight/obesity (14.7 ± 1.6 years; 73.9% girls; body mass index adjusted for age and sex 1.9 ± 0.5) completed a disordered-eating interview; parents completed a measure assessing their child\u27s internalizing and externalizing difficulties. Multiple linear regressions examined parental military-status as a moderator of the relationship of participant race with eating, internalizing, and externalizing difficulties. RESULTS: White civilian youth with overweight/obesity reported significantly greater disordered-eating than their Black peers (p \u3c .001); there were no other significant racial differences. In all regressions, parental military-status significantly moderated the association between race and each dependent variable (ps \u3c .047). Black military-dependents (vs. civilians) reported more disordered-eating and internalizing difficulties (ps = .01). White military-dependents (vs. civilians) reported fewer externalizing difficulties (p = .01). CONCLUSIONS: Black adolescent military-dependents with overweight/obesity may experience more eating and internalizing difficulties (vs. civilians), a pattern not observed among White participants. Future work should examine if being a military-dependent and a historically marginalized racial group member accounts for these findings. Such data may inform providers of youth with intersecting minority identities

Standards and learning outcomes for undergraduate research projects

Undergraduate research projects are increasingly being incorporated as assessed components, into a range of degrees, both those aimed at elite students and in standard BSc degrees. While students and staff are generally positive about such experiences, the literature shows that students report a wide range of learning outcomes and that the student experience is very dependent on the supervisor. This immediately raises issues of equity and standards; each research project is unique and how each one compares to other research projects or normal courses can be problematic. In principle, clearly articulated learning outcomes can help implement benchmarking or moderation processes; however if these learning outcomes focus on advances in disciplinary knowledge, methods and skills, comparison across sub-disciplines (let alone disciplines) remains hard. We propose an alternative, complementary focus for assessing learning: the development of students’ understanding of the processes and practice of science, together with generic skills such as critical thinking, elements which should be developed in all projects

Factors affecting paramedicine students' learning about evidence-based practice: a phenomenographic study

Background: Evidence-based practice is an important component of pre-service professional learning in medicine and allied health degrees, including new programmes in paramedicine. Despite substantial interest in this area, there is still a lack of clear understanding of how the skills and understandings needed to develop the capacity to apply evidence-based practice can best be learned. Evidence-based practice is often described as consisting of five steps: ask, acquire, appraise, apply and assess. This study focuses on paramedicine students’ learning about the first three steps in a final year unit which explicitly aims to develop their skills in relation to these. Methods: We conducted a qualitative study of learning journals recorded by 101 of 121 students in a final year unit of a paramedicine degree (20 students either withheld consent for their journals to be used in the research or did not complete their journal entries). We used phenomenographic approaches to the data analysis in order to identify both variation in students’ learning and the factors affecting this variation. Results: We observed variation in students’ understanding of the purpose of literature analysis, the nature of medical research and its relationship to practice. In all three, we identify two main factors contributing to the variation in student learning outcomes: epistemological stance, and opportunities for metacognitive learning generated through peer interactions and self-reflection. We also found that as students begin to grapple with the complexity of medical research, this sometimes produced negative attitudes towards its value; such unintended outcomes need to be recognised and addressed. Conclusions: We suggest key factors that should be considered in developing coursework intended to enhance students’ understandings about the processes and application of evidence-based practice. Providing collaborative learning opportunities that address the architecture of variation we observed may be useful in overcoming epistemological and metacognitive barriers experienced by students

Recognising the development of expert practice in undergraduate research: a TREASURE project update

The difference between novice and expert researchers lies in much more than possession of explicit, discipline-based knowledge and technical expertise. The transition to expert practice involves the contextualised internalisation of this knowledge and understanding, often through a process of deliberate and extensive practice. Acquisition of such tacit knowledge is coupled with the development of attitudes such as self-confidence, pragmatism, criticality and comfort with uncertainty, leading to a capacity to make (often unconscious) expert judgments. While most teaching and assessment during an undergraduate science degree focuses firmly on declarative (and declared) knowledge, undergraduate research projects can provide the first steps in the transition to expert practice. However, we believe that more could be done to make these experiences as effective as possible in developing the knowledge and attributes described above. The TREASURE project aims to help students recognise their transition to expert-like behaviour by asking them to reflect on their research project throughout the semester. This emphasis on process and practice may also help reduce the student’s focus on results or research products as the sole indicator of success and provide useful input to supervisors about student thinking. Results from TREASURE’s first year will be presented

Do undergraduate research experiences influence student understanding of the uncertainties inherent in scientific knowledge?

Uncertainty is an inescapable facet of life and as our world becomes more complex, the levels of uncertainty also rise. This is the so-called supercomplex world as described by Barnett (2007), a world in which science based disciplines are well placed to deal with these complexities. However, it is hard to argue that current science education pedagogies are preparing our future graduates to become effective advocates for science in this most certainly, uncertain world. Previous studies of graduate students by Perry in the 1960s and Schommer in the 1990s provided evidence that as students progress they start to see knowledge as less deterministic, more uncertain and complex as well as crossing thresholds of understanding (Ross et al., 2010). Recently an OLT project grant was awarded to the Australian National University, University of Western Sydney and Canberra University with the title Teaching Research Evaluation and Assessment Strategies for Undergraduate Research Experiences” (TREASURE). This project evaluated student thinking around research experiences in both a final semester capstone unit as well as independent research units. Over 60 students kept a reflective blog of their learning in these research experiences. It was encouraging to observe that the entire cohort demonstrated appropriate usage of standard scientific research processes. However, few of the students in the capstone unit were able to extend their understanding to encompass the concept that science knowledge and processes can be uncertain, complex and indeterminate. This is of concern in the context of the newly articulated Threshold Learning Outcomes in Science (TLOs) “Demonstrate a coherent understanding of biomedical science by articulating the methods of science and explaining why current scientific knowledge is both contestable and testable by further inquiry.” This observation suggests that we may be overestimating our expectations of the level of learning engendered by the current undergraduate research experiences. This in turn raises the question of whether we are satisfied that many of our graduates do not attain this threshold standard? Perhaps the time has come to develop a formal pedagogy of uncertainty