Assessing by design—constructing a marking guide

NEED Improving assessment is a key aim of the Australian Council of Deans of Science (ACDS). In 2023, it funded workshops to help improve assessment across the sciences, where science educators across Australia worked with assessment experts from the Australian Council of Educational Research and discipline leaders to learn how to write, review and undertake psychometric analysis of assessment items. Effective marking guides (rubrics) are an essential part of aligning assessment to learning outcomes, showing the performance expectations of an assessment, and demonstrating the objectivity of the assessment’s marking. Designing and developing marking guides (rubrics) is both an art and a skill. There is literature, but there is no replacement for experience and for adopting a design approach to constructing excellent marking guides. It is worth spending time on the design and writing of a marking guide as it will likely improve marking both through improved feedback to students, discrimination of performance, reliability and through increasing marking efficiency. WHAT WE WILL DO In this session we will work through three stages to develop skills at constructing marking guides: We will work together to identify the key determinants of a good marking guide. This discussion will be supported by consideration of the literature and of our collective experience. We will apply those determinants to working collaboratively to develop and refine a marking guide as a group. We will work in small groups to refine marking guides brought to the workshop by participants. WHO SHOULD ATTEND All are welcome to join us for this workshop. It follows on from the assessment workshops sponsored by the ACDS in 2023 and attended by over thirty academics around Australia. WHAT TO BRING We encourage participants to bring a marking guide that they would like to improve

The impact of teaching from home during the covid-19 pandemic on the student evaluations of female academics

Student Evaluation of Teaching (SET) results play an important role in academic staff performance evaluation, but also in promotion processes. However, there is much evidence to suggest that the SET used in most universities across the Anglosphere has traditionally penalised female academics. As universities manage the recovery phase of the COVID-19 pandemic, they will also need to take into account the effect of remote teaching on the validity of student evaluation data. Given SET are critical to promotion success, it is important to then understand the gendered effect of remote teaching on student evaluations. We aimed to evaluate how intrusions of family life, academics’ home environment and competence with remote teaching technology of female academics were viewed by students and if there were noticeable differences in SET data. We analysed 22,485 SET data over 2019 (pre-COVID, face-to-face teaching) and 2020 (COVID-lockdowns, remote teaching) for female and male academics, matched with student gender, in the multidisciplinary First Year College at Victoria University, Melbourne Australia. Our results showed that there were no differences in the score ratings for teacher gender. However, the qualitative data showed that whilst overall there were overwhelmingly positive comments for both male and female teachers, there was an increase in the negative comments on teaching style by male students toward their female teachers during remote teaching and overall more comments relating to attitude. We speculate that this would have a negative impact on the confidence of teaching-intensive female academics hindering their leadership aspirations and career progression in academia

The Effects of Hormonal Therapy and Exercise on Bone Turnover in Postmenopausal Women: A Randomised Double-Blind Pilot Study

Abstract Introduction: Hormone replacement therapy (HRT) and walking were investigated independently and in combination, to determine which treatment provided most effect on bone turnover in postmenopausal women. Methods: Using a randomised double-blind pilot study, 10 subjects received HRT (transdermal estradiol, 50 μg/day and oral MPA 5 mg/day) and 12 received placebo for 20 weeks. Following a baseline period of treatment, both groups undertook a graduated walking regimen, which increased in intensity, duration and frequency parameters from weeks 8–20. Measurements of aerobic capacity, female sex hormones, bone formation markers [osteocalcin (OC) and bone alkaline phosphatase (BAP)] and bone resorption markers [deoxypyridinoline (DPD) and pyridinoline (PYR)] were measured at baseline (T1), week 8 (T2) and week 20 (T3). Results: Age, time of postmenopause, weight or body mass index were no different between each groups. The HRT group had significantly higher estradiol levels compared with the placebo group at T2 and T3. FSH and LH levels were significantly reduced following HRT. DPD and PYR were significantly reduced from baseline levels at T2 and T3 with HRT. No significant changes occurred in OC or BAP levels with either HRT or walking. Walking did not change bone turnover markers in either the HRT or placebo group. Conclusion: HRT reduces bone resorption, however, walking alone at the intensity and duration prescribed, or the combination of HRT and walking, provided no additional benefit after menopause. Therefore, HRT, but not walking is an effective treatment in reducing bone turnover in postmenopause women.</jats:p

Methamphetamine: Effects on the brain, gut and immune system

Methamphetamine (METH) is a powerful central nervous system stimulant which elevates mood, alertness, energy levels and concentration in the short-term. However, chronic use and/or at higher doses METH use often results in psychosis, depression, delusions and violent behavior. METH was formerly used to treat conditions such as obesity and attention deficit hyperactivity disorder, but now is primarily used recreationally. Its addictive nature has led to METH abuse becoming a global problem. At a cellular level, METH exerts a myriad of effects on the central and peripheral nervous systems, immune system and the gastrointestinal system. Here we present how these effects might be linked and their potential contribution to the pathogenesis of neuropsychiatric disorders. In the long term, this pathway could be targeted therapeutically to protect people from the ill effects of METH use. This model of METH use may also provide insight into how gut, nervous and immune systems might break down in other conditions that may also benefit from therapeutic intervention

Women’s work: Women leading women in a teaching-focussed first year college

In 2018, a community of practice initiative was introduced to enhance the career opportunities for the 53 women academics in the First Year College (FYC) at Victoria University. The FYC is a teaching-oriented college and thus the traditional career advancement through research is largely not applicable to women in the college – many of whom are teaching-focussed academics. The Women in First Year (WiFY) executive developed and implemented workshops with the aim of aiding women on their career trajectory and in getting more women to senior levels and into leadership positions. In this article we will argue that the WiFY initiative has resulted in greater career achievements for women in the FYC. Using data collected from semi-structured interviews as well as relevant research and FYC statistics on career progression, we will argue that WiFY has had a positive effect on the career trajectory of women in the FYC who have participated in their workshops

Building the capacity of academics to assess higher order skills through improved assessment design

Across the sciences, high-stakes assessment (such as end-of-semester exams) often consist largely, if not solely, of multiple choice questions (MCQs). MCQs have advantages in large-scale testing, including automated marking and high reliability (Haladyna 2004). But in biomedical education, MCQs usually test knowledge recall (the lowest “Remembering” category in the cognitive domain of Blooms taxonomy). Psychometric analysis of MCQs exams in biochemistry and physiology from four universities has shown that they contained a substantial number of questions that were too easy for the cohort and lacked questions testing higher-order skills such as problem-solving, and knowledge analysis and application . It is possible to design MCQs that test higher-order cognitive processes—the UMAT and GAMSAT medical entrance exams routinely include MCQs which test higher-order cognitive skills. But to write questions of this standard requires skill and effort. The objectives of this workshop are: To build academic capability to design and write MCQ exams that test students’ capacity to solve problems and apply their knowledge/competencies in new contexts; and to improve the validity of assessment by adopting a rational assessment design approach, including mapping of questions against subject learning objectives

Development of transferable skills in a low-SES population through peer assisted study sessions

Background Student peer mentoring programs are generally implemented to improve student learning outcomes. Students from low-socioeconomic status (SES) backgrounds are typically under-represented in Australian higher education (Coates and Kraus, 2005). In general it is thought that these students may not be equipped with the skill-sets necessary to complete their academic studies (Kirk, 2008). At Victoria University in Melbourne, there is an increasing number of low SES students (`20%), with a significant proportion of mature aged (Bachelor of Health Sciences (Paramedicine)) and “first in family to University” from immigrant communities (Bachelor of Science (Biomedical Science)) students. Aims The aim of this study was to determine if these students undertaking Peer Assisted Study Sessions (PASS) in an anatomy and physiology unit also developed study skills, time management skills and confidence in their abilities to complete their studies. Description of intervention We introduced a voluntary learning support or academic assistance program (PASS) for first year Bachelor of Health Sciences (Paramedicine) and Bachelor of Science (Biomedical Science) students. Student mentors were selected from the previous year’s cohort, with mentors obtaining a grade of >80% in the same unit. Mentors completed a 2-day training program before the first session and attended weekly workshops throughout the semester. Student mentee numbers were capped at 50 per semester. Design and methods This study was approved by the Human Ethics Research Committee of Victoria University (HRETH 10/20). PASS involves 2nd or 3rd year Student Mentors facilitating weekly review sessions for groups of 1st year students in a specific unit of study focused on anatomy and physiology. Student Mentors use collaborative learning methods to assist students to better understand difficult concepts. Students enrolled in both degree programs voluntarily participated in the PASS scheme in 2011 and 2012. Evaluation surveys were distributed at week 12 of semester. The survey contained 10 questions with a Likert scale of 5, as well as 3 open questions. Results Students in both cohorts improved their final grade for this unit and reduced their failure rate if they participated in PASS. In addition, in both cohorts, students who attended PASS improved their confidence and believed the sessions provided them with important skills. Specifically, students believed that the PASS program helped them to develop skills to approach their other units of study in the degree, time management skills and the development of social networks. All students who participated in the PASS program strongly agreed that it had been a positive experience. Conclusions The PASS program employed at Victoria University produced favorable outcomes for both the Bachelor of Health Sciences (Paramedic) (Hryciw et al., 2013) and Bachelor of Science (Biomedical Science) students. Students in general believed that they had developed confidence in general, as well as necessary skills that could assist them in completing their degree. Future studies should focus on the students’ transferability of the skills learned in the PASS program in other subjects within their degree and whether this cohort had an increased completion rate in their overall degree program. References Coates, H. & Kraus, K. (2005). Investigating ten years of equity policy in Australian higher education. Journal of Higher Education Policy and Management, 27(1), 35-47 Hryciw DH, Tangalakis K, Supple B, Best G. (2013) Evaluation of a peer mentoring program for a mature cohort of first-year undergraduate paramedic students. Advances in Physiology Education. 37:80-4. Kirk, K (2008). Diversity and achievement: Developing the learning of non-traditional HE Students. In Crosling, G., Thomas, L., and Heagney, M. Improving student retention in higher education: The role of teaching and learning. Oxford: Routledge pp 150-9

Creative game-based scenario podcasts to improve the learning experience and outcomes for first-year physiology students

BACKGROUND Educational podcasts have the potential to be an excellent medium to promote student creativity and engagement for learning (Forbes, 2015). AIMS Evaluating the effectiveness of using a creative game-based scenario and learner-generated podcasts on student engagement and assessment performance. DESCRIPTION OF INTERVENTION Students are immersed into a creative game-based scenario and produce a team podcast related to one topic of physiology. Podcasts are then presented, with students participating in their assessment. DESIGN AND METHODS Intervention effectiveness was assessed by comparing test results for 2019 and 2018 (no podcast use). Qualitative data obtained for several constructs of interest, including satisfaction, engagement, learning and creativity have been sourced via surveys, questionnaires, and interviews. RESULTS Statistical analysis (t-tests