Short answer versus multiple choice examination questions for first year chemistry

Multiple choice (MC) examinations are frequently used for the summative assessment of large classes because of their ease of marking and their perceived objectivity. However, traditional MC formats usually lead to a surface approach to learning, and do not allow students to demonstrate the depth of their knowledge or understanding. For these reasons, we have trialled the incorporation of short answer (SA) questions into the final examination of two first year chemistry units, alongside MC questions. Students’ overall marks were expected to improve, because they were able to obtain partial marks for the SA questions. Although large differences in some individual students’ performance in the two sections of their examinations were observed, most students received a similar percentage mark for their MC as for their SA sections and the overall mean scores were unchanged. In-depth analysis of all responses to a specific question, which was used previously as a MC question and in a subsequent semester in SA format, indicates that the SA format can have weaknesses due to marking inconsistencies that are absent for MC questions. However, inclusion of SA questions improved student scores on the MC section in one examination, indicating that their inclusion may lead to different study habits and deeper learning. We conclude that questions asked in SA format must be carefully chosen in order to optimise the use of marking resources, both financial and human, and questions asked in MC format should be very carefully checked by people trained in writing MC questions. These results, in conjunction with an analysis of the different examination formats used in first year chemistry units, have shaped a recommendation on how to reliably and cost-effectively assess first year chemistry, while encouraging higher order learning outcomes

Diversity in Conceptions of Incoming Chemistry Students in the Context of Changing Syllabi

The lowering of entry requirements for several programs of study in science and health has resulted in a greater diversity in academic ability amongst students entering first-year tertiary chemistry units, which are required for many different degree programs. The implementation of a new high school chemistry syllabus in Queensland in 2008 has simultaneously increased the range of prior learning experiences in secondary chemistry. Thus, it is vital for tertiary chemistry educators to focus on addressing both missing and mis-conceptions of incoming first-year students. We have profiled the existing conceptual understanding of incoming students enrolled in chemistry units at two major research-intensive tertiary institutions in Queensland in 2011. Concept inventory items were drawn from across a number of validated literature instruments. An alarming percentage of students were found to be unable to correctly answer simple questions relating to basic concepts. The concepts of bonding and states of matter, which are also well-known in the literature to cause difficulties, were particularly troublesome. Conceptual understanding across the two institutions is differentiated according to academic ability and program of study. The outcomes of this study demonstrate the need to develop strategies across the secondary-tertiary interface in preparation for yet another change in syllabus with the introduction of the national chemistry curriculum in 2013

“BIG BOOM + FIRE = IRON” – EXPERIENCES FROM SYSTEMS THINKING ORIENTED CHEMISTRY OUTREACH IN PRIMARY AND SECONDARY SCHOOLS

Systems thinking tools and approaches have been proposed as a means to holistically integrate authentic contexts of practice within education settings. Systems thinking can also play a community engagement role in re-positioning the public image of chemistry, from one that currently suffers from the consequences of large-scale uptake of its previous successes such as plastic waste and polluting industrial plants, to one that has embraced the principles of sustainability. Despite notable recent publications and a journal special issue on re-orientating chemical education to utilise systems thinking for this purpose (Mahaffy et al., 2018), thus far few systems thinking focussed resources, at any education level, have been reported. In this presentation, the design and evaluation of a recent systems thinking oriented chemistry outreach event (Periodic Table of Sustainable Elements) is described. The whole-day outreach event was conducted in seven low socioeconomic status schools involving over 1000 students in regional and rural Victoria. The event involved a series of hands-on practical activities, focussing on chemistry and its relevance to sustainability, and mentoring of upper secondary chemistry students by university student volunteers. Student and teacher perspectives collected through pre- and post-event surveys and teacher interviews have been analysed and will be presented. REFERENCE Mahaffy, P. G., Brush, E. J., Haack, J. A., & Ho, F. M. (2018). Journal of Chemical Education Call for Papers- Special Issue on Reimagining Chemistry Education: Systems Thinking, and Green and Sustainable Chemistry. Journal of Chemical Education, 95, 1689-1691

How to achieve “buy-in” of academics to a repository sharing teaching resources?

Chemistry is a traditional discipline in which research results are closely-guarded secrets, until their publication (or patent) with attendant recognition. We are trying to establish an open repository for teaching resources that will enable chemistry teachers around Australia to share their practical experiments, in-class activities or assessment tools, with a CreativeCommons license. The difficulty is in overcoming the natural reluctance of chemists to share anything! The repository will assist academics to improve their teaching to a diverse student cohort through shared experiences and successes

STUDENT ENGAGEMENT IN FIRST YEAR CHEMISTRY – ONLINE VS FACE TO FACE

Rapid campus closure and an unexpected transition to an online offering of a large first year chemistry unit provided an opportunity to explore the level of student engagement and factors associated with success in this online transition. In this presentation, we compare the standard, face-to-face offering in 2019 to the online offering in 2020 by examining the completion of weekly assessment tasks and the uptake of a variety of learning activities. All measures of engagement have been correlated to the students’ final scores in the unit. We found that students were able to quickly adapt to using the technology including accessing live streamed or recorded lectures, joining online tutorials and discussion forum use. Participation in in-class polling was unchanged, and was associated with the same increase in final score. The only learning activity with a significant change was tutorials; face-to-face tutorial attendance gained students 2.7 marks out of 100 in their final score per tutorial, whereas online tutorial attendance only gained 1.8 marks per tutorial. The decrease in completion of the weekly low stakes assessment tasks over the semester was unchanged, but overall quiz scores were slightly higher. Completion of online practical activities was the same in both years. Scores for a video recorded practical activity with drop-in consultation were 0.9/10 lower compared to the identical activity and worksheet conducted face to face. Our data show that some aspects of learning can be moved online seamlessly, but face-to-face practical sessions and tutorials have a greater impact on student success than online versions

Investigating the use of non-digital visual interaction tools in STEM education

Collaborative tools such as Post-it® notes are widely used in business settings in exploring new ideas, visualising processes and organising priorities. Businesses are moving towards open office space settings to encourage spontaneous collaboration and networking between workers and similar scene is implemented across education sector to generate synergy in student-student and teacher-student interaction. To facilitate such interactions, a vast array of digital tools is available, yet with greater focus in remote interaction, benefits of face to face engagement are diminished and students are under threat of losing the articulation skills in a face to face collaboration environment. It is speculated that use of Post-it® related products can enhance the student-student and student-teacher learning experiences in promoting expressive thinking process especially for informative, evaluative and generative collaboration. Data will be gathered during the conference through facilitation of interactive dialogue between the researcher and conference participants. The participants will be engaged to write a note on a piece of Post-it® note describing current state of student-teacher non-digital visual interaction and how Post-it® related products can be used in facilitation of STEM education. Information gathered will be sorted into three core areas of Informative Collaboration – Delivery of STEM content through shared ideas Evaluative Collaboration – Understanding of core STEM concepts through collaborative interaction Generative Collaboration – Exploration of STEM content through ideation Expected outcome will provide a valuable data on the current state of use of non-digital visualization methods in student-teacher interaction in aiding understanding of core STEM content

Articulating your own pedagogical content knowledge

Pedagogical content knowledge (PCK) encompasses carefully selected analogies, examples, explanations and demonstrations used by a teacher to make a topic comprehensible to students. It includes an understanding of what makes the big ideas difficult to grasp, along with an awareness of common misconceptions. PCK is developed by teachers through practitioner experience. John Loughran and his colleagues have spent over a decade creating and refining tools to articulate and develop PCK at the secondary level. Their framework consists of two elements: CoRe (content representation) and PaP-eR (pedagogical and professional experience repertoire). The CoRe contains eight questions for a teacher to reflect on, each to be answered for each big idea to be taught in a module, and should be developed and refined over time among small groups of teachers

Use of an online system for student responses in first year chemistry

Although the benefits of clickers for monitoring student understanding during lectures are well-established (Gebru, Phelps, & Wulfsberg, 2012; Lin, Liu, & Chu, 2011; MacArthur & Jones, 2008; Patry, 2009; Smith, Wood, Krauter, & Knight, 2011), their cost makes them inaccessible in many Faculties. Recently, several websites have been launched that offer academics the ability to monitor student understanding in a similar way, using any mobile internet enabled device (phone, tablet, laptop computer). We have trialed the use of one such system with first semester chemistry students and the results are reported here. There were some technical difficulties to be overcome and the importance of these should not be understated; in a time-limited setting with a very large class, a system needs to run flawlessly in every session. Student feedback was largely positive in other respects, although a few felt that the use of expensive devices that not every student has access to, was inequitable. The trial was undertaken as an action learning project within the SaMnet framework (http://www.samnet.edu.au) and aimed to promote the use of student response systems among other academic staff. To this end, staff were invited to attend lectures, participate in the student response questions and provide feedback on the system. This feedback also indicated that technical ease of use and stability are critical factors to encourage uptake of the system. This and other aspects of adoption have been investigated in the United States (Emenike & Holme, 2012). In this poster, we propose mechanisms to improve uptake and to minimise inequity of access. REFRENCES Emenike, M. E., & Holme, T. A. (2012). Classroom response systems have not “crossed the chasm”: Estimating numbers of chemistry faculty who use clickers. Journal of Chemical Education, 89, 465-469. Gebru, M. T., Phelps, A. J., & Wulfsberg, G. (2012). Effect of clickers versus online homework on students’ long-term retention of general chemistry course material. Chemistry Education Research and Practice, online doi: 10.1039/c2rp20033c Lin, Y.-C., Liu, T.-C., & Chu, C.-C. (2011). Implementing clickers to assist learning in science lectures: The Clicker-Assisted Conceptual Change model. Australasian Journal of Educational Technology, 27, 979-996. MacArthur, J. R., & Jones, L. L. (2008). A review of literature reports of clickers applicable to college chemistry classrooms. Chemistry Education Research and Practice, 9, 187-195. Patry, M. (2009). Clickers in large classes: From student perceptions towards an understanding of best practices. International Journal for the Scholarship of Teaching and Learning, 3. Smith, M. K., Wood, W. B., Krauter, K., & Knight, J. K. (2011). Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE - Life Sciences Education, 10, 55-63

Distilling tertiary pedagogical content knowledge to support a new generation of academics

Novice tertiary teachers rarely receive professional development in discipline-specific teaching practices. Instead they rely on a mixture of their own learning experiences, advice or mentoring provided by their colleagues and learning from their own mistakes as they gain teaching experience. This process is in stark contrast to secondary chemistry teacher training, where awareness of how students learn is linked to disciplinary context to help teachers develop their pedagogical content knowledge (PCK). Ask a chemistry academic about their PhD background and they will typically identify a specific chemistry sub-discipline such as inorganic, organic or physical chemistry. For the vast majority, this is not chemical education (Barthelemy, Henderson & Grunert, 2013). As tertiary teachers, their background influences their epistemological perspectives and informs their understanding and explanations of basic chemical concepts. As part of a study into the development and transfer of PCK to support the development of early career academics, we have collected a range of qualitative data. Participants in workshops and interviews were diverse in their experience and research alignment. Their explanations of their teaching strategies have been analysed inductively according to their background and the sub-disciplinary context of the teaching example that they provided. Several key elements of tertiary chemistry PCK have emerged from analysis of the qualitative data. Tertiary teachers integrated their PCK strategies in a manner parallel to that used by secondary teachers, including the use of analogies and metaphors, awareness of problematic concepts and use of representations (such as carefully selected demonstrations) (Loughran, Mulhall, & Berry, 2004). Findings indicate that the tertiary teachers’ beliefs, goals and practices align with PCK frameworks deriving from the secondary context (Fraser, 2015). A significant outcome was that experienced tertiary teachers expressed awareness of supporting students to progress to a deeper understanding of complex concepts. This presentation will identify and explore the ways that sub-discipline culture informs teaching practice Barthelemy, R. S., Henderson, C. & Grunert, M. L. (2013). How do they get here?: Paths into physics education research. Physical Review Special Topics - Physics Education Research, 9, 020107(14). Fraser, S (2015). Pedagogical content knowledge (PCK): Exploring its usefulness for science lecturers in higher education. Research in Science Education. Published online 22 March 2015. DOI 10.1007/s11165-014-9459-1 Loughran, J., Mulhall, P., & Berry, A. (2004). In Search of Pedagogical Content Knowledge in Science: Developing Ways of Articulating and Documenting Professional Practice. Journal of Research in Science Teaching, 41, 370-391. DOI: 10.1002/tea.2000

COLLECTING EVIDENCE OF GOOD PRACTICE AND LEADERSHIP IN A TUMULTUOUS TIME

GOAL To deliver an outcomes-focused workshop that guides participants in recognising and communicating potential sources of evidence as part of their teaching practice and leadership. BACKGROUND With the increase in education-focused roles around Australia, many tertiary institutions have established new pathways for recognition, reward and progression. However, the wave of new and transitioning tertiary educators in recent times may be unfamiliar with navigating through these new expectations and pathways. Fortunately, there are many commonalities in the reward and recognition processes for tenure, promotion and awards across institutions and a strong, supportive science education community to share experiences and advice! AIMS In this session, we will share our collective experiences and expectations across a range of Australian institutions. We will highlight proactive approaches to the collection and organisation of teaching and leadership evidence in different teaching and service contexts, paying close attention to the challenges posed by the transition to online teaching during the COVID-19 pandemic. Through this workshop, we intend to develop strategies that individual participants may employ to build their teaching and leadership portfolios. Participants from all science disciplines and academic levels are invited. DELIVERABLES Through this workshop we aim to facilitate the following: • A landscape view of commonalities in the awards and academic progression requirements across tertiary institutions; • Tips, tricks and strategies for the collection and organisation of teaching and leadership evidence; • Reflection on your own academic portfolio and plans for future evidence collection. WORKSHOP Introduction (15 minutes) We will begin this workshop by breaking down a few of the key expectations of institutions, including important similarities and differences. The promotion and award experiences of some of our most respected members within the science education community will be shared. Workshop task 1 (30 minutes) Participants will be split into small groups (2-3) to spend a short period of time evaluating the impact of different types of evidence. Coming back together, each group will summarise key points from their discussion.   Communicating your evidence (30 minutes) An important step in communicating your evidence is the consider your own, personal teaching philosophy. Through a short activity, this will be explored before splitting into small groups once more to spend time focused, through key prompts, on dot pointing some evidence of impact of their recent activities. Each member of the group will discuss their own experiences and provide each other with feedback regarding additional evidence they might seek and include. Wrap-up (15 minutes) To conclude, we will come together to once more share this experience with the wider group and discuss where-to from here. A set of tips and tricks for collecting and organising evidence will be provided and discussed