This note (2 of 2) provides a brief overview of a framework that can be used to measure performance of knowledge transfer during teaching-learning interactions in Science Technology Engineering and Mathematical (STEM) education. The framework works on a micro level, evaluating individual lectures, tutorials and laboratories

Gill, SK

UCL Discovery

   ARGnote   Vol. 1 No. 2 2014   ISSN 2054-4561 STEM Education: Part 2 How to measure performance at Higher Education institutions? Simrn Kaur Gill Overview  This note (2 of 2) provides a brief overview of a framework that can be used to measure performance of knowledge transfer during teaching-learning interactions in Science Technology Engineering and Mathematical (STEM) education. The framework works on a micro level, evaluating individual lectures, tutorials and laboratories. Aims & Objectives The note poses the following question:  1. How can performance be measured in STEM education (based on the key findings identified in ARGnotes Vol.1 No.1)?  Background  Laws et al.(1) identified that impressions made on the first day last to the end of the module. Yet the majority of student evaluation of teaching questionnaires are only used at the end of a module or course. As such performance of STEM education is currently measured using a number of different techniques (see ARGnote Vol 1 No. 1). This note provides a brief overview of an improved performance measurement system that can be used to help the “the quality, efficiency and effectiveness of teaching-learning interactions for all stakeholders … within the teaching-learning interactions that occur in the course of STEM education in lectures, tutorials and laboratories” (2).   Results The performance measurement system is called the ‘PERMEATE Framework’ – Process Engineering for Real-time Monitoring, Evaluation and Analysis of Teaching Excellence Framework. The PERMEATE implementation methodology includes a set of tools that were specifically developed and developed for the PERMEATE framework. These tools include PERMEATE Best Practice Traits, PERMEATE Best Practice Traits Checklists and PERMEATE Key Performance Key Findings The performance measurement system outlined in this note incorporate the following areas:   Combining evaluation techniques. Providing a qualitative and quantitative analysis of the knowledge transfer teaching setting.   Supports an analysis of both the instructor and the students during knowledge transfer teaching-learning interactions.  Provides support documentation. This includes documentation to help identify the stages in the knowledge transfer process and a best practice guide to help improve teaching effectiveness and performance measurement evaluation.  Support the use of the performance measurement system through a clearly defined closed loop flowchart that outlines the eight steps of the implementation. These steps are supported by the use of various tools that are integrated within these steps.  Supports analysts to determine corrective actions. By taking a micro view of knowledge transfer teaching-learning interactions corrective actions can occur at an earlier stage.   UCL ACCESSIBILITY RESEARCH GROUP ARGnote Vol. 1 No. 2 2014         Page 2   Contact: pamela@ucl.ac.uk Website: http://www.cege.ucl.ac.uk/arg/Pages/default.aspx © UCL Accessibility Research Group, University College London 2014. http://creativecommons.org/licenses/by/3.0/ Acknowledgements: I would like to thank my supervisor Prof. Nick Tyler and everyone who contributed to the design, development and implementation of the study. Indicators (KPIs) (see Table 1) in a unified approach as shown in Figure 1. The implementation methodology was adapted from the performance measurement process developed by United States Department Of Energy (USDOE) (3). Table 1 PERMEATE Framework tools Name  Description PERMEATE Best Practice Traits The best practice element of the toolset incorporates findings from numerous areas of research which include; aspects of; cognitive psychology, organisational strategy, and organisational behaviour. The findings from the literature are incorporated to create a best practice document that provides guidelines for effective and efficient knowledge transfer over four stages: acquisition, communication, application and assimilation. PERMEATE Best Practice Traits Checklists The best practice traits can be used to evaluate the quality and effectiveness of the knowledge that has been transferred.  They can also be used to assess the absorptive capacity of the individual and to identify the areas where the knowledge transfer process was deficient. This is accomplished by applying the best practice traits checklists to both the instructor and the student in the knowledge transfer process and then examining the gaps that are identified between both, over four stages: acquisition, communication, application and assimilation. PERMEATE Key Performance Indicators (KPIs) The key performance indicators are used to quantify knowledge transfer performance in the assessment of the effectiveness and efficiency of the process. The implementation methodology involves eight steps:  1. Identify the application area in the higher education teaching setting 2. Review PERMEATE Best Practice Traits to identify appropriate deployment of PERMEATE Best Practice Traits Checklists and PERMEATE Key Performance Indicators. 3. Collect data 4. Analyse/report actual performance 5. Compare actual performance to goal(s) 6. Are corrective actions necessary? 7. Make changes to bring back in line with goal(s) 8. Are new goal(s) needed? Future Research Areas Test the PERMEATE framework in various different teaching settings (lectures, tutorials, laboratories) within STEM education. Identify if greater transparency can be realised through the comparison of different universities to one another and identify what potential benefits this would result in for all stakeholders. The PERMEATE framework could be the first part of a much larger strategic assessment of education from primary to lifelong learning. StartIdentify application area in higher education teaching setting Review PERMEATE best practice traits  to identify appropriate deployment of PERMEATE best practice traits checklists and PERMEATE key performance indicatorsCollect dataAnalyse/report actual performanceCompare actual performance to goal(s)1Are corrective actions necessary?Make changes to bring back in line with goal(s)Are new goal(s) needed?NoYesYesNo2345678Database Figure 1 PERMEATE Implementation methodology adapted from United States Department of Energy (3) Related ARGnotes  Gill, SK, STEM Education: Part 1 What are the criteria for performance at Higher Education institutions? 2014, Vol 1. No.1 Endnotes: 1.Laws EL, Apperson JM, Buchert S, Bregman NJ. Student Evaluations of Instruction: When Are Enduring First Impressions Formed? North American Journal of Psychology. 2010;12(1):81-91. 2.Gill SK. STEM Education: Part 1 What are the criteria for performance at Higher Education institutions? ARGnote. 2014;1(1). 3.USDOE (United States Department of Energy). How to Measure Performance: A Handbook of Techniques and Tools1996.  

STEM Education: Part 2 How to measure performance at Higher Education institutions?

http://discovery.ucl.ac.uk/1464283/1/ARGnote_Vol1_No2_-_STEM_Education_-_Part_2_How_to_measure_performance_at_Higher_Education_institutions.pdf

STEM Education: Part 2 How to measure performance at Higher Education institutions?

Abstract

Similar works

Full text

Available Versions

UCL Discovery