Guide to using Evidence in Higher Education

This Guide to Using Evidence has been designed to, to support and encourage students and students’ association and union staff to actively engage with data and evidence. It offers an accessible introduction to a range of key ideas and concepts and a range of activities which allow readers to develop their own thinking and confidence in key areas. The ambition of its authors, QAA Scotland and the students who reviewed early drafts, is that students and students’ association and union staff will reach for this resource as they prepare for committees, devise new campaigns, deliver services, and do all of the other things they do to enhance students’ experiences and outcomes. Underpinning all of this is a belief that students themselves, the institutions they are working with, and the sector as a whole, are better served when students are, and are seen to be, agents in the ‘data landscape’, not just subjects of it. Engaging with this Guide will help students and students’ association and union staff to develop that sense of agency in themselves and foster it in others. This Guide is a product of a student-led project coordinated by QAA Scotland as part of the Evidence for Enhancement Theme (2017-20)

Integrative Reviewing for exploring complex phenomena

Integrative Reviews go beyond traditional boundaries of systematic reviewing by welcoming experts as valid sources of evidence and as providers of continuous data collection and synthesis • Development of a robust Integrative Review (IR) protocol is crucial for preserving confidence in the process and quality assurance • IRs are characterised by an underpinning positivist ontology, acknowledging that certain sources of evidence can be treated as real; yet IR furthers that position by acknowledging that such reality is socially constructed, thus allowing a more fluid epistemology to emerge, more aligned to a post-positivist perspective. This aligns IR processes with a Critical Realism traditio

Why is the BME attainment gap such a wicked problem?

This paper outlines a research process which followed a case study approach (Yin 2009) to explore the Black and Minority Ethnic (BME) student attainment gap, and responses to it, at Sheffield Hallam University. A mixed methodology was envisaged, which would triangulate institutional data, measures of student engagement, focus groups and researcher reflections to construct an analysis of interventions aimed at enhancing confidence and belonging for BME students. This discussion focuses on the challenges experienced by the research team and uses the notion of a 'wicked problem' to help understand the limitations faced. 'Wicked problems' (Rittel 1972, see Conklin 2005) are entrenched in social complexity, which increases in line with the diversity of the associated stakeholders. These problems have the ability to divide opinion, provide limited solutions and to lay blame for lack of results. This research examining the BME attainment gap can be critiqued using this notion of a 'wicked problem' noting that, without recognition, this issue has the potential to become ubiquitous and almost unsolvable

Evaluation for All?

This article discusses some of the challenges of current evaluation work in higher education (HE). Despite years of interest in this space, the sector is still struggling to capture what is working to enhance student outcomes. Our own recent work as evaluators for QAA Scotland’s Enhancement Themes seeks to address this impasse. This article will provide an overview of the Enhancement Themes work in Scotland and how evaluation is being re-positioned as the crux of activity within higher education institutions wishing to demonstrate effectiveness. A case study will be presented which illustrates how this could be achieved by describing the ongoing creation and piloting of a ‘Universal Evaluation Framework’ which aims to liberate colleagues from their fears around evaluation capabilities. The intended outcomes of use include the development of evaluation capabilities, increased confidence in evaluating change in HE spaces, and an evidence base for decision making regarding student outcomes. The interactive tool will enable colleagues to plan, design, implement and learn from their evaluations and record their reflections. Digital records extend this learning into publication, possible scholarship and, more tellingly, as evidence. This case study outlines a potentially game changing tool which aims to engage colleagues and capture evidence of what works in an accessible and transparent way. In contrast to other approaches to evaluation capacity building in UK HE, this tool aims to support all, regardless of levels of evaluation experience, role, and starting points. Our framework will be universal and accessible, and intuitively and inclusively designed

Digital capability and teaching excellence: an integrative review exploring what infrastructure and strategies are necessary to support effective use of technology enabled learning (TEL)

This report is set within the evolving landscape of UK Higher Education (HE) in which an increasingly marketised HE economy has led to a changing relationship with students and wider stakeholders. The proposed introduction of the Teaching Excellence Framework (TEF) (Department for Business, Innovation and Skills, 2016) aims to recognise and reward excellent learning and teaching. This integrative review provides timely evidence concerning the relationship of digital capability and teaching excellence. Keywords: teaching excellence, digital capabilit

Are all students 'hard to reach' in a digital higher education (H.E.) context?

Students entering Higher Education (HE) have high levels of digital capability built on the need to be socially-connected; they should be 'easy to reach’ if thinking about connectivity underpinning the digital age. Yet findings based on commissioned work exploring digital capability and teaching excellence (Austen et al, 2016) indicate that student appraisals of this relationship relate to the logistical benefits of technology rather than to the development of capabilities for professional practice, i.e. the multi-faceted set of academic and information-retrieval skills. Our study highlights a pedagogical challenge in needing to re-frame digital capability for all learners new to HE as this 'hard to reach' phenomenon is maintained by a student focus on technological hygiene factors rather than supported risk-taking

Creation and confidence: BME students as academic partners…but where were the staff?

This Case Study documents the REACT project 'Creation and Confidence' based at Sheffield Hallam University, which has a larger-than-expected Black Minority and Ethnic (BME) attainment gap; hence, this student group has been constructed as 'hard to reach'. The project team consisted of a range of academic and professional services staff alongside three dedicated student researchers. The project set out to achieve: gaining evidence-based insights into the use of co-design and peer-learning as conduits of confidence-building for and belonging of BME students; developing a scalable approach to building confidence for and fostering belonging of all students; raising awareness of the need to think differently about explanations for BME underachievement. In reality, the team found that staff engagement constituted the biggest barrier, as - no matter how much incontrovertible evidence was presented - other facets of institutional provision were always identified as having priority, which resulted in inertia. This study documents the emotional labour of trying to effect change within a resistant culture. Whilst some of the aims remain unachieved – and, arguably, were always going to be unachievable - there have been some very positive developments and enlightening lessons

Digital v Pedagogy: does it have to be a Monsters v Aliens scenario for higher education?

Keynote presentation by Professor Stella Jones-Devitt – director of learning and teaching, Staffordshire University presented at digital experience insights community of practice event on 06 November 2019

Evaluation for All? : Why evaluation within and beyond higher education should be universal and accessible

This article discusses some of the challenges of current evaluation work in higher education (HE). Despite years of interest in this space, the sector is still struggling to capture what is working to enhance student outcomes. Our own recent work as evaluators for QAA Scotland’s Enhancement Themes seeks to address this impasse. This article will provide an overview of the Enhancement Themes work in Scotland and how evaluation is being re-positioned as the crux of activity within higher education institutions wishing to demonstrate effectiveness. A case study will be presented which illustrates how this could be achieved by describing the ongoing creation and piloting of a ‘Universal Evaluation Framework’ which aims to liberate colleagues from their fears around evaluation capabilities. The intended outcomes of use include the development of evaluation capabilities, increased confidence in evaluating change in HE spaces, and an evidence base for decision making regarding student outcomes. The interactive tool will enable colleagues to plan, design, implement and learn from their evaluations and record their reflections. Digital records extend this learning into publication, possible scholarship and, more tellingly, as evidence. This case study outlines a potentially game changing tool which aims to engage colleagues and capture evidence of what works in an accessible and transparent way. In contrast to other approaches to evaluation capacity building in UK HE, this tool aims to support all, regardless of levels of evaluation experience, role, and starting points. Our framework will be universal and accessible, and intuitively and inclusively designed