Designing computer security assessments to reduce plagiarism

Plagiarism and other forms of academic dishonesty for computing science assessments is a well documented issue. A common mode of dealing with this is to apply plagiarism detector software to code submissions to check for suspected plagiarism based on how similar submissions are. However, it arguably is less well established how to design computing science specifc assessments which aim to reduce the possibility of plagiarism, whilst not disadvantaging students who may struggle with some aspects of an assessment. This paper aims to report on the design and practice of such an assessment within a computer security course

e-Authentication for online assessment: A mixed-method study

Authenticating the students’ identity and authenticity of their work is increasingly important to reduce academic malpractices and for quality assurance purposes in Education. There is a growing body of research about technological innovations to combat cheating and plagiarism. However, the literature is very limited on the impact of e-authentication systems across distinctive end-users because it is not a widespread practice at the moment. A considerable gap is to understand whether the use of e-authentication systems would increase trust on e-assessment, and to extend, whether students’ acceptance would vary across gender, age and previous experiences. This study aims to shed light on this area by examining the attitudes and experiences of 328 students who used an authentication system known as adaptive trust-based e-assessment system for learning (TeSLA). Evidence from mixed-method analysis suggests a broadly positive acceptance of these e-authentication technologies by distance education students. However, significant differences in the students’ responses indicated, for instance, that men were less concerned about providing personal data than women; middle-aged participants were more aware of the nuances of cheating and plagiarism;while younger students were more likely to reject e-authentication, considerably due to data privacy and security and students with disabilities due to concerns about their special needs

Online delivery and assessment during COVID-19: safeguarding academic integrity

Globally, the number of COVID-19 cases continues to rise daily despite strict measures being adopted by many countries. Consequently, universities closed down to minimise the face-to-face contacts, and the majority of the universities are now conducting degree programmes through online delivery. Remote online delivery and assessment are novel experiences for many universities, which presents many challenges, particularly when safeguarding academic integrity. For example, invigilated assessments, often considered as more secure, are not an option given the current situation and detecting any cheating would be significantly challenging. This paper reviews assessment security in the digital domain and critically evaluates the practices from different universities in safeguarding academic integrity, including associated challenges

Quality Assessment for E-learning: a Benchmarking Approach (Third edition)

The primary purpose of this manual is to provide a set of benchmarks, quality criteria and notes for guidance against which e-learning programmes and their support systems may be judged. The manual should therefore be seen primarily as a reference tool for the assessment or review of e-learning programmes and the systems which support them. However, the manual should also prove to be useful to staff in institutions concerned with the design, development, teaching, assessment and support of e-learning programmes. It is hoped that course developers, teachers and other stakeholders will see the manual as a useful development and/or improvement tool for incorporation in their own institutional systems of monitoring, evaluation and enhancement

A mixed-method triangular approach to best practices in combating plagiarism and impersonation in online bachelor’s degree programs

This study examines the phenomenon of plagiarism and impersonation in online course assignments. Technological advancements, coupled with lower costs and accessibility, have made online courses and programs a practical option for higher education students. Unfortunately, the increasing online enrollment and advancing technology have allowed an increase in the opportunity for students to commit the act of plagiarism and impersonation in online course assignments, thus potentially compromising the academic integrity of online degree programs. This study examines the various practices and approaches of plagiarism and impersonation made available to students. Utilizing the systemic review of literature, the researcher compiles a list of 20 best practices in combating plagiarism and impersonation in online course assignments. A Delphi method approach is employed, utilizing the expertise of professors who teach in fully online bachelor’s degree programs. The 20 best practices established through the literature review will be narrowed down to ten best practices via an ordinal ranking questionnaire using a two-round format. The questionnaire distribution occurs via e-mails. Researching professors that teach in fully online bachelor’s degree programs is how the researcher will obtain the e-mails. The first-round e-mail consists of the consent form and the original set of 20 best practices. In addition, a link to the Qualtrics ranking survey will be included in the e-mail. The second-round e-mail consists of the updated 15 best practices ranked from the initial e-mail and a link to the ranking survey. After completing the second round, the establishment of the ten best practices for reducing plagiarism and impersonation in online assignments will emerge. To further validate the 10 best practices, the researcher interviews 10 professors that participated in the original Delphi study. The original consent form includes a link for the participants to access if they select to participate in the interview. After verifying the professors’ intent to participate, a consent form will be obtained. The interviews will be conducted and recorded virtually through zoom. The recordings will be deleted once they are transcribed. This study potentially benefits all online degree programs by establishing the ten best practices for reducing plagiarism and impersonation in online assignments

Remote assessment: case study from Ukrainian higher education institutions

Avaliação remota: estudo de caso em instituições do ensino superior na Ucrânia Este estudo cita a implementação da avaliação à distância durante a educação pandêmica da COVID-19, com base nas reflexões de 600 estudantes e 150 educadores. Os dados foram recolhidos através do formulário do Google Forms. A investigação centra-se nos desafios da avaliação remota do e-learning durante a crise da COVID-19 enfrentada por educadores e estudantes ucranianos e nas abordagens adotadas para ultrapassá-los: projetos de aprendizagem, soluções tecnológicas, sistemas de avaliação e medidas de integridade acadêmica. O estudo atual também analisa as plataformas e aplicações atualmente em uso que podem ajudar a melhorar a integridade acadêmica e reforçar a educação durante as pandemias e as várias intervenções tecnológicas de ensino, aprendizagem e avaliação que foram introduzidas ou continuadas em instituições de ensino superior em todo o mundo, e como podem ter ajudado a reduzir a probabilidade de mau comportamento dos estudantes durante o ensino à distância de emergência na pandemia. O principal resultado aponta que os problemas de avaliação e plágio têm de ser abordados a vários níveis, incluindo o reforço dos conhecimentos e da ética dos estudantes; a superação da resistência da parte conservadora dos participantes no processo educativo às inovações educativas devido ao nível relativamente baixo do seu domínio das tecnologias educativas modernas; a formação de professores para detectar plágios; e a aplicação de sanções severas àqueles que se dedicam a tais práticas. Palavras-chave: E-learning, avaliação à distância, integridade acadêmica, intervenção tecnológica.   Remote assessment: case study from Ukrainian higher education institutions ABSTRACT. This study narrates the implementation of remote assessment during the COVID-19 pandemic education in Ukrainian higher education institutions, based on exploring the reflections of 600 students and 150 educators. A wide range of student and faculty members experiences from Ukrainian higher education institutions were aggregated and analyzed. The data were collected through the Google Forms survey. The research focuses on the challenges of remote e-learning assessment during the COVID-19 crisis encountered by Ukrainian educators and learners and the approaches adopted to overcome them: learning designs, technological solutions, evaluation systems, and academic integrity measures. The current study also looks at the currently used platforms and applications that can assist in enhancing academic integrity and reinforcing education during pandemics and the various technological interventions to teaching, learning, and assessing that have been introduced or continued in higher education institutions around the world, and how they may have helped reduce the likelihood of students committing misconduct during the pandemic's emergency distance learning. The major outcome implies that addressing the issues of online assessment, cheating, and plagiarism on multiple levels is necessary, including boosting student knowledge and ethics; overcoming the resistance of the conservative part of the participants in the educational process to educational innovations due to the relatively low level of their mastery of modern educational technologies; training teachers to detect cheating methods; and applying severe sanctions on those who engage in such practices. Keywords: E-learning, remote assessment, academic integrity, technological intervention.   Evaluación a distancia: estudio de caso de instituciones de educación superior de Ucrania RESUMEN. Este estudio narra la implementación de la evaluación remota durante la educación pandémica de COVID-19, basado en las reflexiones de 600 estudiantes y 150 educadores. Los datos fueron recolectados a través de la encuesta Google Forms. La investigación se centra en los desafíos de la evaluación del aprendizaje electrónico a distancia durante la crisis de la COVID-19 a los que se enfrentaron los educadores y estudiantes ucranianos y los enfoques adoptados para superarlos: diseños de aprendizaje, soluciones tecnológicas, sistemas de evaluación y medidas de integridad académica. El estudio actual también analiza las plataformas y aplicaciones utilizadas actualmente que pueden ayudar a mejorar la integridad académica y reforzar la educación durante las pandemias y las diversas intervenciones tecnológicas para la enseñanza, el aprendizaje y la evaluación que se han introducido o continuado en las instituciones de educación superior de todo el mundo, y cómo pueden haber ayudado a reducir la probabilidad de que los estudiantes cometan malas conductas durante el aprendizaje a distancia de emergencia de la pandemia. El resultado principal implica que es necesario abordar los problemas de la evaluación en línea, las trampas y el plagio en múltiples niveles, lo que incluye impulsar el conocimiento y la ética de los estudiantes; superar la resistencia de la parte conservadora de los participantes en el proceso educativo a las innovaciones educativas debido al nivel relativamente bajo de su dominio de las tecnologías educativas modernas; capacitar a las maestras para detectar métodos de engaño; y aplicar severas sanciones a quienes incurran en tales prácticas. Palabras clave: E-aprendizaje, evaluación remota, integridad académica, intervención tecnológica.Avaliação remota: estudo de caso em instituições do ensino superior na Ucrânia Este estudo cita a implementação da avaliação à distância durante a educação pandêmica da COVID-19, com base nas reflexões de 600 estudantes e 150 educadores. Os dados foram recolhidos através do formulário do Google Forms. A investigação centra-se nos desafios da avaliação remota do e-learning durante a crise da COVID-19 enfrentada por educadores e estudantes ucranianos e nas abordagens adotadas para ultrapassá-los: projetos de aprendizagem, soluções tecnológicas, sistemas de avaliação e medidas de integridade acadêmica. O estudo atual também analisa as plataformas e aplicações atualmente em uso que podem ajudar a melhorar a integridade acadêmica e reforçar a educação durante as pandemias e as várias intervenções tecnológicas de ensino, aprendizagem e avaliação que foram introduzidas ou continuadas em instituições de ensino superior em todo o mundo, e como podem ter ajudado a reduzir a probabilidade de mau comportamento dos estudantes durante o ensino à distância de emergência na pandemia. O principal resultado aponta que os problemas de avaliação e plágio têm de ser abordados a vários níveis, incluindo o reforço dos conhecimentos e da ética dos estudantes; a superação da resistência da parte conservadora dos participantes no processo educativo às inovações educativas devido ao nível relativamente baixo do seu domínio das tecnologias educativas modernas; a formação de professores para detectar plágios; e a aplicação de sanções severas àqueles que se dedicam a tais práticas. Palavras-chave: E-learning, avaliação à distância, integridade acadêmica, intervenção tecnológica.   Remote assessment: case study from Ukrainian higher education institutions ABSTRACT. This study narrates the implementation of remote assessment during the COVID-19 pandemic education in Ukrainian higher education institutions, based on exploring the reflections of 600 students and 150 educators. A wide range of student and faculty members experiences from Ukrainian higher education institutions were aggregated and analyzed. The data were collected through the Google Forms survey. The research focuses on the challenges of remote e-learning assessment during the COVID-19 crisis encountered by Ukrainian educators and learners and the approaches adopted to overcome them: learning designs, technological solutions, evaluation systems, and academic integrity measures. The current study also looks at the currently used platforms and applications that can assist in enhancing academic integrity and reinforcing education during pandemics and the various technological interventions to teaching, learning, and assessing that have been introduced or continued in higher education institutions around the world, and how they may have helped reduce the likelihood of students committing misconduct during the pandemic's emergency distance learning. The major outcome implies that addressing the issues of online assessment, cheating, and plagiarism on multiple levels is necessary, including boosting student knowledge and ethics; overcoming the resistance of the conservative part of the participants in the educational process to educational innovations due to the relatively low level of their mastery of modern educational technologies; training teachers to detect cheating methods; and applying severe sanctions on those who engage in such practices. Keywords: E-learning, remote assessment, academic integrity, technological intervention.   Evaluación a distancia: estudio de caso de instituciones de educación superior de Ucrania RESUMEN. Este estudio narra la implementación de la evaluación remota durante la educación pandémica de COVID-19, basado en las reflexiones de 600 estudiantes y 150 educadores. Los datos fueron recolectados a través de la encuesta Google Forms. La investigación se centra en los desafíos de la evaluación del aprendizaje electrónico a distancia durante la crisis de la COVID-19 a los que se enfrentaron los educadores y estudiantes ucranianos y los enfoques adoptados para superarlos: diseños de aprendizaje, soluciones tecnológicas, sistemas de evaluación y medidas de integridad académica. El estudio actual también analiza las plataformas y aplicaciones utilizadas actualmente que pueden ayudar a mejorar la integridad académica y reforzar la educación durante las pandemias y las diversas intervenciones tecnológicas para la enseñanza, el aprendizaje y la evaluación que se han introducido o continuado en las instituciones de educación superior de todo el mundo, y cómo pueden haber ayudado a reducir la probabilidad de que los estudiantes cometan malas conductas durante el aprendizaje a distancia de emergencia de la pandemia. El resultado principal implica que es necesario abordar los problemas de la evaluación en línea, las trampas y el plagio en múltiples niveles, lo que incluye impulsar el conocimiento y la ética de los estudiantes; superar la resistencia de la parte conservadora de los participantes en el proceso educativo a las innovaciones educativas debido al nivel relativamente bajo de su dominio de las tecnologías educativas modernas; capacitar a las maestras para detectar métodos de engaño; y aplicar severas sanciones a quienes incurran en tales prácticas. Palabras clave: E-aprendizaje, evaluación remota, integridad académica, intervención tecnológica.Avaliação remota: estudo de caso em instituições do ensino superior na Ucrânia Este estudo cita a implementação da avaliação à distância durante a educação pandêmica da COVID-19, com base nas reflexões de 600 estudantes e 150 educadores. Os dados foram recolhidos através do formulário do Google Forms. A investigação centra-se nos desafios da avaliação remota do e-learning durante a crise da COVID-19 enfrentada por educadores e estudantes ucranianos e nas abordagens adotadas para ultrapassá-los: projetos de aprendizagem, soluções tecnológicas, sistemas de avaliação e medidas de integridade acadêmica. O estudo atual também analisa as plataformas e aplicações atualmente em uso que podem ajudar a melhorar a integridade acadêmica e reforçar a educação durante as pandemias e as várias intervenções tecnológicas de ensino, aprendizagem e avaliação que foram introduzidas ou continuadas em instituições de ensino superior em todo o mundo, e como podem ter ajudado a reduzir a probabilidade de mau comportamento dos estudantes durante o ensino à distância de emergência na pandemia. O principal resultado aponta que os problemas de avaliação e plágio têm de ser abordados a vários níveis, incluindo o reforço dos conhecimentos e da ética dos estudantes; a superação da resistência da parte conservadora dos participantes no processo educativo às inovações educativas devido ao nível relativamente baixo do seu domínio das tecnologias educativas modernas; a formação de professores para detectar plágios; e a aplicação de sanções severas àqueles que se dedicam a tais práticas. Palavras-chave: E-learning, avaliação à distância, integridade acadêmica, intervenção tecnológica.   Remote assessment: case study from Ukrainian higher education institutions ABSTRACT. This study narrates the implementation of remote assessment during the COVID-19 pandemic education in Ukrainian higher education institutions, based on exploring the reflections of 600 students and 150 educators. A wide range of student and faculty members experiences from Ukrainian higher education institutions were aggregated and analyzed. The data were collected through the Google Forms survey. The research focuses on the challenges of remote e-learning assessment during the COVID-19 crisis encountered by Ukrainian educators and learners and the approaches adopted to overcome them: learning designs, technological solutions, evaluation systems, and academic integrity measures. The current study also looks at the currently used platforms and applications that can assist in enhancing academic integrity and reinforcing education during pandemics and the various technological interventions to teaching, learning, and assessing that have been introduced or continued in higher education institutions around the world, and how they may have helped reduce the likelihood of students committing misconduct during the pandemic's emergency distance learning. The major outcome implies that addressing the issues of online assessment, cheating, and plagiarism on multiple levels is necessary, including boosting student knowledge and ethics; overcoming the resistance of the conservative part of the participants in the educational process to educational innovations due to the relatively low level of their mastery of modern educational technologies; training teachers to detect cheating methods; and applying severe sanctions on those who engage in such practices. Keywords: E-learning, remote assessment, academic integrity, technological intervention.   Evaluación a distancia: estudio de caso de instituciones de educación superior de Ucrania RESUMEN. Este estudio narra la implementación de la evaluación remota durante la educación pandémica de COVID-19, basado en las reflexiones de 600 estudiantes y 150 educadores. Los datos fueron recolectados a través de la encuesta Google Forms. La investigación se centra en los desafíos de la evaluación del aprendizaje electrónico a distancia durante la crisis de la COVID-19 a los que se enfrentaron los educadores y estudiantes ucranianos y los enfoques adoptados para superarlos: diseños de aprendizaje, soluciones tecnológicas, sistemas de evaluación y medidas de integridad académica. El estudio actual también analiza las plataformas y aplicaciones utilizadas actualmente que pueden ayudar a mejorar la integridad académica y reforzar la educación durante las pandemias y las diversas intervenciones tecnológicas para la enseñanza, el aprendizaje y la evaluación que se han introducido o continuado en las instituciones de educación superior de todo el mundo, y cómo pueden haber ayudado a reducir la probabilidad de que los estudiantes cometan malas conductas durante el aprendizaje a distancia de emergencia de la pandemia. El resultado principal implica que es necesario abordar los problemas de la evaluación en línea, las trampas y el plagio en múltiples niveles, lo que incluye impulsar el conocimiento y la ética de los estudiantes; superar la resistencia de la parte conservadora de los participantes en el proceso educativo a las innovaciones educativas debido al nivel relativamente bajo de su dominio de las tecnologías educativas modernas; capacitar a las maestras para detectar métodos de engaño; y aplicar severas sanciones a quienes incurran en tales prácticas. Palabras clave: E-aprendizaje, evaluación remota, integridad académica, intervención tecnológica.Avaliação remota: estudo de caso em instituições do ensino superior na Ucrânia Este estudo cita a implementação da avaliação à distância durante a educação pandêmica da COVID-19, com base nas reflexões de 600 estudantes e 150 educadores. Os dados foram recolhidos através do formulário do Google Forms. A investigação centra-se nos desafios da avaliação remota do e-learning durante a crise da COVID-19 enfrentada por educadores e estudantes ucranianos e nas abordagens adotadas para ultrapassá-los: projetos de aprendizagem, soluções tecnológicas, sistemas de avaliação e medidas de integridade acadêmica. O estudo atual também analisa as plataformas e aplicações atualmente em uso que podem ajudar a melhorar a integridade acadêmica e reforçar a educação durante as pandemias e as várias intervenções tecnológicas de ensino, aprendizagem e avaliação que foram introduzidas ou continuadas em instituições de ensino superior em todo o mundo, e como podem ter ajudado a reduzir a probabilidade de mau comportamento dos estudantes durante o ensino à distância de emergência na pandemia. O principal resultado aponta que os problemas de avaliação e plágio têm de ser abordados a vários níveis, incluindo o reforço dos conhecimentos e da ética dos estudantes; a superação da resistência da parte conservadora dos participantes no processo educativo às inovações educativas devido ao nível relativamente baixo do seu domínio das tecnologias educativas modernas; a formação de professores para detectar plágios; e a aplicação de sanções severas àqueles que se dedicam a tais práticas. Palavras-chave: E-learning, avaliação à distância, integridade acadêmica, intervenção tecnológica.   Remote assessment: case study from Ukrainian higher education institutions ABSTRACT. This study narrates the implementation of remote assessment during the COVID-19 pandemic education in Ukrainian higher education institutions, based on exploring the reflections of 600 students and 150 educators. A wide range of student and faculty members experiences from Ukrainian higher education institutions were aggregated and analyzed. The data were collected through the Google Forms survey. The research focuses on the challenges of remote e-learning assessment during the COVID-19 crisis encountered by Ukrainian educators and learners and the approaches adopted to overcome them: learning designs, technological solutions, evaluation systems, and academic integrity measures. The current study also looks at the currently used platforms and applications that can assist in enhancing academic integrity and reinforcing education during pandemics and the various technological interventions to teaching, learning, and assessing that have been introduced or continued in higher education institutions around the world, and how they may have helped reduce the likelihood of students committing misconduct during the pandemic's emergency distance learning. The major outcome implies that addressing the issues of online assessment, cheating, and plagiarism on multiple levels is necessary, including boosting student knowledge and ethics; overcoming the resistance of the conservative part of the participants in the educational process to educational innovations due to the relatively low level of their mastery of modern educational technologies; training teachers to detect cheating methods; and applying severe sanctions on those who engage in such practices. Keywords: E-learning, remote assessment, academic integrity, technological intervention.   Evaluación a distancia: estudio de caso de instituciones de educación superior de Ucrania RESUMEN. Este estudio narra la implementación de la evaluación remota durante la educación pandémica de COVID-19, basado en las reflexiones de 600 estudiantes y 150 educadores. Los datos fueron recolectados a través de la encuesta Google Forms. La investigación se centra en los desafíos de la evaluación del aprendizaje electrónico a distancia durante la crisis de la COVID-19 a los que se enfrentaron los educadores y estudiantes ucranianos y los enfoques adoptados para superarlos: diseños de aprendizaje, soluciones tecnológicas, sistemas de evaluación y medidas de integridad académica. El estudio actual también analiza las plataformas y aplicaciones utilizadas actualmente que pueden ayudar a mejorar la integridad académica y reforzar la educación durante las pandemias y las diversas intervenciones tecnológicas para la enseñanza, el aprendizaje y la evaluación que se han introducido o continuado en las instituciones de educación superior de todo el mundo, y cómo pueden haber ayudado a reducir la probabilidad de que los estudiantes cometan malas conductas durante el aprendizaje a distancia de emergencia de la pandemia. El resultado principal implica que es necesario abordar los problemas de la evaluación en línea, las trampas y el plagio en múltiples niveles, lo que incluye impulsar el conocimiento y la ética de los estudiantes; superar la resistencia de la parte conservadora de los participantes en el proceso educativo a las innovaciones educativas debido al nivel relativamente bajo de su dominio de las tecnologías educativas modernas; capacitar a las maestras para detectar métodos de engaño; y aplicar severas sanciones a quienes incurran en tales prácticas. Palabras clave: E-aprendizaje, evaluación remota, integridad académica, intervención tecnológica.Avaliação remota: estudo de caso em instituições do ensino superior na Ucrânia Este estudo cita a implementação da avaliação à distância durante a educação pandêmica da COVID-19, com base nas reflexões de 600 estudantes e 150 educadores. Os dados

Biometric surveillance in schools : cause for concern or case for curriculum?

This article critically examines the draft consultation paper issued by the Scottish Government to local authorities on the use of biometric technologies in schools in September 2008 (see http://www.scotland.gov.uk/Publications/2008/09/08135019/0). Coming at a time when a number of schools are considering using biometric systems to register and confirm the identity of pupils in a number of settings (cashless catering systems, automated registration of pupils' arrival in school and school library automation), this guidance is undoubtedly welcome. The present focus seems to be on using fingerprints, but as the guidance acknowledges, the debate in future may encompass iris prints, voice prints and facial recognition systems, which are already in use in non-educational settings. The article notes broader developments in school surveillance in Scotland and in the rest of the UK and argues that serious attention must be given to the educational considerations which arise. Schools must prepare pupils for life in the newly emergent 'surveillance society', not by uncritically habituating them to the surveillance systems installed in their schools, but by critically engaging them in thought about the way surveillance technologies work in the wider world, the various rationales given to them, and the implications - in terms of privacy, safety and inclusion - of being a 'surveilled subject'

An Evaluation Methodology Applied to Trust-Based Adapted Systems for e-Assessment: Connecting Responsible Research and Innovation with a Human-Centred Design Approach

This chapter describes a novel evaluation methodology designed, deployed and refined during the development of the EU-funded TeSLA system which was produced to check student authentication and authorship. This methodology was underpinned by a Responsible Research and Innovation approach combined with human-centred design. Participants were 4,058 students, which included 330 with special needs, together with 54 teaching staff and 21 institutional members from seven universities who completed consultation, focus groups, questionnaires and interviews. The findings suggest that the evaluation methodology was able to identify a broadly positive acceptance of and trust in e-authentication for online assessments by both women and men, with neither group finding the e-authentication tools to be either particularly onerous or stressful. The methodology facilitated the development of a framework with five features related to “trust”: 1. The system will not fail, 2. be compromised, 3. data will be kept safely and privately, 4. the system will not affect students’ performance and 5. the system will ensure fairness

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research