Eliciting students' preferences for the use of their data for learning analytics. A crowdsourcing approach.

Research on student perspectives of learning analytics suggests that students are generally unaware of the collection and use of their data by their learning institutions, and they are often not involved in decisions about whether and how their data are used. To determine the influence of risks and benefits awareness on students’ data use preferences for learning analytics, we designed two interventions: one describing the possible privacy risks of data use for learning analytics and the second describing the possible benefits. These interventions were distributed amongst 447 participants recruited using a crowdsourcing platform. Participants were randomly assigned to one of three experimental groups – risks, benefits, and risks and benefits – and received the corresponding intervention(s). Participants in the control group received a learning analytics dashboard (as did participants in the experimental conditions). Participants’ indicated the motivation for their data use preferences. Chapter 11 will discuss the implications of our findings in relation to how to better support learning institutions in being more transparent with students about the practice of learning analytics

Eliciting students' preferences for the use of their data for learning analytics. A crowdsourcing approach.

Research on student perspectives of learning analytics suggests that students are generally unaware of the collection and use of their data by their learning institutions, and they are often not involved in decisions about whether and how their data are used. To determine the influence of risks and benefits awareness on students’ data use preferences for learning analytics, we designed two interventions: one describing the possible privacy risks of data use for learning analytics and the second describing the possible benefits. These interventions were distributed amongst 447 participants recruited using a crowdsourcing platform. Participants were randomly assigned to one of three experimental groups – risks, benefits, and risks and benefits – and received the corresponding intervention(s). Participants in the control group received a learning analytics dashboard (as did participants in the experimental conditions). Participants’ indicated the motivation for their data use preferences. Chapter 11 will discuss the implications of our findings in relation to how to better support learning institutions in being more transparent with students about the practice of learning analytics

CrowdCO-OP : sharing risks and rewards in crowdsourcing

Paid micro-task crowdsourcing has gained in popularity partly due to the increasing need for large-scale manually labelled datasets which are often used to train and evaluate Artificial Intelligence systems. Modern paid crowdsourcing platforms use a piecework approach to rewards, meaning that workers are paid for each task they complete, given that their work quality is considered sufficient by the requester or the platform. Such an approach creates risks for workers; their work may be rejected without being rewarded, and they may be working on poorly rewarded tasks, in light of the disproportionate time required to complete them. As a result, recent research has shown that crowd workers may tend to choose specific, simple, and familiar tasks and avoid new requesters to manage these risks. In this paper, we propose a novel crowdsourcing reward mechanism that allows workers to share these risks and achieve a standardized hourly wage equal for all participating workers. Reward-focused workers can thereby take up challenging and complex HITs without bearing the financial risk of not being rewarded for completed work. We experimentally compare different crowd reward schemes and observe their impact on worker performance and satisfaction. Our results show that 1) workers clearly perceive the benefits of the proposed reward scheme, 2) work effectiveness and efficiency are not impacted as compared to those of the piecework scheme, and 3) the presence of slow workers is limited and does not disrupt the proposed cooperation-based approaches

Explanation-based human debugging of nlp models: a survey

Debugging a machine learning model is hard since the bug usually involves the training data and the learning process. This becomes even harder for an opaque deep learning model if we have no clue about how the model actually works. In this survey, we review papers that exploit explanations to enable humans to give feedback and debug NLP models. We call this problem explanation-based human debugging (EBHD). In particular, we categorize and discuss existing work along three dimensions of EBHD (the bug context, the workflow, and the experimental setting), compile findings on how EBHD components affect the feedback providers, and highlight open problems that could be future research directions

Open World Learning

This book provides state-of-the-art contemporary research insights into key applications and processes in open world learning. Open world learning seeks to understand access to education, structures, and the presence of dialogue and support systems. It explores how the application of open world and educational technologies can be used to create opportunities for open and high-quality education. Presenting ground-breaking research from an award winning Leverhulme doctoral training programme, the book provides several integrated and cohesive perspectives of the affordances and limitations of open world learning. The chapters feature a wide range of open world learning topics, ranging from theoretical and methodological discussions to empirical demonstrations of how open world learning can be effectively implemented, evaluated, and used to inform theory and practice. The book brings together a range of innovative uses of technology and practice in open world learning from 387,134 learners and educators learning and working in 136 unique learning contexts across the globe and considers the enablers and disablers of openness in learning, ethical and privacy implications, and how open world learning can be used to foster inclusive approaches to learning across educational sectors, disciplines and countries. The book is unique in exploring the complex, contradictory and multi-disciplinary nature of open world learning at an international level and will be of great interest to academics, researchers, professionals, and policy makers in the field of education technology, e-learning and digital education

Open World Learning

This book provides state-of-the-art contemporary research insights into key applications and processes in open world learning. Open world learning seeks to understand access to education, structures, and the presence of dialogue and support systems. It explores how the application of open world and educational technologies can be used to create opportunities for open and high-quality education. Presenting ground-breaking research from an award winning Leverhulme doctoral training programme, the book provides several integrated and cohesive perspectives of the affordances and limitations of open world learning. The chapters feature a wide range of open world learning topics, ranging from theoretical and methodological discussions to empirical demonstrations of how open world learning can be effectively implemented, evaluated, and used to inform theory and practice. The book brings together a range of innovative uses of technology and practice in open world learning from 387,134 learners and educators learning and working in 136 unique learning contexts across the globe and considers the enablers and disablers of openness in learning, ethical and privacy implications, and how open world learning can be used to foster inclusive approaches to learning across educational sectors, disciplines and countries. The book is unique in exploring the complex, contradictory and multi-disciplinary nature of open world learning at an international level and will be of great interest to academics, researchers, professionals, and policy makers in the field of education technology, e-learning and digital education

An Investigation of Privacy and Utility Tensions in Learning Analytics

Through learning analytics (LA), higher education institutions have put student data to various uses which aim to be beneficial for students, lecturers, and the institutions. Despite the potential benefits of LA, however, there are research gaps in understanding inherent privacy and utility tensions. Using four research studies, this thesis is an investigation of factors that contribute to these tensions. An examination, using Delphi study techniques, of how LA experts (n=12) conceptualised privacy in LA and what they thought of as key privacy issues demonstrated a collective agreement on institutional responsibility, including to empower students to manage their privacy. As such, the findings exposed gaps between existing institutional applications of LA and the views of the experts. A laboratory study (n=111) with follow-up semi-structured interviews (n=4) identified that students are not concerned about the use of their data for LA. However, knowing that student data could be shared with third parties evoked feelings of discomfort. The qualitative data suggested that students’ privacy concepts differed from those of the LA experts, highlighting a need to operationalise LA with a shared understanding of what privacy means for stakeholders. Using an experimental design (n=447), privacy concern was further examined through the lens of students’ data use preferences. The findings suggested that participants’ data use preferences were not influenced by an awareness of the possible privacy risks and benefits of data use for LA. Consequently, other factors might influence students’ data use preferences. The qualitative data shed light on a “dual nature” to participants’ data use preferences, suggesting both support for and reservation about the use of student data for LA, the latter due to ethics and privacy concerns. Further examination using follow-up interviews (n=15) suggested a need to align institutional data use practices with students’ expectations. Taken together, the research findings suggest that privacy in LA combines several concepts, expressed in different ways across stakeholder groups. To better understand students’ perspectives of privacy in LA requires unpacking the dimensions of privacy that contribute to students’ privacy concern, or lack thereof. Most importantly, while some uses of student data for LA do not concern students, other data uses might not meet their expectations. Taking steps to address these tensions will contribute towards ethical LA

Open World Learning: Research, Innovation and the Challenges of High-Quality Education

This book provides state-of-the-art contemporary research insights into key applications and processes in open world learning. Open world learning seeks to understand access to education, structures, and the presence of dialogue and support systems. It explores how the application of open world and educational technologies can be used to create opportunities for open and high-quality education. Presenting ground-breaking research from an award winning Leverhulme doctoral training programme, the book provides several integrated and cohesive perspectives of the affordances and limitations of open world learning. The chapters feature a wide range of open world learning topics, ranging from theoretical and methodological discussions to empirical demonstrations of how open world learning can be effectively implemented, evaluated, and used to inform theory and practice. The book brings together a range of innovative uses of technology and practice in open world learning from 387,134 learners and educators learning and working in 136 unique learning contexts across the globe and considers the enablers and disablers of openness in learning, ethical and privacy implications, and how open world learning can be used to foster inclusive approaches to learning across educational sectors, disciplines and countries. The book is unique in exploring the complex, contradictory and multi-disciplinary nature of open world learning at an international level and will be of great interest to academics, researchers, professionals, and policy makers in the field of education technology, e-learning and digital education. The Open Access version of this book, available at www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 license