Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency, and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3/4), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models

Organizational Readiness Concept for AI: A Quantitative Analysis of a Multi-stage Adoption Process from the Perspective of Data Scientists

Artificial intelligence (AI) is reshaping the business world in ways that enable organizations to create business value and reinvent their business models. Despite the great potential, organizations have difficulties in moving beyond the pilot stage and fully adopting AI applications. To better understand how organizations can implement AI into their core practices, we examine the impact of organizational readiness factors along the adoption process of AI through a quantitative research design. By integrating the organizational readiness factors into the multi-stage adoption process of AI, we unpack the interdependencies between these two literature streams. Due to the multi-faceted nature of organizations, we investigate the differentiating and opposing effects of the organizational readiness factors on the initiation, adoption, and routinization stages of AI

Assessing Organizational Readiness for Data-driven Innovation: A Review of Literature

The growing demand for data has provided many opportunities for organizations to launch data-driven innovation (DDI) initiatives. DDI enables organizations to continuously respond to market opportunities and challenges and thereby sustain competitive advantage. However, many organizations fail in their attempt to implement DDI due to poor organizational readiness. This study investigates key factors that assist organizations in assessing their readiness for DDI. An extensive examination of literature was performed to identify readiness factors. The results highlighted nine organizational readiness factors for DDI based on the theoretical foundations of Technology-Organization and Environment framework and organizational readiness theory. The findings of this study contribute to the growing body of DDI literature and provide insights for organizations interested in implementing DDI initiatives

Information Technology decision makers’ readiness for artificial intelligence governance in institutions of higher education in South Africa

Artificial intelligence (AI) can enhance the educational experience for academics and students. However, research has inadequately examined AI ethics and governance, particularly in the higher education sector of developing economies such as South Africa. AI governance ensures that envisioned AI benefits are realized while reducing AI risks. Against this backdrop of huge research deficit, the current study reports on a qualitative exploratory study that investigates the state of readiness for AI governance and AI governance maturity in South African higher education institutions. Informed by the combination of the TOE framework, the traditional IT governance model and the adapted IT governance maturity assessment model, semi-structured interviews were conducted with academic and ICT decision makers from two public and three private higher education institutions in South Africa to determine their insights on the state of readiness and maturity of AI governance. Results reveal high proliferation of AI elements in higher education information systems. However, results revealed low levels of AI governance readiness by higher education institutions. The study recommends for recognition of AI risks and taking lessons from AI regulatory frameworks advanced in developed countries

Uncovering Cultural Differences in Organizational Readiness for Artificial Intelligence: A Comparison between Germany and the United States

Artificial Intelligence (AI) transforms the business world by enabling organizations to leverage new business opportunities through its unique capabilities of self-learning and autonomous decision-making. To unlock the disruptive potential of AI, organizations seek to implement AI applications throughout their business landscape. However, from a cross-cultural perspective, national culture can influence the way organizations implement AI applications. To better understand cross-cultural differences on AI adoption, our study combines Hofstede’s national cultural framework with the organizational readiness concept for AI. We examined the moderating role of Hofstede’s national cultural dimensions on the organizational readiness factors of AI-process fit, financial resources, upskilling, collaborative work, and data quality. By conducting a multi-group analysis, we aim to identify national cultural differences between Germany and the US in AI adoption

Exploring artificial intelligence adoption in public organizations: a comparative case study

Despite the enormous potential of artificial intelligence (AI), many public organizations struggle to adopt this technology. Simultaneously, empirical research on what determines successful AI adoption in public settings remains scarce. Using the technology organization environment (TOE) framework, we address this gap with a comparative case study of eight Swiss public organizations. Our findings suggest that the importance of technological and organizational factors varies depending on the organization’s stage in the adoption process, whereas environmental factors are generally less critical. Accordingly, this study advances our theoretical understanding of the specificities of AI adoption in public organizations throughout the different adoption stages

Ready or Not, AI Comes— An Interview Study of Organizational AI Readiness Factors

Artificial intelligence (AI) offers organizations much potential. Considering the manifold application areas, AI’s inherent complexity, and new organizational necessities, companies encounter pitfalls when adopting AI. An informed decision regarding an organization’s readiness increases the probability of successful AI adop- tion and is important to successfully leverage AI’s business value. Thus, companies need to assess whether their assets, capabilities, and commitment are ready for the individual AI adoption purpose. Research on AI readiness and AI adoption is still in its infancy. Consequently, researchers and practitioners lack guidance on the adoption of AI. The paper presents five categories of AI readiness factors and their illustrative actionable indicators. The AI readiness factors are deduced from an in-depth interview study with 25 AI experts and triangulated with both scientific and practitioner literature. Thus, the paper provides a sound set of organizational AI readiness factors, derives corresponding indicators for AI readiness assessments, and discusses the general implications for AI adoption. This is a first step toward conceptualizing relevant organizational AI readiness factors and guiding purposeful decisions in the entire AI adoption process for both research and practice

Re-thinking the Competitive Landscape of Artificial Intelligence

In recent years, Artificial Intelligence (AI) has emerged from its traditional domain of computer science research to be a management reality. This can be seen in the remarkable increase in the adoption of AI technology in organisations resulting in increased revenue, reduced costs and improved business efficiency [19]. Despite this trend, there are still many organisations that are facing the decision whether to adopt AI. Thus, to evaluate the adoption of AI at organizational-level we draw on two-grounded theories: Technology-Organisations-Environment (TOE) framework and Diffusion of Innovation theory (DOI) to identify factors that influence the adoption of AI. Survey data collected from 208 large, medium-sized and small organisations in Australia is used to test the proposed framework. We offer a method of how examining AI over a set of organizations. Besides offering a number of important recommendations for AI adoption future directions for research in this area are also included in this paper