Learning Analytics for Academic Writing through Automatic Identification of Meta-discourse

Effective written communication is an essential skill which promotes educational success for undergraduates. Argumentation is a key requirement of successful writing, which is the most common genre that undergraduates have to write particularly in the social sciences. Therefore, when assessing student writing academic tutors look for students’ ability to present and pursue well-reasoned and strong arguments through scholarly argumentation, which is articulated by meta-discourse. Today, there are some natural language processing systems which automatically detect authors’ rhetorical moves in scholarly texts. Hence, when assessing their students’ essays, educators could benefit from the available automated textual analysis which can detect meta-discourse. However, previous work has not shown whether these technologies can be used to analyse student writing reliably. The aim of this thesis therefore has been to understand how automated analysis of meta-discourse in student writing can be used to support tutors’ essay assessment practices. This thesis evaluates a particular language analysis tool, the Xerox Incremental Parser (XIP) as an exemplar of this type of automated technology. The studies presented in this thesis investigates how tutors define the quality of undergraduate writing and suggests key elements that make for good quality student writing in the social sciences, where XIP seems to work best. This thesis also sets out the changes that needs to be made to the XIP and proposes in what ways its output can be delivered to tutors so that they make use of this output to give feedback on student essays. The findings reported also show problems that academic tutors experience in essay assessment, which potentially could be solved by automated support. However, tutors have preconceptions about the use of automated support. The study revealed that tutors want to be assured that they retain the ‘power’ themselves in any decision of using automated support to overcome these preconceptions

Ethics in AIED: Who cares?

The field of AIED raises far-reaching ethical questions with important implications for students and educators. However, most AIED research, development and deployment has taken place in what is essentially a moral vacuum (for example, what happens if a child is subjected to a biased set of algorithms that impact negatively and incorrectly on their school progress?). Around the world, virtually no research has been undertaken, no guidelines have been provided, no policies have been developed, and no regulations have been enacted to address the specific ethical issues raised by the use of Artificial Intelligence in Education. This workshop, ETHICS in AIED: Who Cares?, is proposed as a first step towards addressing this critical problem for the field. It will be an opportunity for researchers who are exploring ethical issues critical for AIED to share their research, to identify the key ethical issues, and to map out how to address the multiple challenges, towards establishing a basis for meaningful ethical reflection necessary for innovation in the field of AIED. The workshop will be in three parts. It will begin with ETHICS in AIED: What’s the problem?, a round-table discussion introduced and led by Professor Beverly Woolf, one of the world’s most accomplished AIED researchers. This will be followed by Mapping the Landscape, in which up to six AIED conference participants will each give a five-minute ‘lightning’ presentation on ethics in AIED research. The workshop will conclude with Addressing the Challenges, a round-table discussion session in which we will agree on a core list of ethical questions/areas of necessary research for the field of AIED, and will set out to identify next steps

Critical perspectives on writing analytics

Writing Analytics focuses on the measurement and analysis of written texts for the purpose of understanding writing processes and products, in their educational contexts, and improving the teaching and learning of writing. This workshop adopts a critical, holistic perspective in which the definition of "the system" and "success" is not restricted to IR metrics such as precision and recall, but recognizes the many wider issues that aid or obstruct analytics adoption in educational settings, such as theoretical and pedagogical grounding, usability, user experience, stakeholder design engagement, practitioner development, organizational infrastructure, policy and ethics

Teaching with Generative AI: moving forward with content creation

Generative AI is widely available and has raised the expectation that it will impact Education. Models, such as ChatGPT, can quickly produce plausible texts on a wide range of topics, and this capability may be of potential use in course content production. This paper selects several important course content production tasks, describes the prompts used, and assesses the quality of the automatically generated texts by a team of experts. Across all tasks, the Generative AI produced content that could help solve specific tasks by aiding with brainstorming, creating outlines, and adhering to particular writing guidance. In all cases, the content required adjustments and checking by human experts

Tools and Technologies for Advancing Professional Development

A large number of professional development studies have focused on the possible impact of professional development activities to improve practice and learning; for example, the impacts of the professional development on the learner (Buczynski & Hansen, 2010; Desimone, 2009; Ingvarson, Meiers, & Beavis, 2005). Other studies have looked at developing and implementing evaluation models to improve practice and learning (e.g. Guskey, 2000; Van Veen, Zwart, & Meirink, 2012). However, these approaches tend to focus on the end outcome of a process rather than a continue cycle for improvement, change and reflection. As Merchie et al. (2018) state, investigators and practitioners have been working towards making more innovative, thoughtful and informed ways to develop relevant, updated and personalised professional development initiatives that goes beyond processes (Desimone, 2009; King, 2014). Professional development can be viewed from both a top-down, organisational perspective and also from a bottom-up, life-long learning view. This presentation will show how these perspectives can be related through 3 key factors: Context, Philosophy and Delivery (CPD), which through a connection of top-down and bottom-up, leads to what we describe as the CPD2 cycle. A number of different delivery tools, methods and approaches have been used in the CPD2 cycle through the work of colleagues at the OU, such as games-based learning, evidence cafés, tricky topics, social media, video conferencing, personalisation and multimodal delivery. We focus on games-based learning, social media and multimodal delivery as examples of technologies that have produced positive impacts on professional development and professional digital learning

The CPD2 Change Cycle: Enabling development for Online Higher Education

The Covid-19 Pandemic has challenged Higher Education to change practices towards online provision requiring a huge shift in developing teachers, administrators and university management to understand these activities. The Institute of Educational Technology has a long history of providing academic development for the Open University in its 50 years as an international distance learning university. Key to our development practices to support change in crisis has been to use innovation, adaptability and a wholistic approach to professional development. In particular, the professional and personal development required to facilitate online learning is truly interdisciplinary and can be viewed both from a bottom-up, life-long learning or a top-down, learning organisation perspective. However, bottom-up innovation can suffer from poor sustainability and top-down strategies can have limited adaptation and relevance to contextual needs. This presentation provides a research-informed connection between these perspectives of change during crisis by pulling together the advances and research evaluations, often related to technology enhanced learning. These wholistic perspective reviews cross-disciplinary professional development activities that come from, or have relevance to, the Higher Education context. Through this work we have identified three key factors that are critical elements of professional development for developing online learning in a crisis: Context, Philosophy and Delivery (CPD). We have further identified, through connecting the individual and organisational level of change, a CPD2 cycle. This cycle can support effective implementation (from organisational to individual needs) and evaluation (from personal to contextual impacts) of professional development enablers and barriers. There is an evidenced account of delivery tools, methods and approaches that have been used within three CPD2 cycle case studies (online learning design and Tricky Topics, knowledge exchange and Evidence Cafés, and personalisation in online learning). The findings identify impacts from these professional development activities which are discussed with reference to ‘squaring the CPD2 cycle’ in crisis change situations (i.e. completing the impact evaluation side of the cycle). Finally, we conclude with insights for HE to help implement and advance their own philosophies and contexts for professional development to support developing online HE learning in a time of crisis

Working Across Time Zones in Cross-Cultural Student Teams

The ability to collaborate with fellow workers from different cultures on international projects is a key asset in today's job market. International projects add new dimensions to student teamwork. These types of projects give students the opportunity to participate in collaboration that is remote, cross-cultural, and linguistically challenging. This proposal examines an international term project completed by the computing students of Rose-Hulman Institute of Technology and Bilkent University

Use of Intuitive Tools to Enhance Student Learning and User Experience

Most user interfaces today present system functions by use of verbal or iconic symbols on static 2D menu pages organized in a hierarchical system. This is unnatural to all human beings and must be learned, thus being a barrier between the full use and understanding of computer systems. With this problem in mind we set out to build a website and collaborative application for International Children's Center (ICC) which could be used not only across languages but across ages as well. This effort was tested on a daily basis by a multinational team consisting of students from the United States and Turkey as well as children from the respective countries. This attention to usability will not only lead to an intuitive tool for the client but also teach the students in the course how to build intuitive user interfaces

XIPIt: Updating the XIP Dashboard to Support Educators in Essay Marking at Higher Education

Effective written communication is an essential skill which promotes educational success for undergraduates. However, undergraduate students, especially those in their first year at university, are unused to this form of writing. After their long experience with the schoolroom essay, for most undergraduates academic writing development is painstakingly slow. Thus, especially those with poor writing abilities, should write more to be better writers. Yet, the biggest impediment to more writing is that overburdened tutors would ask limited number of drafts from their students. Today, there exist powerful computational language technologies that could evaluate student writing, saving time and providing timely, speedy, reliable feedback which can support educators marking process. This paper motivates an updated visual analytics dashboard, XIPIt, to introduce a set of visual and writing analytics features embedded in a marking environment built on XIP output