A Study on the Effectiveness of Automated Essay Marking in the Context of a Blended Learning Course Design

This paper reports on a study undertaken in a Chinese university in order to investigate the effectiveness of an online automated essay marking system in the context of a Blended Learning course design. Two groups of undergraduate learners studying English were required to write essays as part of their normal course. One group had their essays marked by an online automated essay marking and feedback system, the second, control group were marked by a tutor who provided feedback in the normal way. Their essay scores and attitudes to the essay writing tasks were compared. It was found that learners were not disadvantaged by the automated essay marking system. Their mean performance was better (p<0.01) than the tutor marked control for seven of the essays and showed no difference for three essays. In no case did the tutor marked essay group score higher than the automated system. Correlations were performed that indicated that for both groups there was a significant improvement in performance (p<0.05) over the duration of the course and that there was a significant relationship between essay scores for the groups (p<0.01). An investigation of attitude to the automated system as compared to the tutor marked system was more complex. It was found that there was a significant difference in the attitudes of those classified as low and high performers (p<0.05). In the discussion these findings are placed in a Blended Learning context

Testing academic literacy in reading and writing for university admissions

A thesis submitted to the University of Bedfordshire, in partial fulfilment of the requirements for the degree of Master of Arts by Research.Currently university entrance decisions are heavily reliant on further education qualifications and language proficiency tests, with little focus on academic literacy skills that are required to succeed at university. This thesis attempts to define what academic literacy skills are and to what extent they correlate with three measures of university success. To answer these two research questions, I first investigated what academic literacy skills are through a survey of the literature, university study skills websites and existing academic literacy tests, and from these results drew up a checklist for academic literacy test validation. I then attempted to validate a new academic literacy test through a mixed methods study: first by calculating the correlations between performance in this test and university grades, self-assessment and tutor assessment, then through a case study approach to investigate these relationships in more detail. My tentative findings are that, within the humanities and social sciences, the academic literacy test is likely to correlate strongly with university grades, both in the overall results and in two of the four marking criteria: coherence and cohesion, and engagement with sources, with some possibility of correlation in the argument criterion. The fourth criterion – academic language use – did not correlate, but this may be an effect of this particular participant sample rather than the test itself. I also suggest two areas that may be difficult to elicit under timed exam conditions: eliciting appropriate source use when sources are provided, and eliciting synthesis of ideas across two or more given sources

Conceptual Review of Literature on Student Plagiarism: Focusing on Nigerian Higher Education Institutions

This paper presents a conceptual review on student plagiarism focusing mainly on International postgraduate Nigerian students. The aim of this review is to provide an insight to issues that relate to the concept, which will present information for the Higher Education Institutions in Nigeria and those overseas where the students decide to further their studies. The paper reviews studies on eight themes: the origin of plagiarism, forms of plagiarism, possible consequences of student plagiarism, general views on student plagiarism, possible causes of student plagiarism, Methods of detecting, deterring and mitigating student plagiarism, and proposed solutions.The author concluded that a lot of Nigerian students struggle with the right perception of plagiarism and in most cases, do not understand the long-term consequences, besides the implementation of a holistic approach at managing student plagiarism, the higher institutions need to monitor and evaluate results and adapt measures to the institutional context. Also, there is a need for overseas universities to adjust their management framework in a way that will cater for international students. In addition, there is needfor more empirical studies to be carried out in Nigeria and other African Higher institutions

Unlocking the power of generative AI models and systems such asGPT-4 and ChatGPT for higher education

Generative AI technologies, such as large language models, have the potential to revolutionize much of our higher education teaching and learning. ChatGPT is an impressive, easy-to-use, publicly accessible system demonstrating the power of large language models such as GPT-4. Other compa- rable generative models are available for text processing, images, audio, video, and other outputs and we expect a massive further performance increase, integration in larger software systems, and diffusion in the coming years. This technological development triggers substantial uncertainty and change in university-level teaching and learning. Students ask questions like: How can ChatGPT or other artificial intelligence tools support me? Am I allowed to use ChatGPT for a seminar or final paper, or is that cheating? How exactly do I use ChatGPT best? Are there other ways to access models such as GPT-4? Given that such tools are here to stay, what skills should I acquire, and what is obsolete? Lecturers ask similar questions from a different perspective: What skills should I teach? How can I test students competencies rather than their ability to prompt generative AI models? How can I use ChatGPT and other systems based on generative AI to increase my efficiency or even improve my students learning experience and outcomes? Even if the current discussion revolves around ChatGPT and GPT-4, these are only the forerunners of what we can expect from future generative AI-based models and tools. So even if you think ChatGPT is not yet technically mature, it is worth looking into its impact on higher education. This is where this whitepaper comes in. It looks at ChatGPT as a contemporary example of a conversational user interface that leverages large language models. The whitepaper looks at ChatGPT from the perspective of students and lecturers. It focuses on everyday areas of higher education: teaching courses, learning for an exam, crafting seminar papers and theses, and assessing students learning outcomes and performance. For this purpose, we consider the chances and concrete application possibilities, the limits and risks of ChatGPT, and the underlying large language models. This serves two purposes: First, we aim to provide concrete examples and guidance for individual students and lecturers to find their way of dealing with ChatGPT and similar tools. Second, this whitepaper shall inform the more extensive organizational sensemaking processes on embracing and enclosing large language models or related tools in higher education. We wrote this whitepaper based on our experience in information systems, computer science, management, and sociology. We have hands-on experience in using generative AI tools. As professors, postdocs, doctoral candidates, and students, we constantly innovate our teaching and learning. Fully embracing the chances and challenges of generative AI requires adding further perspectives from scholars in various other disciplines (focusing on didactics of higher education and legal aspects), university administrations, and broader student groups. Overall, we have a positive picture of generative AI models and tools such as GPT-4 and ChatGPT. As always, there is light and dark, and change is difficult. However, if we issue clear guidelines on the part of the universities, faculties, and individual lecturers, and if lecturers and students use such systems efficiently and responsibly, our higher education system may improve. We see a greatchance for that if we embrace and manage the change appropriately

Digital writing technologies in higher education : theory, research, and practice

This open access book serves as a comprehensive guide to digital writing technology, featuring contributions from over 20 renowned researchers from various disciplines around the world. The book is designed to provide a state-of-the-art synthesis of the developments in digital writing in higher education, making it an essential resource for anyone interested in this rapidly evolving field. In the first part of the book, the authors offer an overview of the impact that digitalization has had on writing, covering more than 25 key technological innovations and their implications for writing practices and pedagogical uses. Drawing on these chapters, the second part of the book explores the theoretical underpinnings of digital writing technology such as writing and learning, writing quality, formulation support, writing and thinking, and writing processes. The authors provide insightful analysis on the impact of these developments and offer valuable insights into the future of writing. Overall, this book provides a cohesive and consistent theoretical view of the new realities of digital writing, complementing existing literature on the digitalization of writing. It is an essential resource for scholars, educators, and practitioners interested in the intersection of technology and writing

Game of Tones: Faculty detection of GPT-4 generated content in university assessments

This study explores the robustness of university assessments against the use of Open AI's Generative Pre-Trained Transformer 4 (GPT-4) generated content and evaluates the ability of academic staff to detect its use when supported by the Turnitin Artificial Intelligence (AI) detection tool. The research involved twenty-two GPT-4 generated submissions being created and included in the assessment process to be marked by fifteen different faculty members. The study reveals that although the detection tool identified 91% of the experimental submissions as containing some AI-generated content, the total detected content was only 54.8%. This suggests that the use of adversarial techniques regarding prompt engineering is an effective method in evading AI detection tools and highlights that improvements to AI detection software are needed. Using the Turnitin AI detect tool, faculty reported 54.5% of the experimental submissions to the academic misconduct process, suggesting the need for increased awareness and training into these tools. Genuine submissions received a mean score of 54.4, whereas AI-generated content scored 52.3, indicating the comparable performance of GPT-4 in real-life situations. Recommendations include adjusting assessment strategies to make them more resistant to the use of AI tools, using AI-inclusive assessment where possible, and providing comprehensive training programs for faculty and students. This research contributes to understanding the relationship between AI-generated content and academic assessment, urging further investigation to preserve academic integrity

A Study on Teacher Feedback and AES Feedback in Chinese College students’ English Writings

Feedback holds significant importance in second language writing instruction, as evidenced by numerous studies indicating its direct or indirect impact on the quality of learners’ written work. Despite this recognition, there has been a dearth of comprehensive research concerning the distinct influences of teacher feedback and Automated Essay Scoring (AES) feedback on the writing quality of learners. Addressing this gap, the present study employs a mixed-method approach, integrating both qualitative and quantitative methodologies. By meticulously examining AES feedback, teacher feedback, writing revision logs, and conducting interviews, this investigation identifies noteworthy differentials in the multidimensional aspects of writing quality attributed to these two distinct feedback modalities. Primarily, within the realm of syntactic complexity, lexical richness, fluency, and accuracy, the cohort exposed to teacher feedback demonstrated notably superior performance relative to their AES feedback counterparts. Secondarily, an assessment of revised text quality revealed compelling insights. The ultimate version of the text, stemming from the AES feedback group’s iterative revisions, exhibited marked enhancements in terms of accuracy, total word count, and average word length. In contrast, the initial and final drafts of the teacher feedback group unveiled discernible disparities in vocabulary intricacy, accuracy, total word count, and average word length. Evidently, while the ultimate version did not witness a significant surge in average word length or total word count, it showcased heightened vocabulary sophistication and enhanced accuracy in relation to the initial draft. This study underscores the value of judiciously deploying these two categories of feedback within the landscape of writing instruction. The nuanced benefits of each feedback type can be strategically harnessed to suit distinct writing contexts, thereby augmenting the caliber of learners’ written compositions

The prosody underlying spoken language proficiency : Cross-lingual investigation of non-native fluency and syllable prominence

Prosodic structures are one of the most challenging features for second or foreign language (L2) speakers to learn. Since prosody is also crucial for speech intelligibility and fluency, the ability to quantify language learners' proficiency in terms of prosody can be of use not only to language teaching but also to the developers of language testing and assessment methods or tools. This doctoral dissertation explores non-native prosody with new multidisciplinary methods and cross-lingual research data. The focus is on investigating the relations between the assessment of prosodic proficiency and fluency-related temporal features as well as syllable-level prominence realizations. This dissertation presents three original publications (Studies I-III). In these studies, the relations of the selected prosodic features to human assessments are investigated from Finland Swedish as an L2 (produced by Finnish speaking students) and from L2 English produced by Czech, Slovak, Hungarian, and Polish speakers. Objective temporal fluency features are measured based on previous research on L2 speech fluency. In addition, a state-of-the-art method based on continuous wavelet transform (CWT) is used for estimating syllable prominence. All analyzed speech data were assessed using the Common European Framework of Reference (CEFR) scale for prosodic proficiency. The results of Study I and III indicate that articulation rate and certain types of disfluencies in speech can reliably predict the perceived prosodic proficiency level regardless of the language context. However, results from Study I reveal that assessors seem to weigh temporal features differently depending on the speech type (read vs. spontaneous) as well as their individual foci. Study II provides promising results on the use of CWT-based prominence estimation in predicting L2 proficiency. Correlations of prominence estimates for L2 utterances with estimates for native speakers' corresponding productions were used as a predictive measure, and the the level of agreement conceptualized this way correlated significantly with the human assessments of prosodic proficiency. In Study III, manually annotated temporal fluency measures were compared to CWT-based prominence estimates as predictors of prosodic proficiency. Temporal measures served as more reliable predictors of prosodic proficiency, but prominence measures provided a significant improvement to the prediction of prosodic proficiency. The predictive power of the individual measures varied both quantitatively and qualitatively with respect to the speaker's first language (L1). In conclusion, this dissertation supports the earlier observations on the role of temporal fluency measures, especially articulation rate, in estimating L2 speaker's oral proficiency. The CWT method, in turn, revealed differences in the productions of L2 prominence with regard to speaker's L1 and thus provided complementary information for the prediction of prosodic proficiency. The acoustic features underlying L2 stress production should therefore be further studied with respect to speaker's L1. Furthermore, the speech type as well as speaker's L1 should be acknowledged in developing robust and reliable automatic spoken language learning and assessment tools.Tämä väitöskirja koostuu tutkimuksista, joissa selvitetään suullisen kielitaidon arvioinnin taustalla vaikuttavia puheen prosodisia piirteitä. Aiemmissa tutkimuksissa on havaittu, että prosodia – puheen intonaatio, painotus ja rytmi – on kielenoppijoille yksi haastavimmista kielitaidon osa-alueista. Samalla prosodian hallinnan on todettu olevan hyvin olennaista puheen ymmärrettävyydelle ja sujuvuudelle. Prosodisten piirteiden tutkiminen kielenoppijoiden puheesta auttaa kehittämään paitsi suullisen kielitaidon opetusta myös automaattisia arviointimenetelmiä. Väitöskirja tuo uutta tietoa kielenoppijoiden prosodiasta monikielisen aineiston avulla sekä esittelee uuden, aallokemuunnoksiin pohjautuvan puheen analyysimenetelmän, jota ei ole aiemmin käytetty kielenoppijan puheen tutkimisessa. Kolmessa osatutkimuksessa kielenoppijoiden puheesta analysoidaan sujuvuuteen liitettyjä temporaalisia piirteitä, kuten artikulaationopeutta ja tauotusta. Lisäksi analysoidaan sana- ja lausepainojen toteutumista aallokemuunnoksiin pohjautuvalla työkalulla. Akustisten parametrien yhteyksiä ihmisten tekemiin arvioihin tutkitaan logististen regressiomallien avulla kahdesta erikielisestä aineistosta: suomenkielisten puhumasta ruotsista (Tutkimukset I ja II) sekä tsekin-, slovakian-, puolan- ja unkarinkielisten puhumasta englannista (Tutkimus III). Tutkimusten I ja III tulokset vahvistavat temporaalisten sujuvuuspiirteiden kieliriippumatonta merkitystä suullisen kielitaidon objektiivisessa mittaamisessa. Lisäksi Tutkimus I osoittaa, että eri piirteiden merkitys riippuu sekä arvioijien yksilöllisistä mieltymyksistä että siitä, onko arvioitava puhe luettua vai spontaania. Tutkimus II puolestaan osoittaa, että aallokemuunnosten avulla mitattujen sana- ja lausepainojen toteutumilla voidaan ennustaa kielenoppijoiden prosodista taitotasoa. Tutkimuksessa III vertailtiin temporaalisten sujuvuuspiirteiden ja aallokemuunnoksella mitattujen sana- ja lausepainojen voimaa prosodisen taitotason ennustajina erikielisillä englanninoppijoilla. Tulokset osoittavat, että temporaaliset sujuvuuspiirteet ovat mitattuja sana- ja lausepainoja luotettavampia ennustamaan ihmisten antamia arvioita, mutta sana- ja lausepainojen huomioiminen parantaa tilastollisen mallin selitysvoimaa. Lisäksi tulokset osoittavat, että oppijan äidinkieli todennäköisesti vaikuttaa siihen, mitä keinoja kielenoppija käyttää sana- ja lausepainojen tuottamiseen. Tutkimustulosten perusteella artikulaationopeus on tärkein yksittäinen piirre kielenoppijan prosodisen taitotason arvioinnissa, ja tätä piirrettä voidaan käyttää myös suullisen kielitaidon automaattisessa arvioinnissa puhetyypistä ja kielikontekstista riippumatta. Sen sijaan tauotuksessa näyttää olevan erilaiset standardit luetussa ja spontaanissa puheessa. Lisäksi äidinkielen vaikutusta vieraan kielen painotusten tuottamiseen tulee tutkia entistä kattavammin, jotta tätä piirrettä voidaan luotettavasti käyttää kehittämään suullisen kielitaidon automaattista arviointia

Recognition of Bangladeshi Sign Language (BdSL) Words using Deep Convolutional Neural Networks (DCNNs)

In a world where effective communication is fundamental, individuals who are Deaf and Dumb (D&D) often face unique challenges due to their primary mode of communication—sign language. Despite the interpreters' invaluable roles, their lack of availability causes communication difficulties for the D&D individuals. This study explores whether the field of Human-Computer Interaction (HCI) could be a potential solution. The primary objective is to assist D&D individuals with computer applications that could act as mediators to bridge the communication gap between them and the wider hearing population. To ensure their independent communication, we propose an automated system that could detect specific Bangla Sign Language (BdSL) words, addressing a critical gap in the sign language detection and recognition literature. Our approach leverages deep learning and transfer learning principles to convert webcam-captured hand gestures into textual representations in real-time. The model's development and assessment rest upon 992 images created by the authors, categorized into ten distinct classes representing various BdSL words. Our findings show the DenseNet201 and ResNet50-V2 models achieve promising training and testing accuracies of 99% and 93%, respectively. Doi: 10.28991/ESJ-2023-07-06-019 Full Text: PD