The effect of the prompt on writing product and process: a mixed methods approach

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of PhilosophyThe aim of this thesis is to investigate the effect of the writing prompt on test takers in terms of their test taking processes and the final written product in a second language writing assessment context. The study employs a mixed methods approach, with a quantitative and a qualitative strand. The quantitative study focuses on an analysis of the responses to six different writing prompts, with the responses being analyzed for significant differences in a range of key textual features, such as syntactic complexity, lexical sophistication, fluency and cohesion. The qualitative study incorporates stimulated recall interviews with test takers to learn about the aspects of the writing prompt that can have an effect on test taking processes, such as selecting a prompt, planning a response, and composing a response. The results of the quantitative study indicate that characteristics of the writing prompt (domain, response mode, focus, number of rhetorical cues) have an effect on numerous textual features of the response; for example, fluency, syntactic complexity, lexical sophistication, and cohesion. The qualitative results indicate that similar characteristics of the writing prompt can have an effect on how test takers select a prompt, and that the test time constraint interacts with the prompt characteristics to affect how test takers plan and compose their responses. The topic and the number of rhetorical cues are the prompt characteristics that have the greatest effect on test taking processes. The main conclusion drawn from the study findings are that several prompt characteristics should be controlled if prompts are to be considered equivalent. Without controlling certain prompt characteristics, both test taking processes and the written product will vary as a result of the prompt. The findings raise some serious questions regarding the inferences that may legitimately be drawn from writing scores. The findings provide clear guidance on prompt characteristics that should be controlled to help ensure that prompts present an equivalent challenge and opportunity to test takers to demonstrate their writing proficiency. This thesis makes an original contribution to the second language writing assessment literature in the detailed understanding of the relationships between specific prompt characteristics and textual features of the response

University entrance language tests : a matter of justice

University entrance language tests are often administered under the assumption that even if language proficiency does not determine academic success, a certain proficiency level is still required. Nevertheless, little research has focused on how well L2 students cope with the linguistic demands of their studies in the first months after passing an entrance test. Even fewer studies have taken a longitudinal perspective. Set in Flanders, Belgium, this study examines the opinions and experiences of 24 university staff members and 31 international L2 students, of whom 20 were tracked longitudinally. Attention is also given to test/retest results, academic score sheets, and class recordings. To investigate the validity of inferences made on the basis of L2 students' scores, Kane's (2013) Interpretation/Use Argument approach is adopted, and principles from political philosophy are applied to investigate whether a policy that discriminates among students based on language test results can be considered just. It is concluded that the receptive language requirements of university studies exceed the expected B2 level and that the Flemish entrance tests include language tasks that are of little importance for first-year students. Furthermore, some of the students who failed the entrance test actually managed quite well in their studies - a result that entails broad implications concerning validation and justice even outside the study's localized setting

The Relationship Between Vocabulary Depth and Argumentative Essay Writing Proficiency Among IELTS Test Takers in Pakistan

The main object of this correlational study was to determine the relationship between vocabulary depth and argumentative essay writing proficiency among Pakistani IELTS test takers. Students with greater vocabulary knowledge and discourse expertise would produce texts of higher quality as compared to students with less of these skills. For this purpose, it manipulated three instruments for collecting the required data: a prompt for writing an argumentative essay, an IELTS band descriptor for scoring writing tasks, and the Word Association Test (WAT) for assessing vocabulary depth knowledge. It involved 129 Pakistani IELTS test-takers as its population. The sample comprised 42 females and 87 male participants. The data was analysed using SPSS version 23. Pearson\u27s product moment analysis was used to determine the relationship between both continuous variables (vocabulary depth and argumentative essay writing), and the results revealed a negative or weak relationship between vocabulary depth and IELTS essay band scores (r = -.032, p =.721). The current findings concluded that there was no statistically significant correlation between both variables. Furthermore, this research study would have certain suggestions for teachers, test designers, and material originators on how to incorporate the dimension of word associations into the creation of word knowledge. It would have some pedagogical implications and instructions for increasing linguistic aptitude levels, which would be useful in academic institutions, language assessment, and language teaching and learning centres such as IELTS training centers. &nbsp

Evaluación de la escritura académica: una experiencia genérica

EFL/ESL researchers and teachers usually find the assessment of writing as a serious challenge, especially when several writing tasks are demanded in a curriculum. Having this in mind, this study aims to see through assessment priorities given to undergraduates’ writing performances. For this aim, seventy Iranian university instructors with different teaching backgrounds were requested to assess two separate writing tasks entailing different rhetorical demands. Research findings disclosed important details of undifferentiated assessment approaches adopted by instructors. Despite a quantitatively significant difference in comments on content richness and leniency toward discourse, several writing constructs were accorded the same level of importance whatever the writing task type. This study casts fresh light on inadequacies in academic writing assessment and offers fruitful information to curriculum developers to consider a serious rethinking of language teachers education to sharpen teachers’ assessment skills.Los investigadores y profesores EFL/ESL ven la evaluación de la escritura como un reto, en particular cuando se exigen varias tareas en un currículum. Teniéndolo en cuenta, ese estudio apunta cómo establecer las prioridades de evaluación de las destrezas de escritura universitarias. Para alcanzar ese objetivo, se pidió a setenta profesores universitarios iraníes, con diferentes experiencias de enseñanza, evaluar dos tareas de escritura diferentes implicando varias preguntas retóricas. El resultado de la investigación reveló detalles importantes por lo que concierne a la aproximación indiferenciada de la evaluación adoptada por los profesores. No obstante se observó una diferencia cualitativa significativa en los comentarios sobre la riqueza del contenido y la indulgencia hacia el discurso, ya que se dio el mismo nivel de importancia a cualquier tipo de tarea escrita. Ese estudio arroja luz sobre la inadecuación de la evaluación de los escritos académicos y da información fructífera a los que redactan un currículum proponiendo una reflexión sobre la educación de los profesores de idiomas extranjeros, al fin de aclarar las habilidades de evaluación de los profesores

Patterns of debate in tertiary level asynchronous text-based conferencing

Argumentation can be defined at different levels and serve different purposes, but its role in knowledge understanding and construction has given it a central place in education, particularly at tertiary level. The advent of computer-supported text-based conferences has created new sites where such educational dialogues can take place, but the quality of the interaction and whether it is serving its educational purpose is still uncertain. This paper reports on a framework of analysis that has been developed to illuminate the arguing process within an asynchronous electronic conferencing environment, showing how it is both similar to, and different from, argumentation in the more traditional forums of multi-party, face-to-face discussion and traditional written essays. The framework develops earlier work by the authors and is applied to two electronic conferences within the same postgraduate course, comparing overall patterns of argumentation. Findings are presented on the extent to which the technology of electronic conferencing shapes and supports students’ participation in academic literacy practices relating to argumentation, proposing, at the same time, that the teaching strategy adopted by the lecturer is also an important variable

Using keystroke logging to understand writers’ processes on a reading-into-writing test

Background Integrated reading-into-writing tasks are increasingly used in large-scale language proficiency tests. Such tasks are said to possess higher authenticity as they reflect real-life writing conditions better than independent, writing-only tasks. However, to effectively define the reading-into-writing construct, more empirical evidence regarding how writers compose from sources both in real-life and under test conditions is urgently needed. Most previous process studies used think aloud or questionnaire to collect evidence. These methods rely on participants’ perceptions of their processes, as well as their ability to report them. Findings This paper reports on a small-scale experimental study to explore writers’ processes on a reading-into-writing test by employing keystroke logging. Two L2 postgraduates completed an argumentative essay on computer. Their text production processes were captured by a keystroke logging programme. Students were also interviewed to provide additional information. Keystroke logging like most computing tools provides a range of measures. The study examined the students’ reading-into-writing processes by analysing a selection of the keystroke logging measures in conjunction with students’ final texts and interview protocols. Conclusions The results suggest that the nature of the writers’ reading-into-writing processes might have a major influence on the writer’s final performance. Recommendations for future process studies are provided

Understanding the EAP writing and online learning needs of Chinese university students: A multiple-method needs assessment case study

The development of educational partnerships between U.S. and Chinese universities looking to internationalize is leading to a growing demand for online English language courses for students seeking to improve their English prior to U.S. arrival. The purpose of this study was to identify the current English for Academic Purposes writing (EAPW) and online learning needs of the students from a major Chinese university. A multiple-source/multiple-method approach to data collection was implemented. The results showed that half of the participating Chinese students were ready for basic EAPW and the other half for first-year EAPW courses. Although most of the students in the study were already exposed to (mostly passive) online learning practices, they would need to be taught interactive and collaborative online learning techniques in order to perform well in an online EAPW course. The study found that the Chinese participants have good command over some important EAPW features, especially when these occur in familiar tasks. Overall, the study suggests that EAPW course designers do not need to overhaul their EAPW curricula, but rather shift their focus to incorporating and scaffolding culturally-sensitive assignments, interaction, and technical support