Taylor, Lynda

English

University of Bedfordshire Repository

⚫CRELLALynda TaylorCRELLA, University of Bedfordshire, UKCentre for Research in English Language Learning and AssessmentFraming academic literacy:considerations and implications for language assessmentSome questions to start us thinking…Q1: What is understood by the term Academic Literacy? Q2: How do language tests commonly used for university admission reflect this understanding? Q3: How might such tests evolve to be fit for purpose in the future?2Presentation overview1. Approaches to defining a construct of Academic Literacy – differing paradigms that have emerged over recent decades2. Approaches to operationalising the construct – as reflected in different tests commonly used for admission to higher education3. Approaching the future – in light of developments in pedagogy and advances in technology3‘ assessing academic literacy: the work of a lifetime’4Weir and Chan, forthcomingResearch and practice in assessing academic reading: the case of IELTSCyril J Weir and Sathena ChanStudies in Language Testing, CUP51. DEFINING A CONSTRUCT OF ACADEMIC LITERACY63 possible paradigms?1. The generic study skills approach2. The genre-specific approach3. The socially situated discourse approach71. A generic study skills approachFrom the 1970s, English for Academic Purpose (EAP) teachers approached the teaching of academic literacy through the traditional building blocks of general (high-level) reading and writing skills…Academic language proficiency is ... the literacy of the educated, based on the construct of there being a general language factor relevant to all those entering higher education, whatever specialist subject(s) they will study.Davies (2008:113)8“What is academic language proficiency?”Davies (2008) addresses this question in his book on testing English for academic purposes: • academic proficiency involves performing the appropriate discourse generalizable across all disciplines• the core components of skilled literacy are: argument, logic, implication, analysis, explanation, reporting• the case for a single general approach relevant to all those entering higher education, whatever their area of specialism  9A complementary approach in testing• communicative language testing (CLT) supports the argument against discipline-specific modularity• test the skills and features underpinning communicative behaviour, i.e. the abilities rather than the behaviour itself• a single test of Academic English based on cross cutting EAP skills10Test in English for Academic Purposes (TEAP)• Research: survey of academic activities in UK tertiary-medium education, esp problems encountered with language-related study skills (Weir, 1983)• Sample: 940 overseas students, 530 British students and 560 academic staff; 43 postgraduate courses, 61 undergraduate courses and 39 ‘A’ level • Findings: considerable overlap in reading activities and writing activities across academic levels and across subject boundaries• Outcome: Associated Examining Board (AEB) development of generic language-based study skills test - TEAP112. A genre-specific approach• growing interest in discourse/genre analysis in the 1980s (Brown and Yule 1983, Swales 1990) • developments in corpus linguistics in the 1990s (Sinclair 1991, Biber 1993)• identification of diverse genres according to subject and context, incl. academic disciplines• does a genre-based (as opposed to a generic study skills) approach better prepare students for the discipline-specific demands of academic study? 12Lea and Street (1998)Academic literacy as involving: how to write specific, course-based knowledge for a particular tutor or field of study. Problems lie with a lack of familiarity with the subject matter of a particular discipline and how to write knowledge in that discipline, for example the need to abstract theory rather than attend to factual detail as evidence; deeper epistemological issues associated with knowledge in different disciplines. Lea and Street (1998:164)13Lillis (2003)Lillis characterises the genre-based approach as a form of ‘academic socialisation’ - with two dimensions: Language as discourse practices which learners will/must gradually come to learn implicitly. Socialisation (1) teaching as (implicit) induction into established discourse practices.Language as genres which are characterised by specific clusters of linguistic features. Socialisation (2) explicit teaching of features of academic genres.Lillis (2003:194)14Key researchESRC study: 'An Investigation of Genres of Assessed Writing in British Higher Education' (Project No. RES-000-23-0800) by Hilary Nesi, Sheena Gardner, Paul Thompson and Paul Wickens (2007):• comprehensive description of the genres encountered in academic study• first large-scale attempt to identify and describe the range of writing produced by university students for assessment purposes• students in British universities are required to produce a range of different genres of assessed written work, reflecting a range of rhetorical structures, communicative purposes and audiences 15Findings• clear cultural differences in academic literacy between discipline areas• existence of at least 13 genre families in academic writing - each with own stages of production• even greater diversity beyond that of genre families16… Undergraduate student writing is clearly complex, with many variations in practice dependent not only on discipline, level of study and educational approach, but also on the nature of the higher education institution, the particular focus of the department within that institution, and the idiosyncrasies of the lecturers who assign written work.Nesi and Gardner (2006:102)17Conclusions• academic literacy is more than just acquiring a set of basic, generic study skills• involves awareness of and increasing familiarization with the genres and modes of discourse characterising academic study within field or discipline (in terms of its discoursal and surface language features)• paradoxically, commonalities do exist, e.g. pervasiveness of the argumentative essay across disciplines + agreement on the assessment criteria for marking among academic tutors183. A socially situated approachCritical literacy adherents argue that we must go even further and consider academic discourse as socially situated, involving issues of power and authority (Lea and Street 1998, Murray 2016). 19Lea and Street (1998) Criticised the generic study skills approach because it: …takes no account of the interaction of the student with institutional practices and is based on the underlying principle that knowledge is transferred rather than mediated or constructed through writing practices... the implicit models that have generally been used to understand student writing do not adequately take account of the importance of issues of identity and the institutional relationships of power and authority that surround, and are embedded within, diverse student writing practices across the university... Lea and Street (1998:170)20Lillis (2003) Argues for considering a model based on socially situated discourse practices as well as the generic and disciplinary specific approaches:        The academic literacies frame has helped to foreground many dimensions to student academic writing which had previously remained invisible or had been ignored; these include the impact of power relations on student writing, the centrality of identity in academic writing, academic writing as ideologically inscribed knowledge construction …Lillis (2003:195)21• Lea and Street (1998) argue that the academic literacies model: ‘incorporates both of the other models into a more encompassing understanding of the nature of student writing within institutional practices, power relations and identities’.• According to this third approach, academic literacy embraces an additional dimension on top of generic and discipline-specific study skills: that of the socially situated nature of academic interaction.22To summarise….The theoretical and empirical literature appears to highlight a wide range of issues and considerations associated with defining a rich and comprehensive construct of academic literacy.  23Multiple elements• general study skills• general language abilities• generic activities/tasks• discourse practices• discipline-based genres• subject specificity• epistemological differentiation• author identity/stance• audience idiosyncrasies• processes of socialisation• institutional culture• departmental culture• level of study• power/authority dynamics242. OPERATIONALISING THE CONSTRUCT FOR ASSESSMENT PURPOSES25So…• To what extent do tests commonly used for academic purposes reflect features of academic literacy as articulated across the 3 paradigms highlighted?• To what extent should they?• To what extent can they?26Tests commonly used to sample and evaluate academic literacy skills for the purposes of university admissionA preliminary analysis for reflection and discussion…..(a purely personal perspective!)29IELTS CAE/CPE PTE - Acad TOEFL-IBTgeneral study skills √ √ √ √general language skills √ √ √ √generic activities/tasks √ √ √ √discourse practices √ √ √ √discipline based genres ? ? ? ?subject specificity ? ? ? ?epistemological differentiation ? ? ? ?author identity/stance X X X Xaudience idiosyncrasies X X X Xprocesses of socialisation X X X Xinstitutional culture X X X Xdepartmental culture X X X Xlevel of study X X X Xpower/authority dynamics X X X XSo are these tests fit for purpose?Current tests used for university entry are sometimes criticised for being a ‘blunt instrument’, i.e. failing to be sufficiently well-designed and sensitive for the purposes of academic admissions.In some respects this may be true: recent research has shown they do not always adequately sample some of the higher-level reading and writing skills that are generally considered relevant to academic literacy (issues of cognitive and context validity).30“Couldn’t students take an English language test based on the discipline area in which they intend to study, and therefore tailored accordingly?” A simple question?Or a naïve assumption?The challenge of predictive (or criterion-related) validity…• Is a test score to be interpreted as:(i) an indicator of ‘readiness-to-enter’ an academic domain?OR(ii) a predictor of future academic outcomes?• What skills are required for readiness to enter the academic domain, i.e. prior to entry?• What skills are most likely (or only) acquired within the academic domain post entry?32Generic EAP vs. discipline-specific testsA discipline-specific test might appear a logical option, but does it make sense if:a) we cannot assume that students will come equipped with adequate conversancy in the literacy practices of their future disciplines as a result of their diverse educational experiences? b) those literacy practices are actually part of the academic journey they are about to embark upon and will therefore be embedded within their future curriculum?  33Generic EAP vs. discipline-specific ‘academic literacies’ The high-profile gatekeeping tests that are currently employed by English-medium universities the world over focus on generic EAP and, as we have seen, this fails to take account of the particularity of literacy practices within specific disciplines and associated with an academic literacies perspective. Murray (2016:8)34Generic EAP vs. discipline-specific ‘academic literacies’ …[future academic] performance is largely dependent on students’ conversancy in those practices pertinent to their particular disciplines, conversancy in which, I have argued, we cannot assume or expect students to come equipped to university; hence the need to embed tuition in academic literacies within the curriculum.35The ‘burden of expectation’• Are we tempted to lay too great a burden of expectation on a test?• What is it realistic for us to expect in terms of score interpretation? • What other sources of useful evidence could be used alongside the test?• How might future test design evolve to meet changing requirements, processes and contexts?363. APPROACHING THE FUTURE? OPPORTUNITIES AND CAVEATS37The emergent nature of academic literacies?• The impact of a living and studying in a digital age• The changing nature of academic literacies due to technological advances– rapid growth of digitisation and digital learning– standardised online courses, MOOCs – access to and use of social media– cooperative learning networks and online learning management systems– use of electronic devices, e.g. tablets, smart phones– academic study no longer confined by presentation mode or physical location38The emergent nature of how academic literacy(ies) could be assessed?• The reshaping of assessment tools to harness new technologies• More than simply converting paper and pencil tests to the computer-based format• Options for: more integrated task formats, domain-specific activities, collection of ‘big data’ from/about learners• Increasing personalisation of both learning and assessment, including feedback39Some caveats and questions for research• Research questions?– issues of test format comparability (e.g. print vs computer vs small screen)– impact of increased visual as well as text-based input in academic study (TED talks) – greater integration of word and image, blending of spoken and written modes– likelihood of differentiation – ‘digital immigrants’ vs ‘digital natives’• Ethical considerations?– impact of onscreen study and interactive devices on language processing– protection of personal data (e.g. biometric information)– potential for marginalisation – ‘digital immigrants’ vs ‘digital natives’– potential of artificial intelligence and cyber-physical combinations40A brave new world? or …41…..‘refreshments at Cambridge Assessment’….Cyril’s abiding interest in history …“For everything there is a season, and a time for every matter under heaven.”[Ecclesiastes 3.1]The astronomical clock in the old town of Prague“If I have seen further it is by standing on the shoulders of giants.”Words written in the 17th century by Sir Isaac Newton, mathematician, astronomer and physicist, in a letter to philosopher, architect and polymath, Robert Hooke Sadly missed, fondly remembered …⚫CRELLAThank you!Lynda TaylorCRELLA, University of Bedfordshire, UK

Framing academic literacy: considerations and implications for language assessment

http://uobrep.openrepository.com/uobrep/bitstream/10547/623720/2/taylor-assessing-academic-literacy-final.pdf

Framing academic literacy: considerations and implications for language assessment

Abstract

Similar works

Full text

Available Versions

University of Bedfordshire Repository