Taylor, Lynda

English

University of Bedfordshire Repository

Exploring readers’ mental representation of textDr Lynda TaylorCRELLA Winter Research Seminar 15 December 2011Presentation overview Background The nature of reading comprehension The challenge for testers of reading An experimental study Methodology Analyses Discussion of results Some concluding commentsBackground Research into reading comprehension – L1 and L2, both product and process Research into the assessment of reading - especially test tasks that seek to access the product (i.e. the mental representation of text) Motivation: a discrepancy between accepted theories of reading comprehension, and general theory and practice in reading assessment  The nature of comprehension Recurring use of terms/ideas from 1970s onwards reflecting a process of ‘construction’ Ideational scaffolding (Anderson, 1977) Knowledge structures (van Dijk, 1980) Words as cues to build a mental model (Johnson-Laird, 1983) Structure building framework (Gernsbacher, 1990) Constructionist model of inference generation (Graesser, Singer and Trabasso, 1994)The nature of comprehension Cognitive psychologists and applied linguists seem to agree that text comprehension is an active and constructive process  in which meaning is generated by the cognitive processes of the reader using linguistic knowledge relevant to text content, as well as metacognitive skills, background knowledge, personal purpose/attitude, etc in order to construct a mental representation of the textThe nature of comprehension The construction metaphor is helpful, but… there is always a danger of seeing the mental representation as fixed and static when it is probably more likely to be fluid and flexible with the potential for being influenced and modified  at various stages during and after reading as a result of purpose for reading, integration of existing knowledge, and even the influence of test questionsThe challenge for testers How can we make the ‘invisible’ process and product of reading comprehension sufficiently ‘visible’ for the purposes of assessment?Possible approaches (1) Familiar reading task formats, e.g. MCQ, short answer, open Qs, all seek to make the reading process and product visible but…. these formats can risk interfering with the reader’s constructed mental representation – e.g. over-processingof the text? and this presents a challenge from a cognitive validityperspectivePossible approaches (2) An alternative approach is a summarisation task which can justifiably be regarded as authentic and visible evidence of comprehension but which can bring with it additional problems: compositional maturity extent of prior training nature of task instructions marking procedures ‘there is more to summarisation than adequate comprehension’A possible third approach? Is it possible to design an alternative test task format  that will address the mental representation constructed by the reader  but not require the reader to translate that mental representation into words?Summary completion format Task requires completion of a specially created gapped summary of a reading text The gapped summary is derived from a source text - but is not simply a version of the text with certain words deleted (i.e. it is NOT a cloze)  Operationally, the reading text is given first; after reading, the text is removed; readers then complete the gapped summary (drawing on their mental representation of the text)Summary completion format The specially designed gapped summary must ‘map onto’ a typical representation of the text provoked in readers  But how do we know what ‘most readers’ will construct by way of a ‘typical mental representation’ of a reading text?  We need to investigate the constructed mental representations of a text so we know what to include in a summary of the text, and what to ‘gap’ (as the test items)An experimental study Aims: 1. to explore aspects of readers’ mental representations of a given text in order to identify what might reasonably be included in an adequate summary of the text2. to construct suitable test items from the resulting summary for assessing readers’ comprehension3. to trial the constructed summary completion task on a population of readersAn experimental study Aims: 1. to explore aspects of readers’ mental representations of a given text [in order to identify what might reasonably be included in an adequate summary of the text]2. to construct suitable test items from the resulting summary for assessing readers’ comprehension3. to trial the constructed summary completion task on a population of readersMethodology (1) 2 short reading texts selected, representing 2 different text-types/genres Text A: short story narrative (Journey) – 526 words Text B: newspaper expository editorial (Anorexia) – 389 wordsMethodology (1) (for this talk) 2 short reading texts selected, representing 2 different text-types/genres Text A: short story narrative (Journey) – 526 words Text B: newspaper expository editorial (Anorexia) – 389 words(see handout – time to skim-read)Methodology (2) Collection of ORAL recalls of both texts from 30 mature (17+) L1 readers: ‘free’ recall phase and ‘prompted’ recall (probe questions) a short ‘filler’ task inserted between reading and recall activities  Collection of WRITTEN recalls of both texts gathered from 80 L1 teenagers (14-15), representing the full ability spectrumAnalyses Oral recalls : transcription of recall protocols (orthographic) analysis first of original reading text and then of the oral transcripts to examine presence of text-level propositions, summarising propositions and inferences Written recalls :  analysis to identify levels of quality of comprehension and recall, especially any evidence of comprehension breakdown Analyses (for this talk) Oral recalls : transcription of recall protocols (orthographic) analysis first of original reading text and then of the oral transcripts to examine presence of text-level propositions, summarising propositions and inferences Written recalls :  analysis to identify levels of quality of comprehension and recall, especially any evidence of comprehension breakdown Text-level propositions (1) Examples: TP04  the hour was late TP35  the car stopped  TP45  the man had been waiting TP68  would the driver attack him with that? TP71 the man’s heart beat so fast with fear thatA total of 89 possible TPs for Text 1 (Journey)Only 5 TPs recalled by 20/30 readers in free recall + an additional 4 TPs provoked thru’ prompted recallText-level propositions (2) 25/89 TPs were recalled by 15/30 readers thru’ free and prompted recall 28% TPs constitute a fairly coherent summary of events in the short story noticeable that TPs often seemed to cluster in twos or threes relating to specific elements of the story – a form of ‘spreading activation’ at work?Examples of TP clusters TP01  the man stood alone TP04  the hour was late TP10  the man wanted to get home TP84  no more night passengers for me TP85  exclaimed the driver TP86  with a sigh of relief the driver hurriedly moved offFrequency of TP recall (high) High-frequency TPs basic skeletal outline of narrative plus some specific details of the story relating to  setting: midnight, 10 miles away from home the problem: it was out of the question to walk that distance the car’s approach: headlights drawing near the man’s thoughts/actions: stepping into the road, touching the wallet again, stories of passengers being attacked the driver’s thoughts/actions: carressing the spanner, the taxi’s disappearance, no more night passengersFrequency of TP recall (high) Three particular locations in the story where high-frequency TPs cluster strongly: near the start of the narrative (scene-setting) in the middle of the story (partial resolution of the man’s dilemma) towards the climax of the story (as the man’s fear reaches its peak)A ‘trigger’ effect? Recall of one proposition sometimes seemed to activate recall of others, e.g. money seemed to activate wallet and checking his wallet he couldn’t find his money + well + he was looking for his wallet because + oh he kept going for his wallet all the way through the journey to make sure it was still there (010) the taxi’s moved off before he can pay any money + and also he’s checking the wallet the whole time + and the taxi moves off  (013)The structuring of information Other examples: checking the wallet and looking for something to protect himself being attacked and checking for his wallet So is information structured (or are information units clustered) around nodes in the reader’s mind, potentially also linked to other nodes, with the potential for spreading semantic activation?Frequency of TP recall (low) Low-frequency TPs:  unimportant/non-salient details? inhibitory effect of more salient items?  repetition effect, i.e. recurring features in the story? certain content features that do not register in the mental representation or are not activated during the recall process? In this text, such content points appeared to include reported thoughts and feelings (rather than actions, urges or strong emotions So are these details of the narrative which can reasonably be omitted when constructing a summary of the text?Individual differences Integration of ‘unfamiliar’ elements into mental representation, e.g. Valencia Some readers clearly understood this as the name of a place to which the man wished to go (taxi-ride schema functioned effectively?) Others struggled to make sense of what this word meant (i.e. Valencia not recognised as a place-name?) and thus to integrate it into their mental representationSummarising propositions Examples: SP01  a man was present SP09  a car came along and stopped SP14  the man grew increasingly suspicious of the driverA total of 22 SPs at a higher level of text structure7 SPs (31%) offered by 20/30 readers in free recall - an accurate and partially coherent account of the story (though without descriptive setting at start)+ 2 additional SPs added thru’ prompted recallFrequency of SP recall (low) Relatively poor recall of more subtle emotional aspects of the story into mental representation memorable, story-specific elements were present (i.e. emotions central to understanding  of the story, such as early anxiety, growing fear and ultimate panic)  but less recall of emotional swings/shading – subsidiary to agents and action in a narrative? [NB implications for testing?] distraction of unfamiliar words, e.g. pall, mirthAdditional propositions Presence of material in recalls that could not be matched to corresponding TPs or SPs - evidence of inferencing? Distinction between inferences  that are necessary to support local and global coherence (‘retrospective’ or ‘bridging’ inferences) that are elaborative in nature, not essential for coherence but which aid or ‘enrich’ comprehensionNecessary inferences superordinate goals keeps checking that he’s got his wallet for money (009) kept on checking to see if his wallet was in his pocket (016)thematic inferences (a key point in this text) he’s a bit paranoid that um someone’s going to attack him (003) thought he was going to get robbed (021) could be he was going to get mugged (027)Elaborative inferences (1) inferences about the main character (v individual in nature) an adult male (21/30) (a man, a bloke, a guy), a young man, a middle-aged man, a boy, a little boy (variable instantiation) inferences about the location it’s somewhere in civilisation because of the dustbin (008) there was a dustbin so it’s probably an urban setting (012) a dark lane + like and alleyway + um + like you see in a cartoon (024)Elaborative inferences (2) inferences about the setting/atmosphere I have the impression that it was misty (015) it goes on about how dark it is + how cold he is (016) I think he was cold + well he was sort of standing swaying to and fro + stepping from side to side (002) inferences about why the main character was there has he missed the bus (011) this person who’s obviously just been out for the night (019)Elaborative inferences (3) inferences about where the man sat inside the car inferences about whether the car was a taxi or not inferences about where the spanner was inferences about whether the passenger paid the driverThe nature of inferences Brown and Yule’s  (1983) pragmatic definition of inferences as:The ‘connections people make when attempting to reach an interpretation of what they read or hear’  (p 265)A process which is ‘context-dependent, text-specific and located in the original reader (hearer)’  (p 266)Some concluding comments The relative salience of content material The possible structuring of information via nodes and clusters The relative dominance of agents and actions over emotional content/colouring The role and value of inferences Potential implications for constructing an adequate summary of the original text and then selecting words/phrases to construct as test items in the gapped summary?Analysis of written recallsLevel Description No of recallsLevel 1 A very general statement of content in one or two sentences ands including a collection of discrete facts from one or two parts of the text often in verbatim form. A noticeable lack of coherence.7 (8.97)Level 2 A partial account with some level of detail often in verbatim form. Chronological ordering sometimes distorted and several sections of the text omitted. Generally characterised by at least one major comprehension error, e.g. he paid the driver, the 31 (39.74)driver attacked the man, the passenger reached for the spanner, the wallet was missing. Level 3 Accurate account of text content. Good chronological ordering. Good level of detail and coherence, but often characterised by the omission of a particular section of the text, e.g. the man’s unsuccessful attempt to pay, the driver’s perspective at the end. Evidence of paraphrasing.25 (32.05)Level 4 Full and accurate account of the text content. Good chronological ordering. Substantial level of detail. High level of coherence. Extensive evidence of paraphrasing.15 (19.23)ReferencesAnderson, R C (1977) The notion of schemata and the educational enterprise. In R C Anderson and R J Spiro (eds) Schooling and the Acquisition of Knowledge, Hillsdale, NJ: Erlbaum.van Dijk, T A (1980) Macrostructures. Hillsdale, NJ, Erlbaum.Gernsbacher, M A (1990) Language Comprehension as Structure Building.  New Jersey: Erlbaum.Graesser, A C, Singer, M and Trabasso, T (1994) Constructing inferences during narrative text comprehension. Psychological Review, 101, 371-395.Johnson-Laird, P N (1983) Mental Models. Cambridge University Press.CRELLA

Exploring readers’ mental representation of text

http://uobrep.openrepository.com/uobrep/bitstream/10547/623711/2/crella-winter-seminar-dec2011-compatibility-mode.pdf

Exploring readers’ mental representation of text

Abstract

Similar works

Full text

Available Versions

University of Bedfordshire Repository