　　This research considers the use of Visual Narrative Grammar(VNG) as a means to aid in improving the fluency, accuracy, and complexity of dialogue written by ESL students at Japanese universities. VNG, such as the sequential images found in the panels of comic books, appeal to a non-verbal linguistic ability of the human mind (Gernsbacher, 1983; Cohn, 2013). If that be the case, then it could be hypothesized that VNG should have benefits in aiding language acquisition. This research seeks to explore the benefit of VNG on ESL students’ written production of English dialogue. In addition to a review of current literature pertaining to VNG, a proposed methodology of research to come has been outlined in this essay

Fallon Thomas J.

ベイカー マシュー

Nagoya Gakuin University Repository

Seeking the Effects of Visual NarrativeGrammar on the Written Dialogue Production ofESL Students at Japanese Universities : AProposed Experiment著者 ファロン トーマス J., ベイカー マシューjournal orpublication titleTHE NAGOYA GAKUIN DAIGAKU RONSHU; Journal ofNagoya Gakuin University; LANGUAGE and CULTUREvolume 28number 1page range 63-68year 2016-10-31URL http://doi.org/10.15012/00000777Copylight (c) 2016 Thomas J. FALLON, Matthew BAKER名古屋学院大学論集　言語・文化篇　第 28巻　第 1号　pp. 63―68― 63 ― Seeking the Effects of Visual Narrative Grammar on the Written Dialogue Production of ESL Students at Japanese Universities: A Proposed Experiment 〔Article〕  日本の大学におけるESL学生の文書対話を使ったVNGの効果について トーマス　J. ファロン・マシュー　ベイカー名古屋学院大学 / 九州産業大学発行日　2016年10月31日Abstract　 This research considers the use of Visual Narrative Grammar(VNG) as a means to aid in improving the fluency, accuracy, and complexity of dialogue written by ESL students at Japanese universities. VNG, such as the sequential images found in the panels of comic books, appeal to a non-verbal linguistic ability of the human mind (Gernsbacher, 1983; Cohn, 2013).  If that be the case, then it could be hypothesized that VNG should have benefits in aiding language acquisition.  This research seeks to explore the benefit of VNG on ESL students’ written production of English dialogue.  In addition to a review of current literature pertaining to VNG, a proposed methodology of research to come has been outlined in this essay.Keywords: VNG, ESL, language acquisition, intrinsic learning, psycholinguistics Thomas J. FALLON, Matthew BAKERNagoya Gakuin University / Kyushu Sangyou University ― 64 ―名古屋学院大学論集 　本稿は，日本の大学で第二言語として英語を学ぶ学生が書く文章が，正確さ，流暢さ，複雑さの点で上達する手段として，Visual Narrative Grammarの活用を考察したものである。VNGは，連続するコマ漫画のように，人の意識の非言語能力に訴えるとされており（Gernsbacher, 1983; Cohn, 2013)，そうであれば，VNGは言語の習得を支援するうえで有益だと考えられる。本研究は，英語を学ぶ学生により作られた英会話文に対して，VGNの有益性を検証する。本稿では，VNGに関する最近の文献レビューに加え，将来的な研究手法案についても示した。Introduction  　 From ancient Egyptian hieroglyphics to the Sunday comic strips, pictures conveying meaning have been a common mode of communication across a multitude of cultures throughout the world dating back to before written recorded history (Cohn, 2014; Wilkins, 1997).  In the field of linguistics, the comic strip and its panels have been extensively explored, revealing a conceptual tool known as Visual Narrative Grammar(VNG).  VNG suggests that sequential images take on narrative roles that embed within a constituent structure in visual narratives in much the same way that sequential words take on grammatical roles that embed within a constituent structure in sentences (Cohn 2013). The understanding of VNG and its implications may prove to be a useful tool in English as a Second Language(ESL) classrooms.  If this ability to understand a pictorial language is a common innate human faculty that can act as a bridge between languages, then it could prove exceedingly useful as an aid in the facilitation of 2nd language dialogue production.  In undertaking this research, we seek to explore the possible benefits of incorporating VNG in the ESL classroom with the goal of producing character dialogue.  Given that comic panels tend to focus on character dialogue, this approach to ESL activities could benefit learners in both writing and speech production.  It is the hypothesis of this exploratory research that incorporating VNG materials in the ESL classroom will prove beneficial, allowing for the resulting effects of improving learners’ accuracy, fluency, and linguistic complexity in written dialogue production.  Visual Narrative Grammar and Possible ESL Applications  　 As previously mentioned, the theory of Visual Narrative Grammar(VNG) posits that sequential images take on narrative roles that embed within a constituent structure in visual narratives in much the same way that sequential words take on grammatical roles that embed within a constituent structure in sentences (Cohn 2013).  This narrative structure assigns categorical roles to panels and orders them using hierarchical constituents, beyond the linear semantic relationship between images (Cohn 2016).  As we begin to explore this linguistic theory, it must be stressed that the comparison between narrative grammar and syntax is an analogy at the architectural level; images do not serve as verbs or nouns, and the information conveyed is experienced at a discourse level.  Nevertheless, Seeking the Effects of Visual Narrative Grammar on the Written Dialogue Production of ESL Students at Japanese Universities: A Proposed Experiment― 65 ―narrative grammar employs a similar structural architecture as does syntax, and it is believed that these constructs operate in comprehension similar to the processing of syntactic representations (Cohn 2014).  　 According to Cohn, VNG organizes sequences into divisions via basic narrative categories (2013). These basic categories are as follows:  Establishers passively introduce the relationship between entities;  Initials depict the beginning of an event or interaction;  Peaks show a climax; and  Releases depict an event resolution (Cohn 2016).  In the same way that linguistics research manipulates sentences, evidence for VNG arises from the manipulation of sequences (Cohn 2014).  Deletion or movement of panels from their original placement can have the effect of altering the intended meaning, or in some cases causing coherent meaning to be lost.  This is analogous to mixing and manipulating parts of speech within a linguistic sentence in order to change an intended meaning, or to test the limits of syntactical coherence.  Based on the research done by Cohn (2014), diagnostic results offer support for the presence of constituent structures, and provide the basis for manipulations in experimental research.  　 According to the theory of VNG, there are four key concepts that further illustrate its connection to linguistics, and through this explanation we will be able to further illuminate the usefulness of VNG in the ESL classroom.  The first of these key concepts is  narrative structure .  Narrative structure is the system that allows for meaning to be packaged at a discourse level.  By way of narrative structure, VNG determines categorical roles to be assigned to images based on prototypical correspondences including a conceptual structure of meaning (Cohn 2014).  This concept already has direct implications in ESL classrooms.  The use of pictures to assist in the transmission of 2nd language narratives is common among cultures internationally, as exemplified in the use of drawings or animations in advertisements, business presentations, and academic conferences, as well as in education materials and moreover popular entertainment (Nakazawa and Shwalb, 2012; Short et al., 2013).  　 The second key concept is  external compositional structure .  This concept highlights the physical layout of the comic panels and their pages, as well as the structures governing their organization. Most often these structures divide pages into vertical and horizontal components.  However, these components allow inset panels to be enclosed within a larger dominant panel, as well as gestalt relations such as overlapping, staggered, and separated panels (Cohn 2014).  This concept prospectively allows language learners the opportunity to develop a greater sensitivity to various textures and shades of meaning and/or emotion within a visual narrative.  　 The third key concept explored in VNG theory is that of  attentional framing structure .  This aspect addresses the limiting factors regarding how conceptual information gets framed into panels as well as the process of determining how much content they contain.  This has influence on how those images behave in a narrative, as well as how they are arranged in a page layout (Cohn 2014).  An example of an ESL activity utilizing an understanding of this concept could include panels with missing dialogue ― 66 ―名古屋学院大学論集for learners to determine and develop, or a task in which student must put disorganized VNG dialogue into a coherent order.　 In addition, a reoccurring struggle experienced by many second language educators is that of finding level appropriate yet engaging materials. Quite often adult students find themselves frustrated with the language materials deemed level appropriate by the educator. The educator is aften correct in their assessment of the students’ capabilities, but the subject matter is not on an age appropriate level, and thus not intellectually stimulating. With the incorporation of visual narration into the language acquisition process, it could allow a refreshing way for educators to tailor their choices of materials based on students’ age, interests, and cultural affiliation, as well as using popular media, in an effort to reinforce student engagement.　 A last point in regards to the efficacy of VNG on language acquisition stems from the simple connection between practice and familiarity with common conversational situations as formulated by the sequential VNG images.  This of course falls into the realm of reading comprehension as opposed to writing production, but a connection can certainly be found between both (Olness 2005).         　 Theoretically, these visual narrative images create a practically applicable conversational situation that the reader can consider and digest at their own pace, allowing for the student to try different character ego approaches and intonational variations.  Role playing character perspectives in this manner may prove both effective in improving students’ comprehension as well as spoken fluency and accuracy in a second language.  We intend to research this further and obtain solid data in an additional study. Proposed Methodology  　 In the interest of testing the effects of VNG on the writing and speech production of university students taking ESL coursework, we propose the following methodology.  Our sample size will be between 130 ― 170 students across three different universities.  The students’ skill levels will vary widely from quite low English ability to upper-intermediate and advanced, including both English majors and students majoring in other fields.  　 The proposed research will measure the effects of VNG on ESL learners’ writing capabilities. We will test the effects of VNG on the fluency, accuracy, and complexity of their writing. This will be done by utilizing two different timed writing exercises: a control exercise and a VNG experiment. The control exercise (Group 1) will be to write a narrative story using dialogue about a chosen topic. The control will be without the aid of visual cues, but will include the help of a written setting in order help the students develop a narrative and dialogue without a visual graphic. The experiment (Group 2) will be to write a narrative story using mainly dialogue about a chosen topic (same as control) with in the parameters of illustrated comic panels using images related to the same chosen theme as group 1. Seeking the Effects of Visual Narrative Grammar on the Written Dialogue Production of ESL Students at Japanese Universities: A Proposed Experiment― 67 ―The students’ production will then be analyzed for fluency by checking the word count and calculating the word per minute(WPM) rates. Both the control and experiment will be stopped after 10 minutes. The accuracy will be checked by measuring how many errors occur per clause. A grading rubric will be decided upon by the three universities’ teachers and then the students’ papers, control and experiment, will be checked twice for errors. The complexity will be checked by averaging how many clauses per C-unit were produced. 　 This experiment will hopefully give us insight into how much of an effect visual cues have on ESL learners’ writing, testing the V from VNG. But what about the N for narrative?  How much effect does the narrative aspect from VNG have upon students writing capabilities?  Is it merely the visual cues that have an effect on the learners’ brains and writing performance?  Or does the narrative grammar as well play a part in altering writing performance?  To test this there will be one more experiment exercise (Group 3). It will contain the exact same illustrated comic panels as group 2 but they will be mixed up from their original placement hopefully creating loss of coherent narrative meaning and testing the limits of syntactical coherence of VNG. 　 In all there will be three groups: a control (Group 1), an experiment (Group 2), and an experiment (Group 3). These three exercises will hopefully give us insight in to if VNG has any effect on ESL learners writing performance. And if so, how much of that performance has to do with the visual aspect of it and the narrative aspect of it. 　 Additionally, we hope to eventually further the scope of VGN/ESL research by observing students’ verbal progression in connection with the dialogues created in the aforementioned experiment via smartphone audio/visual recording technology.  Following the VNG writing production exercise, ESL educators will work with students to polish the comic panel dialogue into poper coherence.  Then, students will form groups and read the polished dialogue dramatically while being recorded on their personal smartphone device.  Next, students will review the video data of there speaking performance, taking notes on areas in need of improvement and considering the ESL educators’ advice.  This method of recording and evaluating ones’ speaking performance has show promising benefit in ESL education (Fallon ＆ Baker, 2015).  Furthermore, through this experiment we expect to find a increase in student engagement due the enjoyability of the activity media.  However, due to the scope of this immediate proposed research, verbal progression will be analyzed in a separate study. Conclusion  　 This research team seeks to determine the efficacy of Visual Narrative Grammar as applicable to the field of second language acquisition.  If it proves to be the case that there is an innate ability for humans to associate sequential images with a linear story or dialogue, then perhaps that ability could be harnessed to improve learners’ fluency, accuracy, and complexity in writing, as well as boost student ― 68 ―名古屋学院大学論集engagement and positively impact their speaking performance.  Applied experiments are currently underway to test these points of interest.  References  Crossley, S. A.; Salsbury, T.; McNamara, D. S.; Jarvis, S. (2011)  Predicting lexical proficiency in language learner texts using computational indices .  Language Testing, 28: 561 ― 580.  Cohn, N (2007) A Visual Lexicon.  The Public Journal of Semiotics, 1: 35 ― 56.  Cohn, N (2014)  The architecture of visual narrative comprehension: the interaction of narrative structure and page layout in understanding comics .  Frontiers in Psychology, vol 5, article 680: 1 ― 9.  Cohn, N.; Jackendoff, R.; Holcomb, P. J.; Kuperberg, G. R. (2014)  The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension .  Neuropsychologia, 64: 63 ― 70.  Cohn, N.; Taylor-Weiner, A.; Grossman, S. (2012) Framing attention in Japanese and American comics: Cross-cultural differences in attentional structure.  Frontiers in Psychology, 3: 1 ― 12.  Cohn, N (2016)  A multimodal parallel architecture: A cognitive framework for multimodal interactions .  Cognition, 1 46: 304 ― 323.  Engelhardt, Y. (2007)  Syntactic structures in graphics .  Computational Visualistics and Picture Morphology, 5: 23 ― 35.  Gernsbacher,M. A. (1983) Memory for Surface Information in Non-Verbal Stories: Parallels and Insights to the Language Process. Doctoral Dissertation, University of Texas at Austin, Austin, TX. Nakazawa, J. (2005)  Development of manga literacy in children .  In D. W. Shall, J. Nakazawa, ＆ B. J. Shall (Eds.), Applied developmental psychology: Theory, practice, and research from Japan (pp. 23 ― 42).  Greenwich, CT: Information Age Publishing. Nakazawa, J., and Shwalb, D. W. (2012) “The comparison of manga literacy 983 between Japanese and U.S. Students,” in Paper Presented at the Meeting 984 on the Psychology of Manga (Chiba: Chiba University).Nakazawa, J. (2015) Manga literacy and manga comprehension in Japanese children. In N. Cohn (Ed.), The Visual Narrative Reader (pp. 157-184). London: Bloomsbury.Olness, R. (2005) Using literature to enhance writing instruction (pp. 1-9). Newark, DE: International Reading AssociationShort, J. C., Randolph-Seng, B., and McKenny, A. F. (2013) Graphic presentation: an empirical examination of the graphic novel approach to communicate business concepts. Bus. Comm. Q. 76, 273-303. doi: 10.1177/1080569913482574 

ニホンノダイガクニオケルESLガクセイノブンショタイワヲツカッタVNGノコウカニツイテ

https://ngu.repo.nii.ac.jp/?action=repository_action_common_download&item_id=788&item_no=1&attribute_id=22&file_no=1

ニホンノダイガクニオケルESLガクセイノブンショタイワヲツカッタVNGノコウカニツイテ

Abstract

Similar works

Full text

Available Versions

Nagoya Gakuin University Repository