Klimanova, Liudmila

Lomicka, Lara

English

ScholarSpace at University of Hawai'i at Manoa

Language Learning & Technology  February 2023, Volume 27, Issue 2 ISSN 1094-3501  CC BY-NC-ND pp. 1–5   COMMENTARY    Semiotics in CALL: Signs, meanings, and multimodality in digital spaces Liudmila Klimanova, University of Arizona Lara Lomicka, University of South Carolina APA Citation: Klimanova, L., & Lomicka, L. (2023). Semiotics in CALL: Signs, meanings and multimodality in digital spaces. Language Learning & Technology, 27(2), 1–5. https://hdl.handle.net/10125/73500 Introduction The ever-changing learning landscape of today’s world has significantly restricted movement across physical spaces and geographical borders. More and more instructors have turned to digital tools to offer language learners virtual experiences with target cultures and communities. Studies have begun to explore how digital spaces and dynamic digital imagery can enable learners to interact and practice meaning making in virtual contexts (see Godwin-Jones, 2016, 2017; Han, 2019; Messina Dahlberg & Bagga-Gupta, 2016; Pertusa‐Seva & Stewart, 2000). Images have long been a staple in language instruction, providing a cultural context or introducing new vocabulary in lower levels, and supporting critical thinking and insightful discussions in upper-level literature and culture courses (Bourns et al., 2020; Glisan & Donato, 2017; Johnson & English, 2003; Smith & Bley, 2012). The power of imagery is in its capacity to capture visual depictions of cultural products and practices in close connection to a particular location in the world and to a concrete moment in time (Klimanova & Hellmich, 2021a). Digital spatial and semiotic tools provide access to an infinite number of dynamic images and semiotic symbols that can enhance cultural learning. These include experiences such as digital mapping, linguistic landscapes, virtual study abroad, digital literacies (including open resources), and augmented reality. Digital maps, for example, have been found to create a powerful interactive visualization environment for critical learning experiences (Guerrero‐Rodríguez et al., 2022; Sinton & Lund, 2007). Such tools foster spatial connections between geographical locations and critical historical, cultural, and social issues (Apgar, 2018; Goldfield & Schlichting, 2007), enhance relations between space, time, and personal narrative (Hetland, 2017), validate authenticity of movement and mobility (Sinton & Lund, 2007), and enrich classroom collaboration and communication (Sokolik, 2011). Semiotic technologies, especially interactive ones, enhanced with visual artifacts of urban spaces and place-oriented cultural practices, mentally transport us to another place and/or time and reveal through a sociological analysis the ways that the spatial arrangements of cultural communities are meaningful (Gaspar, 2022; Klimanova & Hellmich, 2021b; Krase & Shortell, 2011).   Semiotics as a field is largely defined as the study of signs, images, and codes that can be used to produce, convey, and interpret messages, as well as the codes that govern their use (Smith et al., 2004). This special issue of LLT offers the first substantive scholarly collection of articles that explore the potential of semiotics in CALL through memes, immersive virtual reality (VR) environments, geosemiotics, critical digital literacy, digital literacy practices, semiotic consciousness, 3D virtual tasks, and semiotic resources for the teaching of languages, literatures, and cultures. Articles in this special issue address the impact of semiotic tools (digital, spatial) on language learners’ access to critical information and their perception and interpretation of linguistic and cultural practices at home and abroad. Recent research on classroom integration is provided. Drawing from a wide range of perspectives and frameworks, this special issue highlights the numerous potentials that semiotics in CALL can offer for language learning and teaching, and address theoretical, empirical, and pedagogical approaches that connect classroom language instruction to the multifaceted contexts and challenges of language use beyond the classroom, particularly in digital 2 Language Learning & Technology     spaces. In This Issue The diverse contributions to this special issue showcase a vast epistemological breadth of topics and issues in the study of semiotics in language learning and teaching. As CALL theory has shifted from a purely structuralist approach towards a holistic and transdisciplinary view of language acquisition (Chun et al., 2016; Kern, 2021), the articles in this special issue shed light on learners’ ability to manipulate and interpret a broad array of linguistic and non-linguistic sign systems. Additionally, these articles navigate across culturally situated semiotic forms, materials, and virtual social contexts. The diversity of thought and ideas in this issue helps to move semiotics in CALL to the forefront as a more prominent subfield with new categorizations and theorizations. The special issue opens with Darvin’s article, Sociotechnical Structures, Materialist Semiotics, and Online Language Learning, which adopts a critical materialist semiotic perspective to examine the sociomaterialist architectures of physical and virtual contexts and their critical impact on framing the actions and digital literacy practices of language learners. Drawing on the comparative case studies of three Filipino immigrants in Canada, the study focuses on semiotic arrangements in various digital platforms to determine how information in these platforms is distributed, accessed, and presented to language learners. It also explores how the material features of digital devices alter navigation pathways and types of information that is made available. The concept of sociotechnical structures is proposed to explain the reconfigurable designs where the linguistic and semiotic features of digital platforms and devices afford or constraint sociality and online actions by providing differential access to information. Lee and Hampel’s article, Geosemiotics as a Multiperspectivist Lens: Theorizing L2 Use of Semiotic Resources in Negotiation of Meaning with Mobiles from Outside the Classroom, draws on the framework of geosemiotics (Scollon & Scollon, 2003) to examine how five cross-cultural dyads in the ESL learning context interact multimodally with one another and convey visual information using mobile devices and cameras while connecting from different locations (e.g., cafes, restaurants, gardens). Such locations are construed as pervasive places which offer possibilities to language learners to deploy visual and place-based semiotic resources (e.g., objects in the environment) and embodied resources (e.g., iconic gestures) to engage in negotiation of meaning and foster language learning. By establishing the ‘shared’ worlds and allowing each other into their individual material places via video conferencing, participants are challenged by the necessity to coordinate the meanings of place, switching in and out of the roles of viewers and visual image creators. Such rich multimodal patterns of interaction and meaning negotiation afforded by mobile technology promise multifarious language usages for language learning outside the classroom. Satar, Hauck, and Bilki’s study, Multimodal Representation in Virtual Exchange: A Social Semiotic Approach to Critical Digital Literacy, employs the concepts of social semiotics, such as transformation, transduction, and mimesis, to explore how teacher trainees acquire critical digital literacy through transformative processes observed in multilingual, multicultural, and multimodal virtual interactions. Semiotic agility and semiotic budget are utilized as an analytical toolkit to detect the achievement of critical digital literacy by capitalizing on multimodal affordances and by exercising and allocating symbolic power. Adopting the view that language learning is semiotic learning, Suvorov and Gruba operationalize the concept of semiotic resources and its integration into CALL activity design. Grounded in the three-tiered interdisciplinary second language acquisition (SLA) framework (Douglas Fir Group, 2016) and the metafunctions of language established by Systemic Functional Linguistics (Eggins, 2004), their article, Integrating Semiotic Resources in CALL Activity Designs, proposes an analysis of the types of knowledge that inform EFL\ESL pre-service teachers’ choice of CALL activity design elements. A number of factors played a role in motivating the selection of semiotic resources for design purposes, including the features Liudmila Klimanova and Lara Lomicka 3           of the technology itself, individual orientations to language, and general considerations of the instructional design principles. The data of this study suggest a critical need in establishing a broader view of language as a socio-semiotic system, and not merely a cognitive-linguistic phenomenon, as well as raising critical semiotic awareness that language teaching is the production and conveyance of semiotic repertoires. Chen and Sevilla-Pavón’s article, Negotiation of Meaning via Virtual Exchange in Immersive Virtual Reality Environments, showcases how learners use semiotic resources (head movements, gestures, facial expressions, body posture, special juxtaposition) in an immersive VR environment as they negotiate meaning. Students in Taiwan and Spain completed tasks as part of a VR virtual exchange project. Based on multimodal interaction analysis and the negotiation of meaning model, the results suggest that learners turned to semiotic resources in constructing and negotiating meaning in order to effectively communicate in the VR space. This study also revealed that space proxemics and object handing are distinct features of VR environments.  The article by Han and Smith, An Ecological Perspective on the Use of Memes for Language Learning, sheds light on learner-produced memes as a powerful semiotic tool for meaning making. Han and Smith use a comparative case study that examines the semiotic affordances of memes in the digital wilds. The study is situated within a higher education Chinese-English intercultural chat group. Their work points to four key affordances: (a) linking learners to emergent semiotic repertoires, (b) L2 user agency, (c) increased motivation, and (d) personhood development. Although participants are all prior “memers,” data suggest that learners showed an awareness of perceived semiotic affordances and their engagement with memes helped to shape their learning experiences.  Guo and Lan’s article, Virtual World-Supported Contextualized Multimodal EFL Learning at a Library, presents an action research study that applies a 3D creation model to picture book reading for EFL learners in the informal setting of a library. Researchers used a semiotic approach to help young learners (ages 10 to 12) connect their meaning making in VR projects with their semiotic choices. VR technologies provide semiotic resources for the redesign and reconstruction of meaning, allowing learners to translate their ideas into various modes: text, posters, and digital content. Results of their study indicate that VR strengthened English vocabulary skills and increased learners’ motivation, while lowering learning anxiety and improving learner attitudes towards technology integration in language learning.  Finally, Burr’s article, entitled Learning a Foreign Language and Locality Through an Animated Documentary Film, makes a case for teaching a Latvian language class through semiotic analysis. Through the screening of a Latvian animated documentary film, students were encouraged to dig deeper into symbolic and functional meanings of marked spaces. Specifically, Burr, in one 4-week long online language learning module, has students become ethnographers as they analyzed a variety of semiotic resources (words, sounds, colors, images, spatial objects and (inter)actions). This semiotic experience encouraged use of multimedia, multimodal, and multi-layered text.  Conclusion Despite the ever-changing landscape of language learning, we believe the contributions of this special issue advance the field by illustrating the range of semiotic technologies in CALL available to learners and teachers, and the potential that semiotics holds for the classrooms and education more generally. These studies highlight the extent to which such semiotic tools shape resources, landscapes, and learning as learners navigate L2 linguistic and cultural practices. It is our hope that this special issue will bring new ideas, frameworks, theories, and thoughts for expanding semiotics in CALL. Further work should place focus on spaces and places of learning, whether in the classroom, at home, or abroad, such as linguistic landscapes, as well as explore a wider diversity of theoretical, methodological, and research-design choices. 4 Language Learning & Technology     Acknowledgements We would like to thank the editors of Language Learning & Technology for the invitation to edit this special issue. We would also like to thank Skyler Riela for excellent support with the management and editing process. Finally, we are grateful for the reviewers and authors who made contributions to this issue and to the field of semiotics in CALL.  References Apgar, R. B. (2018). Emplacing Berlin: Digital itineraries and short-term study abroad in the city. International Journal of Humanities and Arts Computing, 12(1), 26–36.  Bourns, S. K., Krueger, C. L., & Mills, N. A. (2020). Perspectives on teaching language and content. Yale University Press.   Chun, D., Smith, B., & Kern, R. (2016). Technology in language use, language teaching, and language learning. The Modern Language Journal, 100(S1), 64–80. https://doi.org/10.1111/modl.12302  Douglas Fir Group. (2016). A transdisciplinary framework for SLA in a multilingual world. The Modern Language Journal, 100(S1), 19–47. https://doi.org/10.1111/modl.12301  Eggins, S. (2004). An introduction to systemic functional linguistics. Continuum. Gaspar, B. (2022). The effects of multimodal communication on the development of new types of learner imagination. In L. Klimanova (Ed.), Identity, multilingualism, and CALL: Responding to new global realities (pp. 328–352). Equinox.   Glisan, E., & Donato, R. (2017). Focusing on cultural products, practices, and perspective in a dialogic context. In E. Glisan & R. Donato (Eds.), Enacting the work of language instruction, high-leverage teaching practices. ACTFL.  Godwin-Jones, R. (2016). Augmented reality and language learning: From annotated vocabulary to place-based mobile games. Language Learning & Technology, 20(3), 9–19. http://dx.doi.org/10125/44475  Godwin-Jones, R. (2017). Smartphones and language learning. Language Learning & Technology, 21(2), 3–17. https://dx.doi.org/10125/44607 Goldfield, J., & Schlichting, K. (2007). Foreign language and sociology: Exploring French society and culture. In D. Sinton & J. Lund (Eds.), Understanding place: GIS and mapping across the curriculum (1st ed.) (pp. 227–235). ESRI Press. Guerrero‐Rodríguez, P., Lomicka Anderson, L., & Lord, G. (2022). SIFTR‐ing through the development of cultural awareness at home and abroad. Foreign Language Annals, 55(2), 435–454. https://doi.org/10.1111/flan.12617 Han, Y. (2019). Exploring multimedia, mobile learning, and place-based learning in linguacultural education. Language Learning & Technology, 23(3), 29–38. http://hdl.handle.net/10125/44692  Hetland, T. (2017). Teaching literature with digital technology. Bedford / St. Martin's  Johnson, S. P., & English, K. (2003). Images, myths, and realities across cultures. The French Review, 76(3), 492–506. Kern, R. (2021). Twenty-five years of digital literacies in CALL. Language Learning & Technology, 25(3), 132–150. http://hdl.handle.net/10125/73453  Klimanova, L., & Hellmich, E. A. (2021a). Crossing transcultural liminalities with critical virtual exchange: A study of shifting border discourses. Critical Inquiry in Language Studies, 18(3), 273–304. https://doi.org/10.1080/15427587.2020.1867552 Liudmila Klimanova and Lara Lomicka 5           Klimanova, L., & Hellmich, E. A. (2021b). Putting local on the MAP: A model for engaging foreign language students with local cultures. Foreign Language Annals, 54(1), 158–184. https://doi.org/10.1111/flan.12493 Krase, J., & Shortell, T. (2011). On the spatial semiotics of vernacular landscapes in global cities. Visual Communication, 10(3), 367–400. https://doi.org/10.1177/1470357211408821  Messina Dahlberg, G., & Bagga-Gupta, S. (2016). Mapping languaging in digital spaces: Literacy practices at borderlands. Language Learning & Technology, 20(3), 80–106. http://dx.doi.org/10125/44483  Pertusa‐Seva, I., & Stewart, M. (2000). Virtual study abroad 101: Expanding the horizons of the Spanish curriculum. Foreign Language Annals, 33(4), 438–441. https://doi.org/10.1111/j.1944-9720.2000.tb00625.x  Scollon, R., & Scollon, S. W. (2003). Discourses in place: Language in the material world. Routledge.  Sinton, D., & Lund, J. (2007). Understanding place: GIS and mapping across the curriculum. ESRI Press.  Smith, K. L., Moriarty, S., Kenney, K., & Barbatsis, G. (2004). The handbook of visual communication: Theory, methods, and media. Routledge.  Smith, S. H., & Bley, M. (2012). Streets in the sky: The balconies of Lima and the road to intercultural competence. Journal of Global Initiatives: Policy, Pedagogy, Perspective, 7(2), 123–166.  Sokolik, M. E. (2011). A route to communication: Google Maps. TESL-EJ, 15(3), 1–13.   About the Authors Liudmila Klimanova is Assistant Professor of Russian and SLAT with the focus on CALL and Digital Humanities at the University of Arizona. Her research addresses the intersections of identity, multilingualism, and virtual intercultural learning. She is particularly interested in innovative uses of digital semiotics and interactive mapping for scaffolding the development of intercultural competence and critical digital literacies.  E-mail: klimanova@arizona.edu  Lara Lomicka is Professor of French and Applied Linguistics with a focus on CALL at the University of South Carolina. She is the former President of CALICO, NFMLTA, and AAUSC, and works as an associate editor for LLT and JALT-CALL. Her research combines elements of intercultural learning, social media, study abroad, interactive mapping, and digital semiotics as they relate to language learning.  E-mail: lomicka@mailbox.sc.edu   

Semiotics in CALL: Signs, meanings, and multimodality in digital spaces

https://scholarspace.manoa.hawaii.edu/bitstreams/04feec3e-89f2-49d9-8779-7703564e6cd5/download

Semiotics in CALL: Signs, meanings, and multimodality in digital spaces

Abstract

Similar works

Full text

Available Versions

ScholarSpace at University of Hawai'i at Manoa