From book to stage to screen: semiotic transformations of Gothic horror genre conventions

This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scène, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic approach for developing a better understanding of the semiotic transformations that horror genre conventions undergo in different media and the viewership positions that are thus re-drawn for audiences. The paper concludes with a view of multimodal recontextualisation processes which form the underlying basis of human sociocultural life

Addressing international students on Australian and Chinese university webpages: A comparative study

This article investigates the discursive construction of international students on Australian and Chinese university webpages that are directed at international students. The international student theme webpages of three Australian and three Chinese universities were examined in relation to how verbal and visual semiotic resources were co-deployed on the webpages to address international students in the climate of neoliberal thinking in higher education. The tools used for analysis and interpretation were informed by socio-semiotic approaches to multimodal discourse analysis, international education discourses and communication accommodation theories. It was found that the Australian and Chinese university webpages differed in several salient ways to the effect that international students were portrayed as agentive and informed individuals to explore a study abroad experience at the Australian universities versus being explicitly guided through their study abroad at the Chinese universities. These results were compared and interpreted as reflecting each country’s conception of transnational education. The article concludes with a summary of the impacts of cultural and intercultural factors and neoliberal thinking in higher education on multimodal representations of international students on university webpages and directions for further research

Inter-semiotic Translation as Resemiotization: A Multimodal Perspective

Intersemiotic translation is viewed as the basis of cultural communication through which ideas are circulated, translated and explained using language, images and other semiotic resources. From this perspective, intersemiotic translation is conceptualised through the principle of resemiotisation which is concerned with how semiotic choices are translated over time as social practices. Key issues which arise from viewing intersemiotic translation as resemiotisation are investigated in this paper: (a) how can shifts of meaning be conceptualised across semiotic resources which are fundamentally different in nature?; (b) what meanings are retained and changed as a result of resemiotisation?; and (c) how can such meanings be modelled theoretically and tracked analytically? In order to explore these issues, a multimodal social semiotic approach based on Michael Halliday’s systemic functional theory is presented. In this approach, semiotic resources are conceptualised as systems of meaning which differentially fulfil various functions in society: to structure experience and make logical connections in the world, to enact social relations and create a stance towards the world, and to organise these meanings into multimodal messages. The multimodal approach to intersemiotic translation is illustrated through examples which explore shifts of meaning which take place across theses strands of meaning as semiotic choices are resemiotised within and across multimodal texts. The focus of this discussion is intersemiotic translation across language (spoken and written), images (photographs, film tracks and graphs) and mathematical symbolism in a range of different text types (i.e. a news report, an infographic, a video and a mathematical graph). These examples illustrate the principles of intersemiotic translation through which semantic expansions occur as semiotic choices are resemiotised using different semiotic resources, which result in access to a new meaning potential.La traduction intersémiotique est considérée comme la base de la communication culturelle, par le biais de laquelle les idées sont diffusées, traduites et expliquées à l’aide du langage, d’images, et d’autres ressources sémiotiques. Dans une telle perspective, la traduction intersémiotique est conceptualisée par le principe de resémiotisation qui considère comment les choix sémiotiques se traduisent en pratiques sociales au fil du temps. Cet article étudie les questions clés soulevées par le fait de traiter la traduction intersémiotique comme une resémiotisation: (a) comment les glissements de signification peuvent-ils être conceptualisés à travers des ressources sémiotiques qui sont de nature fondamentalement différente ? (b) quelles significations sont conservées et modifiées en raison de la resémiotisation ? et enfin (c) comment ces significations peuvent-elles être modélisées théoriquement et suivies de façon analytique ? Pour explorer ces questions, une approche socio-sémiotique multimodale est présentée, basée sur la théorie systémique fonctionnelle de Michael Halliday. Dans cette approche, les ressources sémiotiques sont conceptualisées comme des systèmes de significations qui remplissent des fonctions variées dans la société, pour structurer l’expérience et établir dans l’univers des connexions logiques, pour établir des relations sociales et une attitude vis-à-vis du monde, et pour organiser toutes ces significations en messages multimodaux. L’approche multimodale à la traduction intersémiotique est illustrée par des exemples qui explorent les glissements de signification qui se produisent à travers la toile des significations, lorsque les choix sémiotiques sont resémiotisés au sein de textes multimodaux, et entre ceux-ci. Au cœur de cette discussion est la traduction intersémiotique entre langage (parlé et écrit), images (photos, films et graphiques) et symbolisme mathématique, dans une gamme de différents types de textes (par exemple reportage, document infographique, vidéo et graphique mathématique). Ces exemples illustrent les principes de la traduction intersémiotique au moyen de laquelle des expansions sémantiques se produisent, lorsque les choix sémiotiques sont resémiotisés à l’aide de différentes ressources sémiotiques, donnant ainsi accès à un nouveau potentiel de signification

A multimodal mixed methods approach for examining recontextualisation patterns of violent extremist images in online media

© 2017 This paper uses a multimodal mixed methods approach for exploring general recontextualisation patterns of violent extremist images in online media. Specifically, the paper reports on the preliminary findings of a preliminary study which investigates various patterns in the reuse of images which appear in ISIS's official propaganda magazines Dabiq and Rumiyah by others across various public online media platforms (e.g. news websites, social media news aggregates, blogs). Using a mixed methods approach informed by multimodal discourse analysis, and combined with data mining and information visualisation, the study addresses questions such as which types of images produced and used by ISIS in its propaganda magazines recirculate most frequently in other online media over time, on which types of online media these images reappear, and in which contexts they are used and reused on these websites, that is that is, whether the tone of the message is corporate (forma l) or personal (informal). Preliminary findings from the study suggest different recontextualisation patterns for certain types of ISIS-related images of over time. The study also found that the majority of violent extremist images used in the sample analysis appear to circulate most frequently on Western news and politics websites and news aggregate platforms, in predominantly formal contexts

The multimodal classroom in the digital age: The use of 360 degree videos for online teaching and learning

© 2019 Taylor & Francis. Digital technology provides access to resources extending beyond the boundaries of the classroom, while offering opportunities for personalised and collaborative learning and innovation and creativity. Off-campus online learning has become a dominant trend in the higher education sector in response to rapid advances in digital technology and increased pressures of rising costs, reduced government support and fierce competition in the global education market. Universities worldwide face the challenge of reinventing themselves to survive the challenges of a changing educational environment in the digital age. However, if digital technologies, redesigned classrooms and off-campus online learning environments are to offer students the opportunity to reach their potential, we need to attain a thorough understanding of the strengths and challenges of digital technologies as pedagogical tools. This chapter presents the initial findings from an ongoing project that aims to enhance the quality of learning outcomes in online teaching and learning situations in tertiary education, by developing and trialling a web-based application for encouraging and mapping student engagement with online content through 360 degree video annotation and data analytics