Where Do I Come From? Metaphors in Sex Education Picture Books for Young Children in China

This study examines the types of verbal, pictorial, and multimodal metaphors in the genre of sex education picture books for young children in Mainland China. Although being an educational discourse genre that is essentially concerned with transmitting scientific facts, sex education picture books employ a range of metaphors that categorize and construe the biological knowledge of human reproduction in a way that not only facilitates young children’s understanding of scientific concepts but also instills in them particular values and moralities that are socioculturally conditioned. An examination of the source domains from which the metaphors are drawn and the target domains onto which the metaphors are mapped reveals three types of metaphor, namely, personification, domestication, and cross-experience metaphors. The analysis of seven sex education picture books for pre-school children suggests that these types of metaphor are used purposefully for addressing pedagogical as well as ideological concerns in the introduction of sex-related knowledge in Mainland China

Meaning between, in, and around words, gestures and postures: multimodal meaning making in children's classroom communication

The view of language from a social semiotic perspective is clear. Language is one of many semiotic resources we employ in our communicative practices. That is to say that while language is at times dominant, it always operates within a multimodal frame and furthermore, at times modes other than language are dominant. The proposed 2014 National Curriculum for the UK, on the other hand, values pupils' face-to-face classroom interaction in terms of standard spoken English (i.e. in terms of the mode of language alone). This paper offers examples demonstrating how embodied modes such as gesture, posture, facial expression, gaze and haptics work in conjunction with speech in children's collaborative construction of knowledge. In other words, what may have been previously conceived as gaps and silences - often interpreted as an absence of language - are in fact instantiations of the work of semiotic modes other than language. In order to consider this closely, this paper offers evidence from a multimodal micro-analysis of pupil-to-pupil, face-to-face interaction in one science lesson in a Year Five UK Primary classroom. It demonstrates how children's meaning-making is achieved through apt use of all available semiotic resources

Matter, meaning and semiotics

We inhabit two worlds – the world of matter and the world of meaning (see Halliday, ‘On matter and meaning: The two realms of human experience, 2005). In this article, these two worlds and the physical, biological, social and semiotic systems that connect them are investigated. In this respect, semiotic systems are the most complex because they involve physical systems (the material sign), biological systems (human beings), social systems (society and culture) and meaning itself. Semiotic frameworks need to take into account these various dimensions as changes in one system reverberate across the meta-system as a whole. With this in mind, the interplay between material and semiotic worlds from a social semiotic perspective, are explored with a focus on meaning and its significance in relation to human existence. Using examples from various industrial ages, the article explores how semiotic resources (in this case, in mathematics, science and computer programming languages) are organized to structure reality in specific ways, and how semiotic combinations and the technologies arising from those constructions have changed the course of human history. In this discussion, attention is paid to the role of visual communication, both in terms of visual semiotic resources (e.g. graphs, digital images) and visual aspects of multimodal texts. It thus becomes evident that the functionalities of any one semiotic resource (including language) must be viewed in relation to its collective co-deployment with other semiotic resources. Lastly, the author examines semiosis in the digital age and considers the social implications of the current digital ecosystem. In doing so, she conceptualizes digital technologies as a one-way mirror where members of society use digital media for every facet of their lives while being watched, analysed and manipulated by those who have designed and own the digital platforms. It is apparent that semiotics has a major role to play in terms of design, policymaking and activism around future digital technologies. </jats:p

The role of images in social media analytics: A multimodal digital humanities approach

The analysis of social media data is an emerging research field that aims to study the dynamics of urban life. In this study, we adopt a multimodal digital humanities approach to combine the analysis of text-based social media data with visual social media data in an interactive map to investigate urban life in Singapore from a social semiotic perspective. Twitter is used as a source of user-defined localised textual data, Instagram as a source of localised user-generated images and Foursquare as a source of user-defined location-based information where is semantically organised according to Wikipedia's classification tree. In this way, we track the multimodal content of social media according to semantically organised location-based sources. This study suggests that users of Twitter express emotion about their own lives and the world around them, but these linguistic resources are differentially deployed according to venue. However, this is less variation in the use of photos to construe personal relationships, suggesting that photos fulfil and intrinsic need to be observed which transcends the nature of the social practice which is taking place. It is envisaged that the role of the visual will continue to expand as digital technologies refashion and transform out semiotic world.status: publishe

Teaching Visual Texts with Multimodal Analysis Software

This exploratory study introduces the systemic approach and the explicit teaching of a meta-language to provide conceptual tools for students for the analysis and interpretation of multimodal texts. Equipping students with a set of specialised vocabulary with conventionalised meanings associated with specific choices in multimodal texts empowers them to support their interpretations with reference to textual evidence. The aim of the current study is to explore the extent to which the systemic approach may be supported by multimodal analysis software, in this case for the study of print advertisements in Singapore’s secondary schools. A mixed method approach is used to derive the findings from this study. Instruments used include lesson observations and interviews with teachers and students, as well as user-experience surveys. Findings from the study indicate that there is value in the systemic approach to teaching visual texts, and that it can be usefully supported by multimodal analysis software as an annotation and analytical tool. This is subject to further customisation of the learning packages, more in-depth professional development for teachers, and improvements to the technical functionalities in the software. The designed affordances for multimodal analysis software derived from the study, which have been implemented in the new version of the software, are discussed

A Digital Mixed Methods Research Design: Integrating Multimodal Analysis With Data Mining and Information Visualization for Big Data Analytics

This article demonstrates how a digital environment offers new opportunities for transforming qualitative data into quantitative data in order to use data mining and information visualization for mixed methods research. The digital approach to mixed methods research is illustrated by a framework which combines qualitative methods of multimodal discourse analysis with quantitative methods of data mining and information visualization in a multilevel, contextual model that will result in an integrated, theoretically well-founded, and empirically evaluated technology for analyzing large data sets of multimodal texts. The framework is applicable to situations in which critical information needs to be extracted from geotagged public data: for example, in crisis informatics, where public reports of extreme events provide valuable data sources for disaster management

Interactive software for multimodal analysis

10.1177/1470357212446414Visual Communication113363-38