Face off – a semiotic technology study of software for making deepfakes

Deepfakes, an algorithm that transposes the face of one person onto the face of another person in images and film, is a digital technology that may fundamentally alter our belief in visual modality and thus presents alarming consequences for an image-centric culture. Not only are these face-translations now so advanced that it is virtually impossible for people to tell that they are fake – this technology is also becoming accessible to laypersons who, with little or no computer skills, can use them for all kinds of purposes, including criminal intentions like revenge porn and identity theft. It is therefore timely and crucial to explore the semiotic potential of deepfakes. This paper presents a semiotic technology perspective, i.e., the study of technology for meaning- making that is an emergent field in social semiotics, to report on findings from an ongoing study of how deepfake software is designed and used as a semiotic resource in erotic and political contexts. The paper advances the argument that the software is able to appropriate all signifiers of the face and their cultural history. Consequently, the semiotic operations of this technology prepare the ground for the problematic perspectives of synthetic facial imagery. On this basis, the paper calls for a critical awareness of taking visual representations of current events at face value and considers how deepfake technology is embedded in unsound sharing practices of visual artefacts that tamper with the rich meaning potential of the face.    &nbsp

Viewing, listening and reading along: Linguistic and multimodal constructions of viewer participation in the net series SKAM

This paper investigates how SKAM viewers are positioned as participants through semiotic resources in the net series, i.e. the filmic means for making meaning, including representations of the characters’ embodied and digitally mediated communication. For this purpose, we combine perspectives from linguistic-multimodality studies of modes for communication and studies building on Goff man’s (1981) work on participation frameworks, i.e. the various ways of participating in co-present and/or mediated communication. The article aims to complement existing media studies of immediacy in SKAM and viewers’ sense of co-presence with characters in the net series (Jerslev 2017, Sundet 2017). It does so by showing how participant frameworks are multimodally constructed at a fictional level and a communicational level. Within each of these frameworks, the viewer is positioned in distinct ways. We describe how the viewer is placed, i.e. physically positioned, in the interactional space of the depicted characters, and how the characters’ communicative means are interactionally organised, accomplished and made available for interpretation by the viewer. Furthermore, we show how characters monitor each other in the shared space of a schoolyard, how embodied and digitally mediated communicative features are foregrounded, and how the viewer is provided access to these resources in ways that reflect and create specific viewing positions in the communicative frames of the characters. We argue that these integrations of semiotic modes exploit affordances related to speech, writing and embodiment, that the positionings mainly work to create a sense of presence and identification for the viewer, and that representations of digitally mediated communication (writing) on the viewer’s screen specifically expose how the digitally mediated communication space of one of the characters is integrated with the digitally mediated viewing space