Digital Media and Textuality: From Creation to Archiving

Due to computers' ability to combine different semiotic modes, texts are no longer exclusively comprised of static images and mute words. How have digital media changed the way we write and read? What methods of textual and data analysis have emerged? How do we rescue digital artifacts from obsolescence? And how can digital media be used or taught inside classrooms? These and other questions are addressed in this volume that assembles contributions by artists, writers, scholars and editors. They offer a multiperspectival view on the way digital media have changed our notion of textuality

Digital Media and Textuality

Due to computers' ability to combine different semiotic modes, texts are no longer exclusively comprised of static images and mute words. How have digital media changed the way we write and read? What methods of textual and data analysis have emerged? How do we rescue digital artifacts from obsolescence? And how can digital media be used or taught inside classrooms? These and other questions are addressed in this volume that assembles contributions by artists, writers, scholars and editors such as Dene Grigar, Sandy Baldwin, Carlos Reis, and Frieder Nake. They offer a multiperspectival view on the way digital media have changed our notion of textuality

Digital Media and Textuality

Due to computers' ability to combine different semiotic modes, texts are no longer exclusively comprised of static images and mute words. How have digital media changed the way we write and read? What methods of textual and data analysis have emerged? How do we rescue digital artifacts from obsolescence? And how can digital media be used or taught inside classrooms? These and other questions are addressed in this volume that assembles contributions by artists, writers, scholars and editors such as Dene Grigar, Sandy Baldwin, Carlos Reis, and Frieder Nake. They offer a multiperspectival view on the way digital media have changed our notion of textuality

Mosaic narrative a poetics of cinematic new media narrative

This thesis proposes the Poetics of Mosaic Narrative as a tool for theorising the creation and telling of cinematic stories in a digital environment. As such the Poetics of Mosaic Narrative is designed to assist creators of new media narrative to design dramatically compelling screen based stories by drawing from established theories of cinema and emerging theories of new media. In doing so it validates the crucial element of cinematic storytelling in the digital medium, which due to its fragmentary, variable and re-combinatory nature, affords the opportunity for audience interaction. The Poetics of Mosaic Narrative re-asserts the dramatic and cinematic nature of narrative in new media by drawing upon the dramatic theory of Aristotle’s Poetics, the cinematic theories of the 1920s Russian Film Theorists and contemporary Neo-Formalists, the narrative theories of the 1960s French Structuralists, and the scriptwriting theories of contemporary cinema. In particular it focuses on the theory and practice of the prominent new media theorist, Lev Manovich, as a means of investigating and creating a practical poetics. The key element of the Poetics of Mosaic Narrative is the expansion of the previously forgotten and undeveloped Russian Formalist concept of cinematurgy which is vital to the successful development of new media storytelling theory and practice. This concept, as originally proposed but not elaborated by Kazansky, encompasses the notion of the creation of cinematic new media narrative as a mosaic – integrally driven by the narrative systems of plot, as well as the cinematic systems of visual style created by the techniques of cinema- montage, cinematography and mise-en-scene

Territorial Violence and Design, 1950-2010: A Human-Computer Study of Personal Space and Chatbot Interaction

Personal space is a human’s imaginary system of precaution and an important concept for exploring territoriality, but between humans and technology because machinic agencies transfer, relocate, enact and reenact territorially. Literatures of territoriality, violence and affect are uniquely brought together, with chatbots as the research object to argue that their ongoing development as artificial agents, and the ambiguity of violence they can engender, have broader ramifications for a socio-technical research programme. These literatures help to understand the interrelation of virtual and actual spatiality relevant to research involving chatrooms and internet forums, automated systems and processes, as well as human and machine agencies; because all of these spaces, methods and agencies involve the personal sphere. The thesis is an ethical tale of cruel techno-science that is performed through conceptualisations from the creative arts, constituting a PhD by practice. This thesis chronicles four chatbots, taking into account interventions made in fine art, design, fiction and film that are omitted from a history of agent technology. The thesis re-interprets Edward Hall’s work on proxemics, personal space and territoriality, using techniques of the bricoleur and rudiments (an undeveloped and speculative method of practice), to understand chatbot techniques such as the pick-up, their entrapment logics, their repetitions of hateful speech, their nonsense talk (including how they disorientate spatial metaphors), as well as how developers switch on and off their learning functionality. Semi-structured interviews and online forum postings with chatbot developers were used to expand and reflect on the rudimentary method. To urge that this project is timely is itself a statement of anxiety. Chatbots can manipulate, exceed, and exhaust a human understanding of both space and time. Violence between humans and machines in online and offline spaces is explored as an interweaving of agency and spatiality. A series of rudiments were used to probe empirical experiments such as the Prisoner’s Dilemma (Tucker, 1950). The spatial metaphors of confinement as a parable of entrapment, are revealed within that logic and that of chatbots. The ‘Obedience to Authority’ experiments (Milgram, 1961) were used to reflect on the roles played by machines which are then reflected into a discussion of chatbots and the experiments done in and around them. The agency of the experimenter was revealed in the machine as evidenced with chatbots which has ethical ramifications. The argument of personal space is widened to include the ways machinic territoriality and its violence impacts on our ways of living together both in the private spheres of our computers and homes, as well as in state-regulated conditions (Directive-3, 2003). The misanthropic aspects of chatbot design are reflected through the methodology of designing out of fear. I argue that personal spaces create misanthropic design imperatives, methods and ways of living. Furthermore, the technological agencies of personal spaces have a confining impact on the transient spaces of the non-places in a wider discussion of the lift, chatroom and car. The violent origins of the chatbot are linked to various imaginings of impending disaster through visualisations, supported by case studies in fiction to look at the resonance of how anxiety transformed into terror when considering the affects of violence

Mytholudics:understanding games as/through myth

This dissertation outlines a mythological framework for understanding how games produce meaning. I first theorise mythology as it applies to games and play. This is expressed through a cycle showing how mythology is embedded in the production of games and how it impacts the interpretation of games. This is then operationalised as a method for the analysis of games. I call my theorisation and analytical approach mytholudics. I then apply mytholudics in ten analyses of individual games or game series, split into two lenses: heroism and monstrosity. Finally, I reflect on these analyses and on mytholudics as an approach.Mythology here is understood through two perspectives: Roland Barthes’ theory outlined in Mythologies (1972/2009) and Frog’s (2015, 2021a) understanding of mythology in cultural practice and discourse from a folklore studies perspective. The Barthesian approach establishes myth as a mode of expression rather than as an object. This has naturalisation as a key feature. Otherwise-arbitrary relations between things are made to seem natural. Frog’s mythic discourse approach understands mythology as “constituted of signs that are emotionally invested by people within a society as models for knowing the world” (2021a, p. 161). Mythic discourse analysis focuses on the comparison of mythic discourses over time and across cultures.Barthes and Frog broadly share an understanding of mythology as a particular way of communicating an understanding of the world through discourse. Mythology is then not limited to any genre, medium or cultural context. It can include phenomena as diverse as systems, rules, customs, rituals, stories, characters, events, social roles and so on. What is important is how these elements relate to one another. Games consist of the same diverse elements arranged in comparable configurations, and so this perspective highlights the otherwise hidden parallels between mythology and games.I argue for analysing games as and through myth. Games as myth means viewing the game as an organising structure that works analogously to mythology. Elements are constructed and put into relation with one another within a gameworld, which the player then plays in and interprets. Games through myth means seeing games as embedded within cultural contexts. The cultural context of development affects the mythologies that can be seen to influence the construction of the game, while the cultural context of the player affects how they relate to the game and the mythologies channelled through it.A mytholudic approach helps us to understand how games make meaning because it focuses on the naturalised and hidden premises that go into the construction of games as organising structures. By analysing the underpinnings of those organising structures, we can outline the model for understanding the world that is virtually instantiated and how they are influenced by, influence and relate to models for understanding the world—mythologies—in the real world

Virtual Heritage

Virtual heritage has been explained as virtual reality applied to cultural heritage, but this definition only scratches the surface of the fascinating applications, tools and challenges of this fast-changing interdisciplinary field. This book provides an accessible but concise edited coverage of the main topics, tools and issues in virtual heritage. Leading international scholars have provided chapters to explain current issues in accuracy and precision; challenges in adopting advanced animation techniques; shows how archaeological learning can be developed in Minecraft; they propose mixed reality is conceptual rather than just technical; they explore how useful Linked Open Data can be for art history; explain how accessible photogrammetry can be but also ethical and practical issues for applying at scale; provide insight into how to provide interaction in museums involving the wider public; and describe issues in evaluating virtual heritage projects not often addressed even in scholarly papers. The book will be of particular interest to students and scholars in museum studies, digital archaeology, heritage studies, architectural history and modelling, virtual environments

Composing with soundscapes : capturing and analysing urban audio for a Raw Musical interpretation

The sound environment is increasingly more relevant in contemporary culture. It is a ‘picture’ of a place, an X-ray of its character and activities, shapeless and amorphous, whilst full of meanings and subtleties. It is a transmission that is complementary to what a picture retains and transports. Therefore it is a unique, tangible impression of the place to which it belongs, and intrinsically, its history, and that of the people who inhabit it, and their relations. Murray Schafer termed this element as Soundscape. Although always a present element, it was only from the end of the Twentieth Century that it has been studied in a systematic way, initially in a more ecological and socio-cultural function approach, but slowly it has been seen as an artistic tool, both in direct use of the sounds as well as being an inspiration, and a starting point, for creation. The exploratory research project presented here aims to study the capture and analysis of the urban soundscape, with a special focus on its temporal evolution, and its application in the creative and artistic context. At the first stage, we try to understand how and, if possible, to capture and represent the most challenging part of this massive sound element, which is its constant changing on the time axis. Grasping its nuances on both small and large time scales. The second phase is to understand the pertinence of such information and its direct influence in the act of music and sonic creation. To achieve these goals a multidisciplinary approach based on two main pillars was necessary: Soundscape and Computer Music. These disciplines determine the scientific areas in which this research had to be focused. Due to the fact that this thesis has a relatively broad scope of interests, there is a survey of the state of the art relating to projects and approaches that relate to the interests of this investigation in different ways. In areas such as: Music Information Retrieval, Algorithmic Composition, Eco Composition, and Sound Maps. This survey confirms the contemporaneity, relevance and originality of the project, both scientifically and artistically. In order to evaluate the proposed approach to the soundscape, URB, an experimental system to capture, analyse and store the urban soundscape was developed and implemented. The system is based on current technology, but is affordable and robust. To ensure the sustainability of the project, development was based in open source technology, to ensure the possibility of the project expanding and shaping according to the interests and needs of specific places, populations, research centres and/or artistic interests. Three main goals have guided the development of this system: addressing the lack of temporal perspective of the sound representation of urban spaces; the design of a free and autonomous system that depends only on the users, which results in a public free access database; to facilitate and systematize the access to data on the sound characteristics of an urban soundscape for both ecological studies as well as for creative and artistic purposes. A server keeps a record of the collected data, which can either be read directly, through an application developed for this purpose, or through a visual interface with a time-line approach, but always allowing for the possibility of creating original access tools that best serve the user’s purposes. After implementing the system, through four listening points, and confirming its robustness, we advance to the system’s application in the development of artistic works. For this purpose a challenge was proposed to a number of artists with distinct aesthetic backgrounds and training. This challenge resulted in different works that were presented to the public in different formats and media, such as works for classical instrumental, electro-acoustic works, improvisational moments with real-time sonification of the soundscape, sonic installations and sculptures powered by the system presented. This practice leads to a fundamental reflection in this research. We describe the different approaches chosen by each creator and primarily assess the pertinence in the artistic result of using a continuous and discrete system of urban soundscape analysis, from a creator’s point of view as well as from a listener’s point of view. With this research we are able to present a functional approach to the lack of temporal perspective in the usual sound maps that focus on the geographic origin of the recordings. We conclude that soundscapes can provide useful data for artistic creation and we promote a direct connection between that kind of approach and the artists. Even before the works developed and the artist’s feedback, whilst not ignoring the different approaches and opinions, we can conclude that this approach seems to promote new and functional soundscape composition ideas, making URB a relevant resource to compose original soundscape based music. Also, through analysis of the pieces, we could point out patterns of the technical and artistic choices on how to deal with information from the soundscape data, both in deferred time and real-time.A paisagem sonora é cada vez mais relevante na cultura contemporânea. É uma ‘fotografia’ de um lugar. Uma radiografia ao seu carácter e atividades, simultaneamente disforme e amorfa como repleta de significados e subtilezas. Uma transmissão complementar ao que a imagem retém e transporta. É portanto uma impressão única , tangível do lugar a que pertence e, intrinsecamente , da sua história , das pessoas que o habitam, das suas relações. Murray Schafer batizou este elemento como Soundscape (Paisagem Sonora). E embora tenha sido sempre um elemento presente, só a partir do final do séc. XX é que tem vindo a ser estudado de uma forma sistematizada, inicialmente numa abordagem mais ecológica e das suas funções socioculturais, mas lentamente encarado como uma ferramenta artística tanto no uso direto dos sons, como sendo o ponto de partida de inspiração para a criação. O projeto de investigação de carácter exploratório que aqui apresentamos visa o estudo da recolha e análise da paisagem sonora urbana, com especial enfoque na sua evolução temporal e da sua aplicação no contexto criativo artística. Como tal, numa fase inicial, procurar entender como e, se possível, capturar e representar o elemento mais desafiante que é a permanente mutação sonora no eixo temporal, apreendendo as suas nuances tanto numa escala de tempo diminuta como em grandes escalas temporais. Numa segunda fase entender a pertinência desta informação e da sua influência direta no ato da criação musical e sonora. Para cumprir estes objectivos foi necessária uma abordagem multidisciplinar fundamentada em dois pilares principais: Paisagem-Sonora e Informática Musical. Estas disciplinas determinam as áreas científicas em que esta investigação se debruçou. Tendo esta tese um âmbito de interesses relativamente amplo, há um levantamento do estado da arte referente a projetos e abordagens de áreas distintas que ,de diferentes formas , se relaciona com os interesses da nossa investigação. Em áreas como: Musical Information Retrieval, Composição Algorítmica, Eco Composição, Mapas Sonoros. Este levantamento confirma a atualidade, a pertinência e a originalidade do projeto, tanto cientificamente como artisticamente. A fim de avaliar a abordagem proposta à paisagem sonora, foi desenvolvido e posto em prática um sistema experimental de captura, análise e armazenamento de ambiente sonoro urbano, de nome URB. Sistema esse baseado em tecnologia atual, mas acessível e robusta para garantir a sustentabilidade do projeto e apoiado em desenvolvimento código aberto para garantir a possibilidade do projeto se expandir e se moldar de acordo com os interesses e necessidades concretas de locais, populações, centros de investigação e interesses artísticos. Três objetivos principais guiaram o desenvolvimento deste sistema: colmatar a falta de perspectiva temporal em grande escala de uma representação sonora de um espaço urbano; o desenho de um sistema que livre e autónomo, que apenas dependa dos utilizadores, que resulte numa base de dados de acesso público e gratuito; facilitar e sistematizar o acesso à informação sobre as caraterísticas sonoras de um ambiente sonoro urbano tanto para estudos ecológicos como para função criativa e artística. É mantido num servidor o registo de toda a informação recolhida, que tanto pode ser consultado diretamente, através de uma aplicação desenvolvida para o efeito, através de um interface visual ao estilo de linha temporal, deixando sempre a possibilidade para a criação de ferramentas de acesso originais que sirvam melhor os propósitos do utilizador. Após a implementação do sistema, através de quatro pontos de escuta, e a confirmação da sua robustez, avançamos para a aplicação do sistema no desenvolvimento de obras artísticas. Para tal, foi lançado um desafio a um conjunto de artistas que partem de diferentes formações e identidades estéticos. Esse desafio resultou em diferentes obras que foram apresentadas ao público com diferentes formatos e suportes, como obras integralmente para formatos instrumentais clássicos, obras electroacústicas, momentos de improvisação em tempo real com a sonorização do ambiente sonoro, instalações e esculturas sonoras alimentadas pelo sistema apresentado. Desta prática resulta uma reflexão fundamental nesta investigação. Descrevemos as diferentes abordagens escolhidas por cada criador e acima de tudo avaliamos a pertinência no resultado artístico do uso de um sistema contínuo e discreto de análise de ambiente sonoro urbano, tanto do ponto de vista do criador como do ponto de vista do ouvinte. A partir desta investigação, apresentamos uma abordagem funcional para a falta de perspectiva temporal nos mapas sonoros clássico que incidem principalmente sobre as origens geográfica das gravações. Concluímos que as paisagens sonoras podem fornecer dados úteis para a criação artística e promovemos uma conexão direta entre essa abordagem e os artistas. Perante os trabalhos e análises desenvolvidos por diferentes criadores, concluímos que esta aproximação promove ideias novas e funcionais para a composição. Além disso, através da análise de peças, apontamos padrões nas escolhas técnicas e estéticas sobre como lidar com as informações a partir dos dados da paisagem sonora, tanto em tempo diferido como em tempo real