A predictive processing theory of sensorimotor contingencies: explaining the puzzle of perceptual presence and its absence in synesthesia

Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of “perceptual presence” has motivated “sensorimotor theories” which understand perception to involve the mastery of sensorimotor contingencies. However, the mechanistic basis of sensorimotor contingencies and their mastery has remained unclear. Sensorimotor theory also struggles to explain instances of perception, such as synesthesia, that appear to lack perceptual presence and for which relevant sensorimotor contingencies are difficult to identify. On alternative “predictive processing” theories, perceptual content emerges from probabilistic inference on the external causes of sensory signals, however, this view has addressed neither the problem of perceptual presence nor synesthesia. Here, I describe a theory of predictive perception of sensorimotor contingencies which (1) accounts for perceptual presence in normal perception, as well as its absence in synesthesia, and (2) operationalizes the notion of sensorimotor contingencies and their mastery. The core idea is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if those actions are not performed. These “counterfactually-rich” generative models encode sensorimotor contingencies related to repertoires of sensorimotor dependencies, with counterfactual richness determining the degree of perceptual presence associated with a stimulus. While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synesthetic concurrents are hypothesized to be counterfactually poor. In addition to accounting for the phenomenology of synesthesia, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. It may also lead to a new view of the (in)determinacy of normal perception

Fluid Dynamics: Representations of Water in Music

Water has remained a subject of all kinds of musical works since at least the middle ages. These musical works lack the concrete representational capacity of paintings, photographs, and films, relying instead on more abstract metaphorical constructs to convey water imagery. Current scholarship on water music typically centers on Romantic and Impressionist works and does not examine the process of signification by which musical signs portray water. The principal goal of this study is to determine how musical devices convey specific aspects of bodies of water and how such devices interact and contribute to musical depictions of streams, rivers, lakes, and oceans. I find that evocations of motion in the form of waves and flow are especially important to portrayals of water; furthermore, music depicting motion can combine with devices evoking water’s other characteristics to create detailed, multifarious depictions. I give special attention to John Luther Adams’s water compositions, which are notable for their thorough depictions of bodies of water and represent a relatively new phenomenon: the focused musical depiction

KEER2022

Avanttítol: KEER2022. DiversitiesDescripció del recurs: 25 juliol 202

Reading Mutant Narratives : The Bodily Experientiality of Contemporary Ecological Science Fiction

Reading Mutant Narratives explores how narratives of environmental and personal transformation in contemporary ecological science fiction can develop more-than-human modes of embodied experience. More specifically, it attends to the conflicted yet potentially transformative experientiality of "mutant narratives". Mutant narratives are viewed as uneasy hybrids of human-centered and posthumanist science fiction that contain potential for ecological understanding. Drawing on narrative studies and empirical reading studies, the dissertation begins from the premise that in suitable conditions, reading fiction may give rise to experiential change. The study traces and describes experiential changes that take place while reading works of science fiction. The bodily, subjective and historical conditions of reading are considered alongside the generic contexts and narrative features of the fictional works studied. As exemplary cases of mutant narratives, the study foregrounds the work of three American science fiction authors known for their critiques of anthropocentrism and for their articulations of more-than-human ecologies: Greg Bear, Paolo Bacigalupi, and Jeff VanderMeer. While much of contemporary fiction naturalizes embodied experience and hides their own narrative strategies, mutant narratives have the potential to defamiliarize readers’ notions of bodies and environments while also estranging their embodied experience of reading fiction. As a theoretical contribution to science fiction studies, the study considers such a readerly dynamic in terms of "embodied estrangement". Building on theoretical and practical work done in both embodied cognitive and posthumanist approaches to literature, the study shows how engagements with fictional narratives can, for their part, shape readers’ habitual patterns of feeling and perception. These approaches are synthesized into a method of close reading, "performative enactivism", that helps to articulate bodily, environmental, and more-than-human aspects of readerly engagement. Attending to such experiential aspects integrates ecological science fiction more deeply into the contemporary experiential situation of living with radical environmental transformation.Reading Mutant Narratives keskittyy ekologista kriisiä käsitteleviin "mutanttikertomuksiin" ja niille tyypilliseen kokemuksellisuuteen. Mutanttikertomukset ovat tieteisfiktiivisiä kertomuksia, joissa ihmiskeskeiset ja posthumanistiset piirteet yhdistyvät ja antavat lukijalle mahdollisuuksia ekologiseen ymmärrykseen. Esimerkkeinä mutanttikertomuksista tutkimus nostaa esiin kolmen yhdysvaltalaisen tieteiskirjailijan, Greg Bearin, Paolo Bacigalupin ja Jeff VanderMeerin, teoksia. Nämä teokset asettavat ihmisen osaksi ekologisia, evolutiivisia ja teknologisia vuorovaikutussuhteita, joissa myös ihmisruumiit ja ruumiillinen kokemus muuttavat muotoaan. Väitöskirja tutkii siis, kuinka tieteisfiktiiviset kertomukset ympäristöllisestä ja kokemuksellisesta muutoksesta voivat kehittää ruumiillista kokemusta. Kertomuksentutkimukseen ja empiiriseen lukijatutkimukseen tukeutuen tutkimus lähtee oletuksesta, että sopivissa olosuhteissa kirjallisuuden lukeminen voi edesauttaa kokemuksellisia muutoksia. Tutkimus tarkastelee tieteisfiktion lukemista eletyn ruumiillisen kokemuksen tasolla. Lukukokemuksen analyysissa otetaan huomioon sekä lukijan ruumiillinen, subjektiivinen ja historiallinen tilanne että luettujen teosten kytkeytyminen romaanikerronnan ja tieteisfiktion lajityypin perinteisiin. Suuri osa nykykirjallisuudesta esittää ruumiillisen kokemuksen luonnollisena ja kätkee oman kerronnallisen vaikutusvaltansa, mutta mutanttikertomusten kerronnalliset keinot saattavat outouttaa lukijoiden ruumiillista kokemusta suhteessa sekä elettyyn ympäristöön että kirjallisuuteen itseensä. Tieteisfiktion tutkimuksen käsitteistöä uudistaen väitöskirja kutsuu tällaista lukemisen dynamiikkaa "ruumiilliseksi vieraannuttamiseksi". Lukukokemuksen analyysien avulla tutkimus esittää, kuinka kirjallisuuden lukeminen ohjaa osaltaan lukijoiden tunne- ja havaintotottumusten muotoutumista. Se tuo yhteen ruumiillis-kognitiivisia ja posthumanistisia lähestymistapoja kirjallisuudentutkimukseen ja muotoilee "performatiivis-enaktiivisen" lähilukemisen metodin, joka auttaa sanallistamaan lukukokemuksen ruumiillisia, ympäristöllisiä ja ei-inhimilliseen kurovia puolia. Ruumiillisen kokemuksellisuuden syventäminen tämän metodin avulla tuo ekologisen tieteisfiktion tiiviimmin osaksi globaalin ympäristömuutoksen kokemuksellista tilannetta

NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

Metaphors of Mind in Fiction and Psychology

Curiosity about the human mind—what it is and how it functions—began long before modern psychology. But because the mind and its processes are so elusive, they could be described only by means of metaphor. Michael Kearns, in this prize-winning study, examines the development of metaphors of the mind in psychological writings from Hobbes through William James and in fiction from Defoe through Henry James. Throughout the eighteenth century and even into the early nineteenth, metaphors of the mind as a relatively simple entity, either mechanical or biological, dominated both those engaged in psychological theorizing and novelists ranging from Richardson and Smollett through Dickens and the Brontes. In the nineteenth century, such psychologists as Herbert Spencer and Alexander Bain conceived of the mind as a complex organism quite different from that embodied in earlier thinking, but their figurative language did not keep pace. The result was a tension between theoretical expression and actual discussion of mental phenomena. Michael S. Kearns is assistant professor of English at Ohio Wesleyan University. Winner of the first annual Midwest Modern Language Association Book Awardhttps://uknowledge.uky.edu/upk_english_language_and_literature_british_isles/1038/thumbnail.jp