Binding tactile and visual sensations via unique association by cross-anchoring between double-touching and self-occlusion

Binding is one of the most fundamental cognitive functions, how to find the correspondence of sensations between different modalities such as vision and touch. Without a priori knowledge on this correspondence, binding is regarded to be a formidable issue for a robot since it often perceives multiple physical phenomena in its different modal sensors, therefore it should correctly match the foci of attention in different modalities that may have multiple correspondences each other. We suppose that learning the multimodal representation of the body should be the first step toward binding since the morphological constraints in self-body-observation would make the binding problem tractable. The multimodal sensations are expected to be constrained in perceiving own body so as to configurate the unique parts of the multiple correspondence reflecting its morphology. In this paper, we propose a method to match the foci of attention in vision and touch through the unique association by cross-anchoring different modalities. Simple experiments show the validity of the proposed method

How touch and hearing influence visual processing in sensory substitution, synaesthesia and cross-modal correspondences

Sensory substitution devices (SSDs) systematically turn visual dimensions into patterns of tactile or auditory stimulation. After training, a user of these devices learns to translate these audio or tactile sensations back into a mental visual picture. Most previous SSDs translate greyscale images using intuitive cross-sensory mappings to help users learn the devices. However more recent SSDs have started to incorporate additional colour dimensions such as saturation and hue. Chapter two examines how previous SSDs have translated the complexities of colour into hearing or touch. The chapter explores if colour is useful for SSD users, how SSD and veridical colour perception differ and how optimal cross-sensory mappings might be considered. After long-term training, some blind users of SSDs report visual sensations from tactile or auditory stimulation. A related phenomena is that of synaesthesia, a condition where stimulation of one modality (i.e. touch) produces an automatic, consistent and vivid sensation in another modality (i.e. vision). Tactile-visual synaesthesia is an extremely rare variant that can shed light on how the tactile-visual system is altered when touch can elicit visual sensations. Chapter three reports a series of investigations on the tactile discrimination abilities and phenomenology of tactile-vision synaesthetes, alongside questionnaire data from synaesthetes unavailable for testing. Chapter four introduces a new SSD to test if the presentation of colour information in sensory substitution affects object and colour discrimination. Chapter five presents experiments on intuitive auditory-colour mappings across a wide variety of sounds. These findings are used to predict the reported colour hallucinations resulting from LSD use while listening to these sounds. Chapter six uses a new sensory substitution device designed to test the utility of these intuitive sound-colour links for visual processing. These findings are discussed with reference to how cross-sensory links, LSD and synaesthesia can inform optimal SSD design for visual processing

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

On the Verge of Photography: Imaging Beyond Representation

Imaging Beyond Representation is a provocative and bold rethinking of photography in light of the digital transformation and its impact on fine art, culture and society. Addressing the centrality of the digital image to our contemporary life, the fourteen new essays in this collection challenge the traditional categories of photographic theory - that of representation, evidence, documentation and the archive - and offer a fresh approach to its impact on aesthetics, contemporary philosophy and the political. Drawing on the networked human condition of embodiment, social-media, and bio-politics, On the Verge of Photography offers an invaluable resource for sutdents of visual culture, researchers in the field of digital imagining and artists working with new media. Reading this extraordinary book, it becomes clear that so much of what we knew or thought we knew about photography is at one and the same time accurate and obsolete. With digital photography the image can no longer be discussed or defined for what is it is conventionally assumed to be - a distinct visual unit. This is not a crisis, claim the editors of this timely volume, but an opportunity to step away from the representational terminology that has over-determined the discourse of photography in order to address the image's actual modes of being and becoming: being digitally-born, constantly transmitted, mutated and shared. When images are 'digitally networked' they cannot be isolated as viewed as distinct or unique. This book is a must read for anyone who shares with the authors collected in it an urge to acknowledge the contemporary image as a kind of living organism that intervenes int eh world we share not only by and through the ways we share them. -- Ariella Azoulay, Media/Comparative Literature and Modern Culture Brown Universit

"The distant skin": a deconstructive analysis of women and polysemic touch in the writing of John McGahern and Anne Enright

This thesis, by providing a deconstructive reading of the work of John McGahern and Anne Enright, elucidates the way in which the place, position and representation of women in modern Irish society is profoundly affected by personal, political, religious and even legal societal forces. The project attempts to utilise the work of both authors to access and reveal the ‘Real’ experience of Irish women, in particular emphasising the impact of physical, emotional and metaphoric touch upon both their bodies and minds. By analysing the work of these writers through the lens of literary theorists such as Julia Kristeva, Jacques Lacan, Luce Irigaray, Jacques Derrida, Hélène Cixous, Michel Foucault and Louis Althusser, this work will attempt to chart the changing perception of women and the lived female experience in modern Irish society from the 1960s right up to the present day, elucidating both the covert methods by which Irish women are currently repressed or silenced within society, and the myriad of ways in which they rebel against such repressive forces. In order to provide a comprehensive investigation of the lived experience of women in modern Irish society, this work aims to look at the ways in which women are touched physically, emotionally and mentally by societal forces. It will focus on such issues as self-identity and touching the inner self, violent or oppressive physical or sexual touch, the complex physical and emotional changes associated with the inner touch that is pregnancy, and finally the stigmas and difficulties facing Irish women who are deemed to be “Touched” by mental illness in Irish society. These analyses and examinations are undertaken with a view to building upon previous socio-cultural and literary academic works in relation to the representation of women in modern Irish society, whilst simultaneously opening new debates and iv discussions in relation to how the Irish female experience has changed in the advent of the twenty-first century.N

Ubiquitous computing and natural interfaces for environmental information

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do Grau de Mestre em Engenharia do Ambiente, perfil Gestão e Sistemas AmbientaisThe next computing revolution‘s objective is to embed every street, building, room and object with computational power. Ubiquitous computing (ubicomp) will allow every object to receive and transmit information, sense its surroundings and act accordingly, be located from anywhere in the world, connect every person. Everyone will have the possibility to access information, despite their age, computer knowledge, literacy or physical impairment. It will impact the world in a profound way, empowering mankind, improving the environment, but will also create new challenges that our society, economy, health and global environment will have to overcome. Negative impacts have to be identified and dealt with in advance. Despite these concerns, environmental studies have been mostly absent from discussions on the new paradigm. This thesis seeks to examine ubiquitous computing, its technological emergence, raise awareness towards future impacts and explore the design of new interfaces and rich interaction modes. Environmental information is approached as an area which may greatly benefit from ubicomp as a way to gather, treat and disseminate it, simultaneously complying with the Aarhus convention. In an educational context, new media are poised to revolutionize the way we perceive, learn and interact with environmental information. cUbiq is presented as a natural interface to access that information

Perceptual Organization

Perceiving the world of real objects seems so easy that it is difficult to grasp just how complicated it is. Not only do we need to construct the objects quickly, the objects keep changing even though we think of them as having a consistent, independent existence (Feldman, 2003). Yet, we usually get it right, there are few failures. We can perceive a tree in a blinding snowstorm, a deer bounding across a tree line, dodge a snowball, catch a baseball, detect the crack of a branch breaking in a strong windstorm amidst the rustling of trees, predict the sounds of a dripping faucet, or track a street musician strolling down the road

Body-ownership and visual perception

The idea that our body plays an important role in visual perception has a long history in storytelling and philosophy. Some ideas are very intuitive. For example, few will disagree with the notion that smaller people perceive the world to be bigger, and vice versa, that larger people perceive the world to be smaller. In contrast, more controversial ideas regarding the role of our body in visual perception have been debated by philosophers. According to these philosophers, the very nature of visual perception lies in the fact that we have a body that moves in space. Since George Berkeley (1685 – 1753) first formalized such a fundamental role for our body in visual perception, different philosophical theories have branched out to account for the latest scientific findings. Although experimental psychologists and neuroscientists have long neglected these intriguing philosophical accounts, the recent development of body-ownership illusions allowed for a more rigorous investigation of the supposed link between our body and visual perception. In body-ownership illusions, research participants experience an artificial body (or body-part) to be their own. These illusions allow for the dissociation between the subjective experience that your body belongs to you (i.e. body-ownership) and the mere visual impression of your body from a first-person perspective. The studies that comprise this thesis use different ownership illusions to investigate the role of body-ownership in visual perception, and the role of visual perception in body-ownership, with visual perception being an umbrella-term for both visuospatial perception and visual awareness. In Study I and Study II we investigated the mechanisms by which body-size influences the perceived size of the world, by having participants experience ownership of different sized (and sometimes invisible) bodies. Our results show that this own-body-size effect does not rely on visual information per se, but instead, on the recalibration between visual and tactile information that updates the representation of external space. In Study III we combine the rubber hand illusion with binocular rivalry to show that body-ownership promotes visual awareness of a fake hand. And in Study IV we combine the rubber hand illusion with continuous flash suppression to show that ownership can be induced in the absence of visual awareness. Such unconscious ownership calls for a reevaluation of the standard definition of body-ownership. Taken together, these studies illuminate the intricate relationship between body-ownership and visual perception. In addition to the scientific research fields of visuospatial perception, visual awareness, and body-ownership, these results are valuable to the philosophical debate on the nature of visual perception and might provide future applications in clinical psychology

A Framework For Abstracting, Designing And Building Tangible Gesture Interactive Systems

This thesis discusses tangible gesture interaction, a novel paradigm for interacting with computer that blends concepts from the more popular fields of tangible interaction and gesture interaction. Taking advantage of the human innate abilities to manipulate physical objects and to communicate through gestures, tangible gesture interaction is particularly interesting for interacting in smart environments, bringing the interaction with computer beyond the screen, back to the real world. Since tangible gesture interaction is a relatively new field of research, this thesis presents a conceptual framework that aims at supporting future work in this field. The Tangible Gesture Interaction Framework provides support on three levels. First, it helps reflecting from a theoretical point of view on the different types of tangible gestures that can be designed, physically, through a taxonomy based on three components (move, hold and touch) and additional attributes, and semantically, through a taxonomy of the semantic constructs that can be used to associate meaning to tangible gestures. Second, it helps conceiving new tangible gesture interactive systems and designing new interactions based on gestures with objects, through dedicated guidelines for tangible gesture definition and common practices for different application domains. Third, it helps building new tangible gesture interactive systems supporting the choice between four different technological approaches (embedded and embodied, wearable, environmental or hybrid) and providing general guidance for the different approaches. As an application of this framework, this thesis presents also seven tangible gesture interactive systems for three different application domains, i.e., interacting with the In-Vehicle Infotainment System (IVIS) of the car, the emotional and interpersonal communication, and the interaction in a smart home. For the first application domain, four different systems that use gestures on the steering wheel as interaction means with the IVIS have been designed, developed and evaluated. For the second application domain, an anthropomorphic lamp able to recognize gestures that humans typically perform for interpersonal communication has been conceived and developed. A second system, based on smart t-shirts, recognizes when two people hug and reward the gesture with an exchange of digital information. Finally, a smart watch for recognizing gestures performed with objects held in the hand in the context of the smart home has been investigated. The analysis of existing systems found in literature and of the system developed during this thesis shows that the framework has a good descriptive and evaluative power. The applications developed during this thesis show that the proposed framework has also a good generative power.Questa tesi discute l’interazione gestuale tangibile, un nuovo paradigma per interagire con il computer che unisce i principi dei più comuni campi di studio dell’interazione tangibile e dell’interazione gestuale. Sfruttando le abilità innate dell’uomo di manipolare oggetti fisici e di comunicare con i gesti, l’interazione gestuale tangibile si rivela particolarmente interessante per interagire negli ambienti intelligenti, riportando l’attenzione sul nostro mondo reale, al di là dello schermo dei computer o degli smartphone. Poiché l’interazione gestuale tangibile è un campo di studio relativamente recente, questa tesi presenta un framework (quadro teorico) che ha lo scopo di assistere lavori futuri in questo campo. Il Framework per l’Interazione Gestuale Tangibile fornisce supporto su tre livelli. Per prima cosa, aiuta a riflettere da un punto di vista teorico sui diversi tipi di gesti tangibili che possono essere eseguiti fisicamente, grazie a una tassonomia basata su tre componenti (muovere, tenere, toccare) e attributi addizionali, e che possono essere concepiti semanticamente, grazie a una tassonomia di tutti i costrutti semantici che permettono di associare dei significati ai gesti tangibili. In secondo luogo, il framework proposto aiuta a concepire nuovi sistemi interattivi basati su gesti tangibili e a ideare nuove interazioni basate su gesti con gli oggetti, attraverso linee guida per la definizione di gesti tangibili e una selezione delle migliore pratiche per i differenti campi di applicazione. Infine, il framework aiuta a implementare nuovi sistemi interattivi basati su gesti tangibili, permettendo di scegliere tra quattro differenti approcci tecnologici (incarnato e integrato negli oggetti, indossabile, distribuito nell’ambiente, o ibrido) e fornendo una guida generale per la scelta tra questi differenti approcci. Come applicazione di questo framework, questa tesi presenta anche sette sistemi interattivi basati su gesti tangibili, realizzati per tre differenti campi di applicazione: l’interazione con i sistemi di infotainment degli autoveicoli, la comunicazione interpersonale delle emozioni, e l’interazione nella casa intelligente. Per il primo campo di applicazione, sono stati progettati, sviluppati e testati quattro differenti sistemi che usano gesti tangibili effettuati sul volante come modalità di interazione con il sistema di infotainment. Per il secondo campo di applicazione, è stata concepita e sviluppata una lampada antropomorfica in grado di riconoscere i gesti tipici dell’interazione interpersonale. Per lo stesso campo di applicazione, un secondo sistema, basato su una maglietta intelligente, riconosce quando due persone si abbracciano e ricompensa questo gesto con uno scambio di informazioni digitali. Infine, per l’interazione nella casa intelligente, è stata investigata la realizzazione di uno smart watch per il riconoscimento di gesti eseguiti con oggetti tenuti nella mano. L’analisi dei sistemi interattivi esistenti basati su gesti tangibili permette di dimostrare che il framework ha un buon potere descrittivo e valutativo. Le applicazioni sviluppate durante la tesi mostrano che il framework proposto ha anche un valido potere generativo