An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe

Image Clusters: A Hermeneutical Perspective on Changes to a Social Function of Photography

Photography can be used for a wide variety of social functions. The hermeneutic method of image cluster analysis presented deals with a comparatively new social use and understanding of photography: as digital montage forming intricate compilations. At the methodological heart of this approach is the 1. figurative analysis of the compositional principles of particular image compilations, i.e., their expressive meaning. This ideographic perspective is expanded and supplemented by 2. investigating the structure of the mediated field of perception and action of each image cluster, as well as the styles of observing and knowing that they predicate. Methodologically speaking, the approach relies on an assumption that far from being an invention of technological media, the "game" (WITTGENSTEIN) which is constitutive of iconic image clusters, complete with relationships of similarity and difference, has its anthropological basis and primary social expression in people's body language and in minor social distinctions. According to this hypothesis, in recent image clusters photography is no longer necessarily understood as depicting or documenting occurrences in the lifeworld, but is instead consolidated as a collectively shared means of expression which can be repeatedly recombined to form new figures of expression—in other words, it achieves a decidedly idiomatic quality.Die Vielzahl möglicher sozialer Gebrauchsweisen der Fotografie ist beträchtlich. Das vorliegend dargestellte hermeneutische Verfahren der Bildclusteranalyse bezieht sich auf eine vergleichsweise neuartige Form der Verwendung und des Verständnisses der Fotografie: auf ihre digitale Montage zu komplexen Bildzusammenstellungen und auf die Präsentation solcher Bildzusammenstellungen in technisierten Umgebungen des sozialen Austauschs. Im methodischen Zentrum des Verfahrens steht 1. die figurative Analyse der Kompositionsprinzipien jeweiliger Bildzusammenstellungen, d.h. ihres Ausdruckssinns. Erweitert und ergänzt wird diese ideografische Perspektive 2. durch die Untersuchung der Struktur des medialen Wahrnehmungs- und Handlungsraumes jeweiliger Bildcluster sowie der Beobachtungs- und Erkenntnisstile, die sie prädizieren. Methodologisch gesehen stützt sich das Verfahren auf die Annahme, dass das für ikonische Bildcluster konstitutive "Spiel" (WITTGENSTEIN) mit Ähnlichkeitsbeziehungen und Differenzverhältnissen keine medientechnische Erfindung ist, sondern seine anthropologische Grundlage und primäre soziale Ausprägung in der Körpersprache des Menschen und den feinen Unterschieden der sozialen Distinktion findet. Dementsprechend wird die Fotografie in rezenten Bildclustern nicht mehr notwendig als Abbildung oder Dokument lebensweltlicher Begebenheiten verstanden, sondern, so die These, verstärkt als kollektiv geteiltes Ausdrucksmittel, das immer wieder zu neuen Ausdrucksfiguren zusammengestellt werden kann – das also eine dezidiert idiomatische Qualität erlangt

Participant responses to virtual agents in immersive virtual environments.

This thesis is concerned with interaction between people and virtual humans in the context of highly immersive virtual environments (VEs). Empirical studies have shown that virtual humans (agents) with even minimal behavioural capabilities can have a significant emotional impact on participants of immersive virtual environments (IVEs) to the extent that these have been used in studies of mental health issues such as social phobia and paranoia. This thesis focuses on understanding the impact on the responses of people to the behaviour of virtual humans rather than their visual appearance. There are three main research questions addressed. First, the thesis considers what are the key nonverbal behavioural cues used to portray a specific psychological state. Second, research determines the extent to which the underlying state of a virtual human is recognisable through the display of a key set of cues inferred from the behaviour of real humans. Finally, the degree to which a perceived psychological state in a virtual human invokes responses from participants in immersive virtual environments that are similar to those observed in the physical world is considered. These research questions were investigated through four experiments. The first experiment focused on the impact of visual fidelity and behavioural complexity on participant responses by implementing a model of gaze behaviour in virtual humans. The results of the study concluded that participants expected more life-like behaviours from more visually realistic virtual humans. The second experiment investigated the detrimental effects on participant responses when interacting with virtual humans with low behavioural complexity. The third experiment investigated the differences in responses of participants to virtual humans perceived to be in varying emotional states. The emotional states of the virtual humans were portrayed using postural and facial cues. Results indicated that posture does play an important role in the portrayal of affect however the behavioural model used in the study did not fully cover the qualities of body movement associated with the emotions studied. The final experiment focused on the portrayal of affect through the quality of body movement such as the speed of gestures. The effectiveness of the virtual humans was gauged through exploring a variety of participant responses including subjective responses, objective physiological and behavioural measures. The results show that participants are affected and respond to virtual humans in a significant manner provided that an appropriate behavioural model is used

Visual recognition of words learned with gestures induces motor resonance in the forearm muscles

Acknowledgements We thank Sabrina Füreder and Edanur Aktan for their contribution to the data collection.Peer reviewedPublisher PD

Landscapes of Affective Interaction: Young Children's Enactive Engagement with Body Metaphors

Empirical research into embodied meaning making suggests specific sensorimotor experiences can support children’s understanding of abstract science ideas. This view is aligned with enactive and grounded cognition perspectives, both centred in the view that our ability to conceptualise emerges from our experiences of interaction with our environment. While much of this research has focused on understanding action and action processes in individual children or children in pairs, less attention has been paid to affective dimensions of young children’s group interaction, and how this relates to meaning making with body metaphors. Indeed, Gallagher describes how no action exists in a vacuum, but rather revolves around a complex web of affective-pragmatic features comprising a ‘Landscape of Interaction’ (2020, p.42). This research project addresses gaps in research in understanding young children’s affective engagement from an enactivist cognition perspective. It takes a Design-Based Research approach with an iterative design orientation to examine young children’s interaction with multisensory body-based metaphors through an embodied participation framework. A series of empirical studies with young children, aged 2-7 years, comprising of experiential workshops, build iteratively upon each other. A novel theoretically informed method, Affective Imagination in Motion, is developed involving several purpose-built multisensory body metaphors prompts to enable access to dimensions of young children’s affective engagement. This research makes theoretical and methodological contributions. It extends the theoretical notion of ‘affect’ from enactive and grounded cognition perspectives through identifying key interactive processes in young children’s engagement with multisensory action metaphors. In addition, the novel method offers a contribution as a way of ‘looking’ at affect within a group situation from affective-pragmatic and social embodiment perspectives. Finally, the research contributes to embodied learning design frameworks offering a guideline for designers wishing to inform their work from enactive cognition perspective

A gaze-contingent framework for perceptually-enabled applications in healthcare

Patient safety and quality of care remain the focus of the smart operating room of the future. Some of the most influential factors with a detrimental effect are related to suboptimal communication among the staff, poor flow of information, staff workload and fatigue, ergonomics and sterility in the operating room. While technological developments constantly transform the operating room layout and the interaction between surgical staff and machinery, a vast array of opportunities arise for the design of systems and approaches, that can enhance patient safety and improve workflow and efficiency. The aim of this research is to develop a real-time gaze-contingent framework towards a "smart" operating suite, that will enhance operator's ergonomics by allowing perceptually-enabled, touchless and natural interaction with the environment. The main feature of the proposed framework is the ability to acquire and utilise the plethora of information provided by the human visual system to allow touchless interaction with medical devices in the operating room. In this thesis, a gaze-guided robotic scrub nurse, a gaze-controlled robotised flexible endoscope and a gaze-guided assistive robotic system are proposed. Firstly, the gaze-guided robotic scrub nurse is presented; surgical teams performed a simulated surgical task with the assistance of a robot scrub nurse, which complements the human scrub nurse in delivery of surgical instruments, following gaze selection by the surgeon. Then, the gaze-controlled robotised flexible endoscope is introduced; experienced endoscopists and novice users performed a simulated examination of the upper gastrointestinal tract using predominately their natural gaze. Finally, a gaze-guided assistive robotic system is presented, which aims to facilitate activities of daily living. The results of this work provide valuable insights into the feasibility of integrating the developed gaze-contingent framework into clinical practice without significant workflow disruptions.Open Acces

Into theatre, under the knife: cultures of dissection and contemporary art practice

Dissection can be considered a process of calculated dividing and a form of meticulous dismantling. As a method of analysis, it remains inextricably linked to explorations of the human body and the investigations staged to uncover its hidden depths. This research examines historical cultures of dissection and their relationship to contemporary art practice. It proposes that such a culture is characterised by an inherent cutting spectacle; one that remains grounded within a complex and pre-existing visual culture that has dissected, divided and dismantled the body and its image. Into theatre, Under the Knife: Cultures of Dissection and Contemporary Art Practice is the outcome of a practice-led research project comprising this written thesis and a diverse body of work that spans sculpture, assemblage, installation and painting. Across the development of both the textual and creative work, cutting has been employed as a methodology for the research, and forms a significant basis for the material and conceptual inquiry of the study. Over the course of this thesis historical source material, theoretical propositions and the work of contemporary artists are gradually and methodically dissected for examination. The paper that has then developed as a result of this process-led research proposes a series of significant intersections, and a framework of ‘cuttings and cross-cuttings’ to consider art practice as a form of anatomical enquiry. In suggesting these points of connection, and by intentionally traversing the historical and contemporary, new ground is proposed to consider a culture of dissection within contemporary art through an analyses of the seminal practitioners who maintain and continue to enact their own distinct practice of cutting. Whether undertaken as a light incision to permeate a surface, or a dramatic slash that severs, this research will attempt to reveal that cutting is a significant yet under-recognised gesture being used by contemporary artists. Much like the surgeon or anatomist who performs incisions in theatre, artists too inflict cutting gestures with strategic and decisive intent. The cutting gesture that embodies both dual creative and destructive possibilities acts also as a sign of authorship, a signature of authenticity from one who wields the knife

The Spectacle of Suffering: Repetition and Closure in the Eighteenth-Century Gothic Novel

Since the publication of The Castle of Otranto in 1764 initiated the genre of the gothic novel, critics have claimed that gothic endings are bland, inadequate, or otherwise unsatisfying. Analyzing works written in the period 1764 to 1820 by Horace Walpole, Ann Radcliffe, Matthew Lewis, Charlotte Dacre, Mary-Anne Radcliffe, Mary and Percy Shelley and Charles Robert Maturin, this dissertation demonstrates that the focus on endings has blinded critics to the reader\u27s source of pleasure in the gothic. I have drawn upon a representative sampling of novels to present a model of the interaction of reader and gothic text focused on the reader\u27s engagement with spectacle, defined here as insistently visual scenes of suffering framed and set apart from the text. This engagement is shown to be the controlling factor in the reader\u27s reaction to closure. The theoretical framework of this discussion is provided by current feminist film theory, based in the psychoanalytic work of Freud and Lacan, which shares with the gothic an interest in the act of looking as an expression of the desire to know. Evidence from the texts, viewed from this perspective, suggests that the encounter with spectacle acts as a switchpoint for an exchange of roles between reader and text. This engagement stimulates the reader\u27s desire to continue reading while simultaneously promising the fulfillment of that desire. The intermittent but frequent repetition of the spectacle of suffering involves the reader in a paradoxical, seemingly endless pursuit of the satisfaction, through spectacle, of a desire whose stimulus lies within spectacle. The intense engagement with spectacle, in which the reader is both aggressor and victim, relegates a novel\u27s ending to the level of secondary interest. The reader\u27s enjoyment is based on the repeated experience of spectacle within the text and across the genre; the end of one novel promises the beginning of another. The dynamic created by spectacle produces in the reader a desire to continue that is stronger than the desire for the end. Models of closure which focus on endings misinterpret both the gothic and its readers

Encountering snakes in early Victorian London: The first reptile house at the Zoological Gardens

This is the author accepted manuscript. The final version is available from SAGE via http://dx.doi.org/10.1177/0073275315580958 This paper examines the first reptile house (1849) at the Zoological Gardens in London as a novel site for the production and consumption of knowledge about snakes, stressing the significance of architectural and material limitations on both snakes and humans. Snakes were familiar and ambiguous, present at every level of British society through the reading of Scripture and as recurrent characters in imperial print culture. For all that snakes engendered feelings of disgust as the most distinctive representatives of a lowly class of animals, they exerted an equivalent fascination over diverse publics spanning the social spectrum. Building on work showing a consideration for the multi-sensory nature of visits to menageries, this paper considers animal display beyond the visual. It explores the emotional economy of encountering snakes in person and the bodily phenomena this engendered. Vicarious visits were offered up to readers of periodicals and newspapers and the reptile house was harnessed as a controversial pedagogical resource for teaching moral, as well as scientific, lessons. </jats:p