23,793 research outputs found
An Experimental Investigation of Modality Effect: Evidence from Eye-Tracking Data
Modality effect is a response that occur when there are manipulations of sensory modality. In this thesis, I present a series of studies about the multimodal processing of visual and auditory presentation. The aim of this dissertation is to investigate how would the different stimulations from different source of modalities affect the oculomotor response. I investigate how different stimuli are processed, recognized and retrieved when they are presented across multiple modalities. Specifically, on question of how would the visual and auditory manipulations influence the oculomotor behaviour. In the research area of the multimodal processing, it has been argued that different kinds of sensory manipulations elicit a distinct kind of cognitive and behavioural response. The study of modality effect is particularly interesting topic for investigations since the world is multimodal in nature. Humans and other living beings are constantly exposed to a wide variety of stimuli rather than to isolated single stimulus. All experiments conducted used an eye-tracking approach since eye-tracking data are known as a reliable measure to study implicit cognitive processing. In Experiment 1, I investigate how different modalities and context interplay on the allocation of visual attention during the perceptual processing of congruent and incongruent multimodal stimuli. In Experiment 2, I investigate recognition memory of multimodal stimuli, focusing on the participants’ reaction to old versus novel stimuli presented in the visual and auditory modalities. In Experiment 3, I monitored looking patterns, to understand how visual and auditory stimuli are mentally reconstructed during mental imagery. I conclude the dissertation with a discussion of how a different kinds of modality manipulations elicit distinct modality effect as revealed by oculomotor response
Enabling mobile microinteractions
While much attention has been paid to the usability of desktop computers, mobile com- puters are quickly becoming the dominant platform. Because mobile computers may be used in nearly any situation--including while the user is actually in motion, or performing other tasks--interfaces designed for stationary use may be inappropriate, and alternative interfaces should be considered.
In this dissertation I consider the idea of microinteractions--interactions with a device that take less than four seconds to initiate and complete. Microinteractions are desirable because they may minimize interruption; that is, they allow for a tiny burst of interaction with a device so that the user can quickly return to the task at hand.
My research concentrates on methods for applying microinteractions through wrist- based interaction. I consider two modalities for this interaction: touchscreens and motion- based gestures. In the case of touchscreens, I consider the interface implications of making touchscreen watches usable with the finger, instead of the usual stylus, and investigate users' performance with a round touchscreen. For gesture-based interaction, I present a tool, MAGIC, for designing gesture-based interactive system, and detail the evaluation of the tool.Ph.D.Committee Chair: Starner, Thad; Committee Member: Abowd, Gregory; Committee Member: Isbell, Charles; Committee Member: Landay, james; Committee Member: McIntyre, Blai
Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin
The relationships between the listener, physical world and virtual
environment (VE) should not only inspire the design of natural multimodal
interfaces but should be discovered to make sense of the mediating action of VR
technologies. This chapter aims to transform an archipelago of studies related
to sonic interactions in virtual environments (SIVE) into a research field
equipped with a first theoretical framework with an inclusive vision of the
challenges to come: the egocentric perspective of the auditory digital twin. In
a VE with immersive audio technologies implemented, the role of VR simulations
must be enacted by a participatory exploration of sense-making in a network of
human and non-human agents, called actors. The guardian of such locus of agency
is the auditory digital twin that fosters intra-actions between humans and
technology, dynamically and fluidly redefining all those configurations that
are crucial for an immersive and coherent experience. The idea of entanglement
theory is here mainly declined in an egocentric-spatial perspective related to
emerging knowledge of the listener's perceptual capabilities. This is an
actively transformative relation with the digital twin potentials to create
movement, transparency, and provocative activities in VEs. The chapter contains
an original theoretical perspective complemented by several bibliographical
references and links to the other book chapters that have contributed
significantly to the proposal presented here.Comment: 46 pages, 5 figures. Pre-print version of the introduction to the
book "Sonic Interactions in Virtual Environments" in press for Springer's
Human-Computer Interaction Series, Open Access license. The pre-print
editors' copy of the book can be found at
https://vbn.aau.dk/en/publications/sonic-interactions-in-virtual-environments
- full book info: https://sive.create.aau.dk/index.php/sivebook
A Connectionist Theory of Phenomenal Experience
When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as
many of them have been doing recently, there are two fundamentally distinct approaches available. Either
consciousness is to be explained in terms of the nature of the representational vehicles the brain deploys; or
it is to be explained in terms of the computational processes defined over these vehicles. We call versions of
these two approaches vehicle and process theories of consciousness, respectively. However, while there may
be space for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because
of the influence exerted, on the one hand, by a large body of research which purports to show that the
explicit representation of information in the brain and conscious experience are dissociable, and on the
other, by the classical computational theory of mind – the theory that takes human cognition to be a species
of symbol manipulation. But two recent developments in cognitive science combine to suggest that a
reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the
experimental methodologies employed in the dissociation studies – so critical, in fact, it’s no longer
reasonable to assume that the dissociability of conscious experience and explicit representation has been
adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in
cognitive science as it once was. It now has a lively competitor in the form of connectionism; and
connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory
of consciousness. In this paper we develop and defend this connectionist vehicle theory of consciousness. It
takes the form of the following simple empirical hypothesis: phenomenal experience consists in the explicit
representation of information in neurally realized PDP networks. This hypothesis leads us to re-assess some
common wisdom about consciousness, but, we will argue, in fruitful and ultimately plausible ways
Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms
The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications
Minds Online: The Interface between Web Science, Cognitive Science, and the Philosophy of Mind
Alongside existing research into the social, political and economic impacts of the Web, there is a need to study the Web from a cognitive and epistemic perspective. This is particularly so as new and emerging technologies alter the nature of our interactive engagements with the Web, transforming the extent to which our thoughts and actions are shaped by the online environment. Situated and ecological approaches to cognition are relevant to understanding the cognitive significance of the Web because of the emphasis they place on forces and factors that reside at the level of agent–world interactions. In particular, by adopting a situated or ecological approach to cognition, we are able to assess the significance of the Web from the perspective of research into embodied, extended, embedded, social and collective cognition. The results of this analysis help to reshape the interdisciplinary configuration of Web Science, expanding its theoretical and empirical remit to include the disciplines of both cognitive science and the philosophy of mind
Recommended from our members
The role of HG in the analysis of temporal iteration and interaural correlation
- …