Unsupervised inference of auditory attention from biosensors

Abstract. We study ways of automatically inferring the level of attention a user is paying to auditory content, with applications for example in automatic podcast highlighting and auto-pause, as well as in a selection mechanism in auditory interfaces. In particular, we demonstrate how the level of attention can be inferred in an unsupervised fashion, without requiring any labeled training data. The approach is based on measuring the (generalized) correlation or synchrony between the auditory content and physiological signals reflecting the state of the user. We hypothesize that the synchrony is higher when the user is paying attention to the content, and show empirically that the level of attention can indeed be inferred based on the correlation. In particular, we demonstrate that the novel method of time-varying Bayesian canonical correlation analysis gives unsupervised prediction accuracy comparable to having trained a supervised Gaussian process regression with labeled training data recorded from other users

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with

AXMEDIS 2007 Conference Proceedings

The AXMEDIS International Conference series has been established since 2005 and is focused on the research, developments and applications in the cross-media domain, exploring innovative technologies to meet the challenges of the sector. AXMEDIS2007 deals with all subjects and topics related to cross-media and digital-media content production, processing, management, standards, representation, sharing, interoperability, protection and rights management. It addresses the latest developments and future trends of the technologies and their applications, their impact and exploitation within academic, business and industrial communities

RIPE--rapid instruction production environment

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1997.Includes bibliographical references (leaves 74-75).by Bruce Alan McHenry.M.S

Consumer behavior in electronic and virtual commerce: merging behavioural and neurophysiological perspectives

In the last two decades, pragmatic shifts in technology have assisted Business-to-Consumers retailers to seek novel avenues for expanding their businesses and excite consumers with newer shopping experiences. On one hand, due to the continuous technological developments retailers have leveraged the power of Web 2.0 to make a shift from traditional brick-and-motor to e-commerce retail stores and empowered consumers with user-generated contents (UGC). With the advent of Web 3.0, retailers are attempting to set up stores in a three-dimensional space of extended reality by using virtual reality (VR), augmented reality (AR), or mixed reality (MR), thereby creating virtual world(s) for consumers to interact and share using Artificial Intelligence (AI). Evidently, this has resulted in a change of consumers’ patterns of shopping and has directly necessitated the need for researchers to develop apparatus to capture and understand the ever-evolving nature of consumer behavior. Parallelly, marketing research has taken huge strides to incorporate tools and techniques to observe consumers’ conscious and subconscious measurements. While conventional marketing techniques involved self-reports, focus group and panel data to understand explicitly stated observations, the advent of neurophysiological tools, such as Electroencephalography (EEG), Heart rate Variability (HRV) and Eye tracking (ET) has made it possible to record and quantify consumer’s unstated subconscious observations vis-à-vis marketing stimulus. These advancements exhibit future course of research as well as carries with itself newfound challenges such as identifying critical aspects of UGC that influences purchase behavior, triangulating neurophysiological data with explicit measurements, and dynamic consumption patterns in immersive marketplaces. Therefore, the development of this thesis partakes various domains– retailing, neuromarketing and virtual reality – only to contribute a fresh perspective on understanding consumer behavior and therefore, overcoming some of the challenges. The overarching aim of this thesis is to extend academic scholarship on consumer behavior through novel theoretical frameworks and methodological techniques. From conceptual perspective, the studies move on to examining purchases behavior in e-commerce to virtual commerce environments. In the e-commerce format, analyzing the attributes of product reviews on purchases decisions for different product categories broadened the classical cue-utilization framework (Chapter 2). In virtual reality format, critical differences in planned and unplanned purchase behavior while shopping in a virtual supermarket are stated (Chapter 4). From a methodological point-of-view, this thesis incorporates implicit (neuro physiological) measurements along with traditional explicit (self-report) measurements. Utilizing neuroscientific tools served two interconnected goals for this thesis – (a) updating classical marketing frameworks for today’s practitioners (Chapters 3 and 4); and (b) for evaluating the impact of marketing interventions (advertisement and retailing) on consumer purchase intentions (Chapter 4 and, in Appendix 1, 2 and 3). The usage of neuroscientific tools, in Chapter 3 and 4 (also in Appendix 1, 2 and 3) allowed for directly capturing real-time subconscious decision-making of consumers’ during the purchase stage of consumer journey. This allowed for updating the classical consumer behavior models (e.g., stimulus-organism-response) to include both explicit and implicit (like Cognitive load) measurements. Therefore, rather than relying on a single technique, this dissertation also contributes to the growing awareness of the benefits of triangulating data to understand consumer decision-making at a more fundamental level in varied contexts.En las últimas dos décadas, los cambios pragmáticos en la tecnología han ayudado a los minoristas Business-to-Consumers a buscar vías novedosas para expandir sus negocios y entusiasmar a los consumidores con nuevas experiencias de compra. Por un lado, debido a los continuos desarrollos tecnológicos, los minoristas han aprovechado el poder de la Web 2.0 para hacer un cambio de las tiendas físicas tradicionales a las tiendas minoristas de comercio electrónico y empoderaron a los consumidores con contenido generado por el usuario (UGC). Con la llegada de la Web 3.0, los minoristas intentan establecer tiendas en un espacio tridimensional de realidad extendida mediante el uso de realidad virtual (VR), realidad aumentada (AR) o realidad mixta (MR), creando así mundos virtuales para que los consumidores interactúen y compartan usando Inteligencia Artificial (AI). Evidentemente, esto ha resultado en un cambio de los patrones de compra de los consumidores y ha hecho necesario directamente que los investigadores desarrollen aparatos para capturar y comprender la naturaleza en constante evolución del comportamiento del consumidor. Paralelamente, la investigación de mercados ha dado grandes pasos para incorporar herramientas y técnicas para observar las mediciones conscientes y subconscientes de los consumidores. Si bien las técnicas de marketing convencionales incluían autoinformes, grupos focales y datos de panel para comprender las observaciones explícitamente establecidas, la llegada de herramientas neurofisiológicas, como la electroencefalografía (EEG), la variabilidad de la frecuencia cardíaca (HRV), el seguimiento ocular (ET), etc., es posible registrar y cuantificar las observaciones subconscientes no declaradas del consumidor frente a estímulos de marketing. Estos avances muestran el curso futuro de la investigación y conllevan nuevos desafíos, como la identificación de aspectos críticos de UGC que influyen en el comportamiento de compra, la triangulación de datos neurofisiológicos con mediciones explícitas y patrones de consumo dinámicos en mercados inmersivos. Por lo tanto, el desarrollo de esta tesis participa en varios dominios (venta minorista, neuromarketing y realidad virtual) para contribuir con una nueva perspectiva para comprender el comportamiento del consumidor y, por lo tanto, superar algunos de los desafíos. El objetivo general de esta tesis es ampliar los estudios académicos sobre el comportamiento del consumidor a través de marcos teóricos y técnicas metodológicas novedosos. Desde una perspectiva conceptual, los estudios pasan a examinar el comportamiento de compras en entornos de comercio electrónico y comercio virtual. En el formato de comercio electrónico, el análisis de los atributos de las reseñas de productos en las decisiones de compra para diferentes categorías de productos amplió el marco clásico de utilización de señales (Capítulo 2). En formato de realidad virtual, se establecen diferencias críticas en el comportamiento de compra planificada y no planificada al comprar en un supermercado virtual (Capítulo 4). Desde un punto de vista metodológico, esta tesis incorpora mediciones implícitas (neurofisiológicas) junto con mediciones tradicionales explícitas (autoinforme). La utilización de herramientas neurocientíficas sirvió para dos objetivos interconectados para esta tesis: (a) actualizar los marcos de marketing clásicos para los profesionales de hoy (Capítulos 3 y 4); y (b) para evaluar el impacto de las intervenciones de marketing (publicidad y venta minorista) en las intenciones de compra de los consumidores (Capítulo 4 y, en el Apéndice 1, 2 y 3). El uso de herramientas neurocientíficas, en los Capítulos 3 y 4 (también en los Apéndices 1, 2 y 3) permitió capturar directamente la toma de decisiones subconscientes en tiempo real de los consumidores durante la etapa de compra del recorrido del consumidor. Esto permitió actualizar los modelos clásicos de comportamiento del consumidor (por ejemplo, estímulo-organismo-respuesta) para incluir mediciones tanto explícitas como implícitas (como la carga cognitiva). Por lo tanto, en lugar de depender de una única técnica, esta disertación también contribuye a la creciente conciencia de los beneficios de triangular datos para comprender la toma de decisiones del consumidor a un nivel más fundamental en contextos variados

Semantic discovery and reuse of business process patterns

Patterns currently play an important role in modern information systems (IS) development and their use has mainly been restricted to the design and implementation phases of the development lifecycle. Given the increasing significance of business modelling in IS development, patterns have the potential of providing a viable solution for promoting reusability of recurrent generalized models in the very early stages of development. As a statement of research-in-progress this paper focuses on business process patterns and proposes an initial methodological framework for the discovery and reuse of business process patterns within the IS development lifecycle. The framework borrows ideas from the domain engineering literature and proposes the use of semantics to drive both the discovery of patterns as well as their reuse

Human-in-the-Loop Cyber-Physical-Systems based on Smartphones

Tese de doutoramento em Ciências e Tecnologias da Informação, apresentada ao Departamento de Engenharia Informática da Faculdade de Ciências e Tecnologia da Universidade de CoimbraTechnological devices increasingly become smaller, more mobile, powerful and efficient. However, each time we have to hurdle through unintuitive menus, errors and incompatibilities we become stressed by our technology. As first put forward by the renowned computer scientist Mark Weiser, the ultimate form of computers may be an extension of our subconscious. The ideal computer would be capable of truly understanding people's unconscious actions and desires. Instead of humans adapting to technology and learning how to use it, it would be technology that would adapt to the disposition and uniqueness of each human being. This thesis focuses on the realm of Human-in-the-loop Cyber-Physical Systems (HiTLCPSs). HiTLCPSs infer the users’ intents, psychological states, emotions and actions, using this information to determine the system's behavior. This involves using a large variety of sensors and mobile devices to monitor and evaluate human nature. Therefore, this technology has strong ties with wireless sensor networks, robotics, machine-learning and the Internet of Things. In particular, our work focuses on the usage of smartphones within these systems. It begins by describing a framework to understand the principles and theory of HiTLCPSs. It provides some insights into current research being done on this topic, its challenges, and requirements. Another of the thesis' objectives is to present our innovative taxonomy of human roles, where we attempt to understand how a human may interact with HiTLCPSs and how to best explore this resource. This thesis also describes concrete examples of the practical usage of HiTL paradigms. As such, we included a comprehensive description of our research work and associated prototypes, where the major theoretical concepts behind HiTLCPS were applied and evaluated to specific scenarios. Finally, we discuss our personal view on the future and evolution of these systems.A tecnologia tem vindo a tornar-se cada vez mais pequena, móvel, poderosa e eficiente. No entanto, lidar com menus pouco intuitivos, erros, e incompatibilidades, causa frustração aos seus utilizadores. Segundo o reconhecido cientista Mark Weiser, os computadores do futuro poderão vir a existir como se fossem uma extensão do nosso subconsciente. O computador ideal seria capaz de entender, em toda a sua plenitude, as ações e os desejos inconscientes dos seres humanos. Em vez de serem os humanos a adaptarem-se à tecnologia e a aprender a usá-la, seria a tecnologia a aprender a adaptar-se à disposição e individualidade de cada ser humano. Esta tese foca-se na área dos Human-in-the-loop Cyber-Physical Systems (HiTLCPSs). Os HiTLCPSs inferem as intenções, estados psicológicos, emoções e ações dos seus utilizadores, usando esta informação para determinar o comportamento do sistema ciber-físico. Isto envolve a utilização de uma grande variedade de sensores e dispositivos móveis que monitorizam e avaliam a natureza humana. Assim sendo, esta tecnologia tem fortes ligações com redes de sensores sem fios, robótica, algoritmos de aprendizagem de máquina e a Internet das Coisas. Em particular, o nosso trabalho focou-se na utilização de smartphones dentro destes sistemas. Começamos por descrever uma estrutura para compreender os princípios e teoria associados aos HiTLCPSs. Esta análise permitiu-nos adquirir alguma clareza sobre a investigação a ser feita sobre este tópico, e sobre os seus desafios e requisitos. Outro dos objetivos desta tese é o de apresentar a nossa inovadora taxonomia sobre os papeis do ser humano nos HiTLCPSs, onde tentamos perceber as possíveis interações do ser humano com estes sistemas e as melhores formas de explorar este recurso. Esta tese também descreve exemplos concretos da utilização prática dos paradigmas HiTL. Desta forma, incluímos uma descrição do nosso trabalho experimental e dos protótipos que lhe estão associados, onde os conceitos teóricos dos HiTLCPSs foram aplicados e avaliados em diversos casos de estudo. Por fim, apresentamos a nossa perspetiva pessoal sobre o futuro e evolução destes sistemas.Fundação Luso-Americana para o DesenvolvimentoFP7-ICT-2007-2 GINSENG projectiCIS project (CENTRO-07-ST24-FEDER-002003)SOCIALITE project (PTDC/EEI-SCR/2072/2014

Ambulatory monitoring of electrodermal and cardiac functioning in anxiety and worry

Emotions are an integral part of the human experience and their interpretation can provide valuable but also misleading clues about oneself and other people’s state of mind. Negative emotional states can be perceived as uncomfortable and – when experienced chronically – can develop into anxiety and mood disorders. The more pervasive these disorders the more severely they affect and disable a person’s everyday functioning and often their sleep as well. According to Lang and colleagues (1998), emotions may be expressed verbally, behaviorally, and physiologically, i.e., emotions can be reported, observed, and objectively measured. Each measurement approach provides important, unique, and often conflicting information that can be used in the assessment and treatment evaluation of psychological disorders affecting the emotions. Autonomic measures have been used to indicate the physiological components of emotions, such as those along the worry-anxiety-fear-panic spectrum. Worry has been shown to suppress cardiac responses to imaginal feared material (see Borkovec, Alcaine, & Behar, 2004) and reduce autonomic variability (Hoehn-Saric, McLeod, Funderburk, & Kowalski, 2004; Hoehn-Saric, McLeod, & Zimmerli, 1989). Results for panic and anticipatory anxiety are less conclusive but theoretically these states should go along with increased autonomic arousal. Abnormal autonomic arousal might also be present during sleep as both panic disorder and worrying have been associated with sleeping difficulties. However, most empirical research has been confined to the laboratory where high internal validity is achieved at the cost of poor ecological validity. Thus, the purpose of this doctoral dissertation is to extend and validate laboratory findings on worry, anticipatory anxiety, and panic using ambulatory monitoring. Twenty-four hour monitoring not only can give valuable insights into a person’s daytime emotional experience but also allows observing how these emotions might affect their sleep in their natural environment. In the following chapter, the reader will be introduced to a conceptual framework that ties together worry, anxiety, fear, and panic, and related anxiety disorders (section 2.1), to autonomic arousal and electrodermal and cardiac arousal in particular (section 2.2), to sleep and its relation to autonomic arousal and anxiety disorders (section 2.3), and to ambulatory monitoring (section 2.4). After illustrating the aims of this thesis (chapter 3), chapters 4 to 6 present the results of three empirical studies conducted as part of this doctoral research. The first study deals solely with electrodermal monitoring and how it is affected by confounding variables in an ambulatory context (chapter 4). The next study then seeks to investigate the relationship between electrodermal arousal and anticipatory anxiety and panic in a sample of panic disorder patients and healthy controls. The last study focuses primarily on the effect of trait and state worry on subjective and objective sleep and electrodermal and cardiac arousal in a group of high and low worriers. Chapters 7 to 9 summarize and integrate the findings from these three empirical studies, discuss methodological limitations, and provide an outlook into future research