Prerequisites for Affective Signal Processing (ASP) - Part III

This is the third part in a series on prerequisites for affective signal processing (ASP). So far, six prerequisites were identified: validation (e.g., mapping of constructs on signals), triangulation, a physiology-driven approach, and contributions of the signal processing community (van den Broek et al., 2009) and identification of users and theoretical specification (van den Broek et al., 2010). Here, two additional prerequisites are identified: integration of biosignals, and physical characteristics

Neurophysiological Assessment of Affective Experience

In the field of Affective Computing the affective experience (AX) of the user during the interaction with computers is of great interest. The automatic recognition of the affective state, or emotion, of the user is one of the big challenges. In this proposal I focus on the affect recognition via physiological and neurophysiological signals. Long‐standing evidence from psychophysiological research and more recently from research in affective neuroscience suggests that both, body and brain physiology, are able to indicate the current affective state of a subject. However, regarding the classification of AX several questions are still unanswered. The principal possibility of AX classification was repeatedly shown, but its generalisation over different task contexts, elicitating stimuli modalities, subjects or time is seldom addressed. In this proposal I will discuss a possible agenda for the further exploration of physiological and neurophysiological correlates of AX over different elicitation modalities and task contexts

Designing an Adaptive Video Game for Children with Attention Deficit Hyperactivity Disorder: Learning Proportional Reasoning through Play

Attention Deficit Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder marked by symptoms of inattention and/or hyperactivity and impulsivity. Many children with ADHD suffer from poor reasoning and decision-making that impedes their readiness for learning. Proportional reasoning is an important skill in everyday decision making, and it is essential for children’s academic success, which is linked to higher self-esteem, lower levels of anxiety and depression, better family and peer relationship. Therefore, the goal of the thesis was to design an educational video game for children with ADHD that will teach them proportional reasoning through enjoyable play. The second goal of the thesis was to verify whether the designed game, which uses a low-cost mobile electroencephalogram (EEG)-based device to adapt the in-game difficulty based on players’ attention, makes the learning of proportional reasoning faster through faster progression and helps players to maintain their attention span for a longer period of time. To answer my research questions, I developed an in-game difficulty level model for the experimental game that consists of 4 designed conditions, which are a combination of the game trial win or loss with focused or not focused attention states. The faster progression is granted after at least 2 wins in the first 3 trials of the same difficulty level. To evaluate the game design and my adaptive difficulty algorithm, I performed a formal experiment. The participants (n=20), randomly assigned to two groups (ADAPTIVE and CONTROL), were asked to play the designed videos game while wearing a mobile EEG-based headset MindWave by Neurosky. During the study, I collected players’ brain activity data, such as attention and meditation states, and the users’ evaluations of the game experience and the difficulty of each level. I also conducted a qualitative analysis of players’ in-game behavior. The data analysis revealed a strong negative correlation between the total number of games played and the average time spent on the task in the CONTROL group, (r(10) = -0.72, p = 0.02), suggesting the presence of a learning effect that supports the ultimate goal to design a game that teaches players proportional reasoning. Although in the ADAPTIVE group we did not observe statistical difference between these parameters, the ADAPTIVE game design showed a potential to be more efficient in teaching proportional reasoning since the players were able to finish the gameplay in a shorter period of time having attention, meditation and flow states no different from the CONTROL group. The quantitative study findings in combination with the qualitative analysis of players’ in-game behavior pointed to the necessity of the minor level design modifications that might improve the quality of game experience. Players’ improved game satisfaction in combination with the already successfully implemented learning component, based on the presence of learning effect in the groups, will make the learning process an enjoyable experience

Affective state elicitation using pictoral and audiovisual stimuli: A comparative study

V experimentálním přístupu k emocím se pro vyvolávání afektivních stavů nejčastěji používají emocionálně nabité obrázky a filmové klipy. Není jisté, která z těchto dvou metod má větší účinek. V teoretické části jsem popsal konceptuální ukotvení těchto metod, jejich teoretické srovnání, a metody jim příbuzné. V empirické části jsem u vzorku 124 studentů přímo porovnal prožitkové, behaviorální a fyziologické reakce vyvolané obrázky a filmy. Pozitivní film vyvolal silnější reakce než pozitivní obrázky. Negativní film a negativní obrázky vyvolaly podobně silné, ale mírně odlišné reakce. Rozdíl mezi nimi lze vysvětlit rozdílnými temporálními charakteristikami obou metod. Jde teprve o druhý výzkum na toto téma, a první, který využívá měření reakcí na více než jedné úrovni. Powered by TCPDF (www.tcpdf.org)Both pictures and films are widely used elicitors in experimental approach to emotions. However, there is no consensus regarding their relative impact. I describe conceptual underpinning of these methods, their theoretical comparison, and related methods. In a study of 124 students I assess and compare the impact of pictures and films on experiential, behavioural and physiological reactions. There were stronger reactions in response to positive film than to positive pictures. Negative film and pictures were similar in strength yet different in kind of reactions they yielded. The difference can be explained by different temporal characteristics of both methods. Present study is the second study dealing with this topic, and the only study that engaged multiple reaction levels. Powered by TCPDF (www.tcpdf.org)Katedra psychologieDepartment of PsychologyFilozofická fakultaFaculty of Art

Gestalt Biometrics and their Applications; Instrumentation, objectivity and poetics

This thesis is about the relationship between human bodies and instrumental technologies that can be use to measure them. It adopts the position that instruments are technological structures that evoke and manifest particular phenomena of embodied life. However, through their history of association and use in the sciences and scientific medicine, instruments tend to be attached to a particular ontology, that of mechanical objectivity. Embarking from research into the artistic uses of physiological sensor technology in creative practices such as performance and installation art, this thesis asks whether it is possible to use instruments in a way that departs from their association with scientific objectivity. Drawing on philosophers who have developed an understanding of the relationship of instrumental technologies and human bodies as co-constructive, it explores how this model of con-construction might be understood to offer an alternative ontology for understanding the use of instruments in practices outside of science and scientific medicine. The project is therefore suggestive of degrees of freedom and flexibility that are open to exploitation by creative practices in the realm of instrumentation as an alternative to orthodox rationalisations of the value of scientific equipment as authentic, revealing and objective. The major contribution of the thesis is that transfers and synthesises arguments and evidence from the history and philosophy of sciences that serve to demonstrate how the instrumental measurement of human bodies can be considered to be a form of creative practice. It assembles a position based on the work of thinkers from a number of disciplines, particularly philosophy of science, technology, and the medical humanities. These offer examples of ontological frameworks within which the difference between the realm of the instrumental, material, biological, and the objective, and the phenomenal, meaningful and subjective, might be collapsed. Doing this, the thesis sheds light on how physical devices might enter into the interplay of making, mattering and objectifying the immaterial, a realm that it might be considered the role of artists to manifest. Drawing on contemporary, and secondary, accounts of the development of empirical testing in the medical sciences, the thesis agues for the recovery of a romantic account of human physiology, in which the imagination and meaning are active and embodied. It therefore offers to link the bodily and the instrumental through an extended-materialist account in which the physiological, rather than the psychological, is central. Developing a response to constructionist models of the body and instrumentation, the thesis concludes that a model of the poetic may be adopted as a method for understanding the opportunities and imperatives inherent in the avoidance of deterministic approaches to biosignalling technologies. In doing this, the thesis contributes particularly to the creative arts and technology research practices concerned with the use of body sensor technologies in humanistic applications. It complements the existing works by artists in this area that make use of instruments by assembling a number of theoretical readings and interpretations of how instruments work – among them the thermometer, lie detector, and automatograph – which illustrate the argument that that is possible to operate from a theoretical position within which instruments are both material, performative and symbolic.Engineering and Physical Sciences Research Counci

Un outil d’évaluation neurocognitive des interactions humain-machine

De plus en plus de recherches sur les Interactions Humain-Machine (IHM) tentent d’effectuer des analyses fines de l’interaction afin de faire ressortir ce qui influence les comportements des utilisateurs. Tant au niveau de l’évaluation de la performance que de l’expérience des utilisateurs, on note qu’une attention particulière est maintenant portée aux réactions émotionnelles et cognitives lors de l’interaction. Les approches qualitatives standards sont limitées, car elles se fondent sur l’observation et des entrevues après l’interaction, limitant ainsi la précision du diagnostic. L’expérience utilisateur et les réactions émotionnelles étant de nature hautement dynamique et contextualisée, les approches d’évaluation doivent l’être de même afin de permettre un diagnostic précis de l’interaction. Cette thèse présente une approche d’évaluation quantitative et dynamique qui permet de contextualiser les réactions des utilisateurs afin d’en identifier les antécédents dans l’interaction avec un système. Pour ce faire, ce travail s’articule autour de trois axes. 1) La reconnaissance automatique des buts et de la structure de tâches de l’utilisateur, à l’aide de mesures oculométriques et d’activité dans l’environnement par apprentissage machine. 2) L’inférence de construits psychologiques (activation, valence émotionnelle et charge cognitive) via l’analyse des signaux physiologiques. 3) Le diagnostic de l‘interaction reposant sur le couplage dynamique des deux précédentes opérations. Les idées et le développement de notre approche sont illustrés par leur application dans deux contextes expérimentaux : le commerce électronique et l’apprentissage par simulation. Nous présentons aussi l’outil informatique complet qui a été implémenté afin de permettre à des professionnels en évaluation (ex. : ergonomes, concepteurs de jeux, formateurs) d’utiliser l’approche proposée pour l’évaluation d’IHM. Celui-ci est conçu de manière à faciliter la triangulation des appareils de mesure impliqués dans ce travail et à s’intégrer aux méthodes classiques d’évaluation de l’interaction (ex. : questionnaires et codage des observations).More and more researches on Human-Computer Interactions (HCI) are trying to perform detailed analyses of interaction to determine its influence on users’ behaviours. A particular emphasis is now put on emotional reactions during the interaction, whether it’s from the perspective of user experience evaluation or user performance. Standard qualitative approaches are limited because they are based on observations and interviews after the interaction, therefore limiting the precision of the diagnosis. User experience and emotional reactions being, by nature, highly dynamic and contextualized, evaluation approaches should be the same to accurately diagnose the quality of interaction. This thesis presents an evaluation approach, both dynamic and quantitative, which allows contextualising users’ emotional reactions to help identify their causes during the interaction with a system. To this end, our work focuses on three main axes: 1) automatic task recognition using machine learning modeling of eye tracking and interaction data; 2) automatic inference of psychological constructs (emotional activation, emotional valence, and cognitive load) through physiological signals analysis; and 3) diagnosis of users’ reactions during interaction based on the coupling of the two previous operations. The ideas and development of our approach are illustrated using two experimental contexts: e-commerce and simulation-based training. We also present the tool we implemented in order to allow HCI professionals (e.g.: user experience expert, training supervisor, or game designer) to use our evaluation approach to assess interaction. This tool is designed to facilitate the triangulation of measuring instruments and the integration with more classical Human-Computer Interaction methods (ex.: surveys and observation coding)