EmojiGrid: A 2D Pictorial Scale for the Assessment of Food Elicited Emotions

Research on food experience is typically challenged by the way questions are worded. We therefore developed the EmojiGrid: a graphical (language-independent) intuitive self-report tool to measure food-related valence and arousal. In a first experiment participants rated the valence and the arousing quality of 60 food images, using either the EmojiGrid or two independent visual analog scales (VAS). The valence ratings obtained with both tools strongly agree. However, the arousal ratings only agree for pleasant food items, but not for unpleasant ones. Furthermore, the results obtained with the EmojiGrid show the typical universal U-shaped relation between the mean valence and arousal that is commonly observed for a wide range of (visual, auditory, tactile, olfactory) affective stimuli, while the VAS tool yields a positive linear association between valence and arousal. We hypothesized that this disagreement reflects a lack of proper understanding of the arousal concept in the VAS condition. In a second experiment we attempted to clarify the arousal concept by asking participants to rate the valence and intensity of the taste associated with the perceived food items. After this adjustment the VAS and EmojiGrid yielded similar valence and arousal ratings (both showing the universal U-shaped relation between the valence and arousal). A comparison with the results from the first experiment showed that VAS arousal ratings strongly depended on the actual wording used, while EmojiGrid ratings were not affected by the framing of the associated question. This suggests that the EmojiGrid is largely self-explaining and intuitive. To test this hypothesis, we performed a third experiment in which participants rated food images using the EmojiGrid without an associated question, and we compared the results to those of the first two experiments. The EmojiGrid ratings obtained in all three experiments closely agree. We conclude that the EmojiGrid appears to be a valid and intuitive affective self-report tool that does not rely on written instructions and that can efficiently be used to measure food-related emotions

Мобільний додаток для покращення психічного стану користувача

В бакалаврській роботі розглянуто і проаналізовано алгоритм для визначення емоційних станів людини. Проведено огляд методів роботи з емоціями людини та їх аналізу. Для реалізації системи було обрано згорткові нейронні мережі та було сформовано план експерименту для збору даних для навчання. Було реалізовано модулі системи, що дозволяють визначити емоційний стан людини. Було проаналізовано якість роботи побудованої системи. Модулі системи реалізовані з використанням мови програмування Java та фреймворку Keras, побудований програмний продукт являє собою мобільний додаток для операційної системи Android. Результати свідчать про доцільність проведення подальшої роботи з покращення роботи системи для отримання точності роботи достатньої для практичного застосування.The algorithm for determining human emotional states is considered and analyzed in the work. A review of methods of working with human emotions and their analysis is presented. To implement the system, convolutional neural networks were selected and an experimental plan was formed to collect data for training. Modules of the system were implemented to determine a person’s emotional state. The quality of the constructed system was analyzed. The system modules are implemented using the Java programming language and the Keras framework, the built software product is a mobile application for the Android operating system. The results indicate the feasibility of further work to improve the system to obtain the accuracy of sufficient work for practical application

Система аналізу біомедичних сігналів для контролю стану водіїв

Магістерська дисертація: 120 с., 22 рис., 26 табл., 2 додатки, 57 джерел. Тема роботи – «Система аналізу біомедичних сигналів для контролю стану водіїв». В роботі розглянуто і проаналізовано алгоритм для визначення емоційних станів людини за ЕКГ. Проведено огляд методів роботи з емоціями людини та аналізу її ЕКГ. Для реалізації системи було обрано згорткові нейронні мережі та було сформовано план експерименту для збору даних для навчання. Було реалізовано модулі системи, що дозволяють визначити емоційний стан людини за ЕКГ. Було проаналізовано якість роботи побудованої системи. Модулі системи реалізовані з використанням мов програмування Python та Java та фреймворку Keras. Результати світчать про доцільність проведення подальшої роботи з покращення роботи системи для отримання точності роботи достатньої для практичного застосування. Мета дослідження – створення системи розпізнавання емоцій за біомедичним сигналом. Об’єкт дослідження – моделі емоційного стану людини. Предмет дослідження – методи виявлення емоційного стану людини на основі біосигналів. Наукова новизна – запропоновано алгоритми виявлення емоційних станів людини на основі біомедичного сигналу та згорткових нейронних мереж. Практична цінність – в роботі розроблено додаток, який дозволяє отримувати біомедичний сигнал в режімі реального часу та визначати емоційний стан з точністю до 95%.Master’s thesis: 120 p., 22 fig., 26 tabl., 2 appendixes, 57 sources. The topic of the research: “System for analyzing biomedical signals for determining the emotional state”. In this work an algorithm for determining the emotional states of a person by an electrocardiogram is considered and analyzed. The review of methods of work with human emotions and analysis of electrocardiogram is carried out. To implement the system, convolutional neural networks were selected and the experiment design was created to collect the dataset for training. Modules of the system, which allow estimation of the emotional states of a person by an electrocardiogram, were realized. The quality of the built-in system was analyzed. System modules are implemented using Python and Java programming languages and the Keras framework. The results indicate the expediency of further work to improve the system to obtain the accuracy of work sufficient for practical application. The purpose of the study is to create a system for recognizing emotions by biomedical signal. The object of research - the model of the human emotional state. Subject of research - methods of identifying the emotional state of a person on the basis of biosignals. Scientific novelty - algorithms for identifying emotional state of a person from biomedical signal and CNNs are proposed. Practical value - in the work developed an application that allows you to get a biomedical signal in realtime mode and determine the emotional state with an accuracy of 95%

Affective state recognition in Virtual Reality from electromyography and photoplethysmography using head-mounted wearable sensors.

The three core components of Affective Computing (AC) are emotion expression recognition, emotion processing, and emotional feedback. Affective states are typically characterized in a two-dimensional space consisting of arousal, i.e., the intensity of the emotion felt; and valence, i.e., the degree to which the current emotion is pleasant or unpleasant. These fundamental properties of emotion can not only be measured using subjective ratings from users, but also with the help of physiological and behavioural measures, which potentially provide an objective evaluation across users. Multiple combinations of measures are utilised in AC for a range of applications, including education, healthcare, marketing, and entertainment. As the uses of immersive Virtual Reality (VR) technologies are growing, there is a rapidly increasing need for robust affect recognition in VR settings. However, the integration of affect detection methodologies with VR remains an unmet challenge due to constraints posed by the current VR technologies, such as Head Mounted Displays. This EngD project is designed to overcome some of the challenges by effectively integrating valence and arousal recognition methods in VR technologies and by testing their reliability in seated and room-scale full immersive VR conditions. The aim of this EngD research project is to identify how affective states are elicited in VR and how they can be efficiently measured, without constraining the movement and decreasing the sense of presence in the virtual world. Through a three-years long collaboration with Emteq labs Ltd, a wearable technology company, we assisted in the development of a novel multimodal affect detection system, specifically tailored towards the requirements of VR. This thesis will describe the architecture of the system, the research studies that enabled this development, and the future challenges. The studies conducted, validated the reliability of our proposed system, including the VR stimuli design, data measures and processing pipeline. This work could inform future studies in the field of AC in VR and assist in the development of novel applications and healthcare interventions