Emotion Recognition based on Heart Rate and Skin Conductance

Information on a customer’s emotional states concerning a product or an advertisement is a very important aspect of marketing research. Most studies aimed at identifying emotions through speech or facial expressions. However, these two vary greatly with people’s talking habits, which cause the data lacking continuous availability. Furthermore, bio-signal data is also required in order to fully assess a user’s emotional state in some cases. We focused on recognising the six basic primary emotions proposed by Ekman using biofeedback sensors, which measure heart rate and skin conductance. Participants were shown a series of 12 video-based stimuli that have been validated by a subjective rating protocol. Experiment results showed that the collected signals allow us to identify user\u27s emotional state with a good ratio. In addition, a partial correlation between objective and subjective data has been observe

Preliminary Test of Affective Virtual Reality Scenes with Head Mount Display for Emotion Elicitation Experiment

Emotion elicitation experiments are conducted to collect biological signals from a subject who is in a state　of emotion. The recorded signals are used as training/test dataset for constructing an emotion recognition system by　means of machine learning. In conventional emotion elicitation experiments, affective images or videos were provided　for a subject to draw out an emotion from them. However, the authors have concerns about the effectiveness. To surely　evoke a specific emotion from subjects, we have produced several Virtual Reality (VR) scenes and provided the　subjects with the scenes through a Head Mount Display (HMD) in emotion elicitation experiments. Usability and　effectiveness of the VR scenes with the HMD for emotion elicitation were experimentally verified. It was confirmed　that experience of the VR scenes with the HMD was effective in evoking emotions, but we have to improve how　subjects learn a way of playing VR scenes and provide measures against VR sickness at any cost. Moreover, Support　Vector Machine classifiers as an emotion recognition system were constructed using the biological signals measured　from the subjects in the emotion elicitation experiments.2017 17th International Conference on Control, Automation and Systems (ICCAS 2017), Oct. 18-21, 2017, Ramada Plaza, Jeju, Kore

Fusi Algoritma K-Means dan CNN untuk Klasifikasi Emosi pada Anak

Emosi adalah perasaan yang diarahkan pada seseorang ataupun sesuatu yang bisa menyebabkan sesorang bertindak atau mengekspresikan diri dan dapat dipicu secara internal ataupun eksternal. Ekspresi wajah merupakan salah satu cara termudah untuk mengetahui emosi seseorang, namun terkadang seseorang dapat mengontrol dan memanipulasi ekspresi wajah mereka sehingga tidak sesuai dengan apa yang dialami. Oleh karena itu, penelitian ini mengembangkan sistem yang dapat mengidentifikasi emosi anak tidak hanya berdasarkan wajah tetapi juga berdasarkan perubahan kondisi tubuhnya. Penelitian ini menggunakan metode klasifikasi Convolutional Neural Network (CNN) dan juga metode clusterisasi K-Means. Penggunaan 2 metode pada penelitian ini berfungsi untuk memperkuat akurasi sistem. Metode K-Means digunakan untuk mengidentifikasi emosi berdasarkan detak jantung dan konduktivitas kulit sedangkan Metode CNN digunakan untuk mengidentifikasi emosi berdasarkan ekspresi wajah. Hasil yang diperoleh dari kedua metode tersebut akan diproses menggunakan metode fusi yang aturannya disesuaikan berdasarkan hasil pengamatan dan pengukuran, sehingga dapat diprediksi emosi pada anak berdasarkan parameter detak jantung, ekspresi wajah, dan konduktivitas kulit. Anak dengan umur 6 hingga 12 tahun digunakan sebagai subjek pada penelitian ini. Dari penelitian ini berhasil didapatkan hasil prediksi emosi anak dengan akurasi keberhasilan sebesar 80%

Social distance through music in EFL students

[EN] Music is constantly around us and, if in English, it fosters social distance in EFL students, consequently increasing the level of affinity with the language, however, can social distance help with other personal aspects? Music lyrics provide different perspectives that can become enriching on an academic level, as music brings cultural aspects of the country of origin and directly affects emotions, therefore song lyrics may become a very powerful instrument when combining social distance and the level of English of students in order to improve academic and future professional performance, among others.The aim of this study was to measure the impact of social distance on the improvement of academic performance, emotional states and future prospects in EFL students. In order to measure those variables, a questionnaire, based on the Likert scale, conducted amongst 82 students from different educational centres. Resulting data analysed with the use of the structural equation modeling (SEM-PLS) revealed the relevance of social distance through music in order to improve learning processes, to feel emotions and to consolidate hopes for future prospects.Sánchez González, MG. (2021). Social distance through music in EFL students. Multidisciplinary Journal for Education, Social and Technological Sciences. 8(2):42-59. https://doi.org/10.4995/muse.2021.15014OJS425982Abbot, M. (2002). Using music to promote L2 learning among adult learners. Tesol journal, 11(1).Alcaraz Varó, E. (2000). El inglés profesional y académico. Alianza Editorial.Aragão, R. (2011). Beliefs and emotions in foreign language learning. Fuel and Energy Abstracts,39(3), 302-313. https://doi.org/10.1016/j.system.2011.07.003Bogt, T., Mulder, J., Raaijmakers, Q.A.W., & Gabhainn, S.N. (2011). Moved by music: A typology of music listeners. Psychology of Music, 39(2), pp. 147-163. https://doi.org/10.1177/0305735610370223Chin, W.W. (1998). The Partial Least Squares Approach to Structural Equation Modeling, In Modern Methods for Business Research, Marcoulides GA [ed]. Lawrence Erlbaum Associates: Mahwah N. J., 295-336.Cooper, D.R., & Schindler, P.S. (2006). Business Research Methods (9th edition). McGraw-Hill.Creswell, J.W., & Clark, V.L.P. (2017). Designing and conducting mixed methods research. Sage Publications.Do, S.L., & Schallert, D.L. (2004). Emotions and classroom talk: Toward a model of the role of affect in students' experiences of classroom ciscussions. Journal of Educational Psychology, 96(4), pp. 619-634. https://doi.org/10.1037/0022-0663.96.4.619Fedorenko, E. (2014). The role of domain-general cognitive control in language comprehension. Frontiers in Psychology, 5, p. 335. https://doi.org/10.3389/fpsyg.2014.00335Flowerdew, J., & Peacock, M. (2001). Research perspectives on English for academic purposes. Cambridge University Press. https://doi.org/10.1017/CBO9781139524766Fonseca-Mora, M.C. (2000). Foreign language acquisition and melody singing. Elt Journal, 54(2), 146-152. https://doi.org/10.1093/elt/54.2.146Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. https://doi.org/10.2307/3151312Fuller, A. (2015). Vocational education. International encyclopedia of the social & behavioral sciences (2nd Ed), 232-238. Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.92091-9Gardner, R.C., & Lambert, W.E. (1972). Attitudes and motivation in second language learning. Newbury House Publishers.González Davies, M., & Celaya Villanueva, M.L. (1992). New teachers in a new education system: A Guide book for the Reforma. PPU.Graddol, D. (1997). The future of English? British Council.Henseler, J., Ringle, C.M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115-135. https://doi.org/10.1007/s11747-014-0403-8Izard, C.E. (2002). Translating emotion theory and research into preventive interventions. Psychological Bulletin, 128(5), 796-824. https://doi.org/10.1037/0033-2909.128.5.796Kachru, B.B., & Nelson, C.L. (2001). World Englishes. In A. Burns & C. Coffin (Eds.). Analysing English in a global context, 9-25. Routledge.Krashen, S.D. (2003). Explorations in language acquisition and use. Heinemann.Levitin, D.J. (2006). This is your brain on music: The science of a human obsession. Dutton/Penguin Books.Macdonald, R., Kreutz, G., & Mitchell, L. (2012). What is music, health, and wellbeing and why is it important? Music, health, and wellbeing. Oxford Scholarship online. https://doi.org/10.1093/acprof:oso/9780199586974.003.0001McLean, A.C. (1980). Destroying the teacher: the need for a learner-centred teaching. E.T.Forum, 28:3, 16-19.Ménard, M., Richard, P., Hamdi, H., Daucé, B., & Yamaguchi, T. (2015). Emotion recognition based on heart rate and skin conductance. In Proceedings of the 2nd international conference on physiological computing systems, vol. 1: PhyCS, 26-32. https://doi.org/10.5220/0005241100260032Nicolás, G., & Azorín, C. (2013). Música y valores. Una relación educativa ineludible. Música y educación: Revista trimestral de pedagogía musical, ISSN 0214-4786, Año nº 26, Nº 93, 16-25.Niittylahti, S., Annala, J., & Mäkinen, M. (2019). Student engagement at the beginning of vocational studies. Nordic journal of vocational education and training, Vol.9, n.1, 21-42. https://doi.org/10.3384/njvet.2242-458X.199121North, A.C., & Hargreaves, D.J. (1997). Music and consumer behaviour. In D.J. Hargreaves & A.C. North (Eds.), The social psychology of music, 268-289. Oxford University Press.Pekrun, R., & Perry, R.P. (2014). Control-value theory of achievement emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education, 120-141. Routledge/Taylor & Francis Group. https://doi.org/10.4324/9780203148211Punset, E. (2012). Una Mochila para el universo. Ediciones Destino.Richards, J.C., Platt, J., & Platt, H. (1992). Dictionary of language teaching and applied linguistics. Longman.Ringle, C.M., Wende, S., & Becker, J.M. (2015). SmartPLS 3. SmartPLS GmbH, Boenningstedt. http://www.smartpls.comSam, D.L, & Berry, J.W. (2010). Acculturation: When individuals and groups of different cultural backgrounds meet. Perspectives on Psychological Science, 5(4), 472-481. https://doi.org/10.1177/1745691610373075Sánchez González, M.G. (2020). Adolescents and values portrayed in music nowadays. Multidisciplinary Journal for Education Social and Technological Sciences, v. 7, 2, 45-60. https://doi.org/10.4995/muse.2020.14001Schumann, J. (1994). Where is cognition? Emotion and cognition in second language acquisition. In Studies in Second Language Acquisition, vol. 16, Issue 2, 231-242. Cambridge University Press. https://doi.org/10.1017/S0272263100012894Shoaib, A., & Dörnyei, Z. (2004). Affect in lifelong learning: Exploring L2 motivation as a dynamic process. In P. Benson, & D. Nunan (Eds.), Learners' stories, 23-41. Cambridge University Press.Sloboda, J.A., & Juslin, P.N. (2001). Handbook of music and emotion: Theory, research, applications, 605-642. Oxford University Press.Tarrant, M, North A.C., & Hargreaves, D.J. (2000). English and American adolescents' reasons for listening to music. Psychology of Music, 28(2), 166-173. https://doi.org/10.1177/030573560028200

Emotion Recognition based on Facial Expression Identification using Deep Learning Algorithm for Automation Music Healing Application

Daily activities, particularly in the workplace or other societal settings, can cause stress and pressure for those who must manage them. This stress can lead to decreased performance and achievement in daily tasks, and in more severe cases, individuals may seek consultation with a psychiatrist to address their stress and pressure. This work presents a system for emotion identification based on face expression recognition system. The system processes the face expression image using Convolution Neural Network (CNN). The face expression image is extracted and modeled based on CNN method. The identification result sent to database which transferred to the Android application played the song based on emotion identification. The system is installed on an Android cell phone, making it flexible and portable. The system has achieved 70% and 80% accuracy in emotion detection during training and testing, respectively

What about if buildings respond to my mood?

This work analyzes the possibilities of interaction between the built environment and its users, focused on the responsiveness of the first to the emotions of the latter. Transforming the built environment according to the mood, feelings, and emotions of users, moment by moment, is discussed and analyzed. The main goal of this research is to define a responsive model by which the built environment can respond in a personalized way to the users’ emotions. For such, computational technical issues, building construction elements and users’ interaction are identified and analyzed. Case studies where occurs an interaction between the physical space and users are presented. We define a model for an architecture that is responsive to the user’s emotions assuming the individual at one end and the space at the other. The interaction between both ends takes place according to intermediate steps: the collection of data, the recognition of emotion, and the execution of the action that responds to the detected emotion. As this work focuses on an innovative and disruptive aspect of the built environment, the recognition of the new difficulties and related ethical issues are discussed.info:eu-repo/semantics/acceptedVersio

Physiological arousal quantifying perception of safe and unsafe virtual environments by older and younger adults

Physiological arousal has been increasingly applied to monitor exploration (or navigation) of a virtual environment (VE), especially when the VE is designed to evoke an anxiety-related response. The present work aims to evaluate human physiological reactions to safe and unsafe VEs. We compared the effect of the presence of handrails in the VE in two different samples, young and older adults, through self-reports and physiological data: Electrodermal activation (EDA) and electrocardiogram (ECG) sensors. After navigation, self-report questionnaires were administered. We found that the VEs evoked a clearly differentiated perception of safety and unsafety demonstrated through self-reports, with older adults being more discriminative in their responses and reporting a higher sense of presence. In terms of physiological data, the effect of handrails did not provoke significant differences in arousal. Safety was better operationalized by discriminating neutral/non-neutral spaces, where the reaction of older adults was more pronounced than young adults. Results serve as a basis for orienting future experiments in the line of VE and applied physiology usage in the architectural spaces design process. This specific work also provided a basis for the development of applications that integrate virtual reality and applied biofeedback, tapping into mobility and ageing.info:eu-repo/semantics/publishedVersio

Modular Audio Platform for Youth Engagement in a Museum Context

The purpose of this thesis is to support museums and other cultural institutes in their mission to attract young visitors by offering engaging experiences. The main goals of the the-sis were to develop a modular and easy-to-use audio story-sharing and audio-augmented reality platform, and evaluate the usefulness of the platform by measuring the level of engagement of participating youth in a workshop context. Design-science research methodology was used for audio platform component development, and mixed-methods were used to study the utility of platform components as case studies. At a more detailed level this means that the expandable and modular platform was developed incrementally one component at a time. When developing Audio Digital Asset Management, action research was used. For the Soundscape Mixer development, combined action research was used until the software was in the α phase after which a separate evaluation method was used in the β phase. For the Audiostory Sharing development de-sign-science research with separate building and evaluation methods was used. After implementation and testing the audio platform also from the usability angle, we moved on to engagement research. Workshops were organized in order to demonstrate the usage of the audio platform. During the workshops engagement was researched using mixed method, namely quantitative self-report questionnaires and qualitative methods in the form of observations. We have succeeded in developing a modular and versatile audio platform. All of the hardware is commonly used including Android phones. Software-wise the backend system is based on open source components. As the backend system provides relevant APIs, new mobile applications can be developed by third parties. In parallel, a concept was also developed, which helps to reach the young target audience and helps to measure the level of engagement. For this purpose, the student engagement structure has been applied in order to find out the level of engagement in workshops where the audio platform is a vital part. As a final summary, the results are promising. There is a general-purpose audio platform, which is modular, expandable and affordable for cultural institutions, and there is a concept to reach young people and a measurement instrument to measure the level of engagement in an audio-related workshop context.Väitöskirjan tavoitteena oli kehittää modulaarinen ja helppo käyttöinen tietojärjestelmä, mikä tukee äänitarinoiden jakamista sekä äänimaisemien rakentelua ja kokemista. Toisena tavoitteena oli arvioida, miten koukuttavia kokemuksia nuorille syntyy työpajoissa, missä tätä tietojärjestelmää hyödynnettiin. Väitöskirjan tarkoitus on tukea museoita ja muita kulttuurilaitoksia houkuttelemaan nuoria kävijöitä tarjoamalla nuorille mahdollisuus kokea kulttuurilaitoksen tarjonta osallistavalla tavalla. Olemme onnistuneet kehittämään modulaarisen ja laajennettavan ääniin keskittyvän tietojärjestelmän. Palvelinohjelmisto perustuu avoimeen lähdekoodiin ja nuorille käyttäjille tarkoitetut mobiilisovellukset on kehitetty Android puhelimille. Järjestelmä tarjoaa avoimet rajapinnat, mikä mahdollistaa myös kolmansien osapuolien uusien mobiilisovellusten kehittämisen kulttuurilaitosten tarpeisiin. Tietojärjestelmän rinnalla kehitimme työpajakonseptit. Työpajoilla on pedagogiset tavoitteet, mikä mahdollistaa yhteistoiminnan koulujen kanssa. Työpajojen arviointiin sovelsimme kasvatustieteiden puolelta ”student engagement” tutkimusta, jolloin kykenemme arvioimaan ja mittaamaan osanottajien kokemuksia perustuen heidän käyttäytymiseen, tunnetiloihin sekä kognitiiviseen oppimiseen. Järjestimme kuusi työpajaa, missä tietojärjestelmä oli keskeisessä roolissa. Neljä työpajaa järjestettiin Suomessa eri-ikäisille ja eri kansallisuuksista tuleville osanottajille. Kaksi työpajaa järjestettiin Puolassa. Perustuen näistä työpajoista kerättyihin kokemuksiin voimme sanoa, että tulokset olivat lupaavia iästä ja kansallisuudesta riippumatta. Saimme sovellukset toimimaan sujuvasti ja nuoret innostumaan sovellusten avulla. Innostuksen määrän saimme selville kehitetyllä mittausmenetelmällä. Suurin osa työstä on tehty osana Luova Eurooppa rahoitteista People’s Smart Sculpture hanketta. Hankkeen saksalaiset partnerit hyödynsivät äänimaisemasovellusta osana kaupunkisuunnittelua, mikä voisi olla yksi jatkotutkimuksen aihe