Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

KEER2022

Avanttítol: KEER2022. DiversitiesDescripció del recurs: 25 juliol 202

Intelligent Avatar on E-Learning using Facial Expression and Haptic

 The process of introducing emotion can be improved through three-dimensional (3D) tutoring system. The problem that still not solved is how to provide realistic tutor (avatar) in virtual environment. This paper  propose an approach to teach children on understanding emotion sensation through facial expression and sense of touch (haptic).The algorithm is created by calculating constant factor (f) based on maximum value of RGB and magnitude force then magnitude force range will be associated into particular colour. The Integration process will be started from rendering the facial expression then followed by adjusting the vibration power to emotion value. The result that achieved on experiment, it show around 71% students agree with the classification of magnitude force into emotion representation. Respondents commented that high magnitude force create similar sensation when respondents feel anger, while low magnitude force is more relaxing to respondents. Respondents also said that haptic and facial expression is very interactive and realistic

IDEOLOGICAL REPRESENTATION OF GLOBALISM IN RADIOHEAD MUSIC VIDEOS (A SEMIOTIC ANALYSIS OF 2+2=5 AND NO SURPRISES MUSIC VIDEOS)

Music video is a media of communication between the musicians and their listener or viewers that conveys message or represents the ideas in a form of images, lyrics and music. This study investigates the way ideological representation of globalism is represented in 2+2=5 and No Surprises music videos. The theoretical framework employs in this research is Barthes’s (1977) semiotic theory of orders signification which focuses on first order signification (denotation) and second order signification (connotation) of the visual elements of music videos. In addition, this study employs Goodwin’s (1992) theory of music video to analyze the key features of music video. The findings of the investigation show that globalism is represented through the lyrics and the visual images in the two music videos. The lyrics analysis shows that the narrative story in 2+2=5 music video signifies public mind control and anger to the totalitarian government. While in No Surprises music video, the narrative story signifies the modern society, an absolute power of the corrupt government, the hopeless and powerless feelings of the lower class society. In the visual analysis, the findings of the investigation show that there are some similarities and differences in the way of illustrating the ideology of globalism in 2+2=5 and No Surprises music videos. The similarities are represented through the use of the symbolic visual style, the use of black and white color saturations and the amplifying relationship between the lyrics and visual. Meanwhile, the differences are represented through the use of warm (red) color saturation that signifies agitation and anger and the intertextual references. Whereas in No Surprises, the use of close up frame size that signifies emotion and the use of cool (blue) color saturation represent the way of illustrating globalism. Keywords: Ideology, Representation, Globalism, Music Vide

INTERACTIVE SONIFICATION STRATEGIES FOR THE MOTION AND EMOTION OF DANCE PERFORMANCES

The Immersive Interactive SOnification Platform, or iISoP for short, is a research platform for the creation of novel multimedia art, as well as exploratory research in the fields of sonification, affective computing, and gesture-based user interfaces. The goal of the iISoP’s dancer sonification system is to “sonify the motion and emotion” of a dance performance via musical auditory display. An additional goal of this dissertation is to develop and evaluate musical strategies for adding layer of emotional mappings to data sonification. The result of the series of dancer sonification design exercises led to the development of a novel musical sonification framework. The overall design process is divided into three main iterative phases: requirement gathering, prototype generation, and system evaluation. For the first phase help was provided from dancers and musicians in a participatory design fashion as domain experts in the field of non-verbal affective communication. Knowledge extraction procedures took the form of semi-structured interviews, stimuli feature evaluation, workshops, and think aloud protocols. For phase two, the expert dancers and musicians helped create test-able stimuli for prototype evaluation. In phase three, system evaluation, experts (dancers, musicians, etc.) and novice participants were recruited to provide subjective feedback from the perspectives of both performer and audience. Based on the results of the iterative design process, a novel sonification framework that translates motion and emotion data into descriptive music is proposed and described

The gesture's narrative : contemporary music for percussion

Musical performance gestures are recognized by the majority of theoreticians as a critical factor of a musical performance. The aim of the musical performance may consist in not only communicating the musical signs that form a musical piece, but conveying the meaningful succession of gestures, facial expressions and body movements. This meaningful succession, or otherwise the “gesture´s narrative” is assumed to be quite important for the process of directing the audience towards the intended interpretation. Recording music allowed audiences to listen to music without having to go to a musical event for this purpose. On the one hand, this made the listening experience more intense, allowing to concentrate on the aural information exclusively, but, on the other hand, it also imposed restrictions on people’s perception, as the syncretic listening and seeing experience became separated into constituents. Gestures can be considered as operating features of a person’s perception-action system. It presupposes significance of a meaning that involves more than just a physical movement. Movements can be subdivided into specific patterns and conceptualized. Conceptualized gestures are kept in people’s minds as single units, and the subdivision operations are carried out both by performers and the audience. Musical communication through gestures is therefore not about movement only, it should be viewed as structured interactions. For this research, solo percussion contemporary music performance will be analyzed. Overall, percussive music performance is extremely wide, and is accompanied by bright visual images provided by musicians themselves. From this perspective, observation over percussionists’ playing manner and it´s audience provides the researcher an opportunity to understand a narrative ability of music through musicians’ gestures. The quantitative research design divided in three experiments was chosen for the purpose of this study, which can be referred to as the description of the objective reality by using numbers in order to construct meaningful models reflecting various relationships between objects or phenomena. These numerical entities are not the reality itself, but a way of representing it. Moreover, the chosen experimental design gives an opportunity to not only establish the existence of certain effects of one variable on the other one, but also study the magnitude of these effects, considering the major two research questions: Is it possible to detect a percussive gesture’s narrative ? How does the percussive gesture influences the perception of musical narrative?Os gestos que produzem o som são reconhecidos pela maioria dos teóricos como um factor determinante da performance musical e da sua percepção. A conexão entre os gestos, sons e percepção de determinado discurso musical foi já abordada por um amplo número de cientistas, ainda que não haja um claro consenso quanto à medida em que essa conexão é fundamental ou quanto às operações cognitivas subjacentes à percepção de uma peça musical. O objetivo da interpretação em música consistirá não apenas em comunicar os sinais musicais que formam uma obra, mas também em transmitir a sucessão significativa de gestos, expressões e movimentos do corpo. Esta sucessão significativa, ou de outra forma exposto, a “narrativa do gesto” é considerada muito importante para o processo de condução de um público para a interpretação pretendida. O avanço da tecnologia neste estádio de desenvolvimento da sociedade, criou excelentes oportunidades para permitir a separação de atividades auditivas e visuais da música. A gravação e posterior difusão musical, permitiu que o público consumisse música sem ter que, para essa finalidade, presenciar um evento musical. Por um lado, esse fenómeno tornou a experiência de escuta mais frequente e porventura mais focada, permitindo ao ouvinte concentrar-se na informação auditiva exclusivamente. Mas, por outro lado, também impôs restrições a uma experiência musical sincrética com o ouvir e ouvir e ver, a separaram-se em constituintes dentro do fenómeno musical. Os gestos podem ser considerados características de funcionamento do sistema de percepção/ acção de um ser humano. Pressupõe isso a atribuição de expressão a um significado que envolve mais do que apenas um movimento físico. Os movimentos podem ser subdivididos em padrões específicos e conceptualizados. Estes gestos conceptualizados são mantidos como unidades singulares, e as operações de subdivisão significantes são levadas a cabo tanto pelos performers como pelo seu público. A comunicação musical através de gestos, não deve, portanto, ser olhada apenas sobre os aspectos do movimento, mas sim como uma interação estruturada e musicalmente contextualizada. Os processos descritos acima resultam em grande parte do ambiente de envolvimento do individuo ouvinte e dependem fortemente da sua singularidade e contexto cultural. Nem todos os movimentos poderão ser chamados gestos performativos para além daqueles aqueles cuja acção é de natureza intencionalmente expressiva ou inerente à produção de som. Nesta pesquisa, a performance de música contemporânea para percussão solo será analisada. De uma maneira geral, o desempenho dos percussionistas é, do ponto de vista visual, extremamente rico na formação de gestos. Nessa perspectiva, a observação de uma audiência sujeita à sua presença, com e sem contacto visual com a sua acção, fornece uma oportunidade de aproximação ao estudo do gesto e da sua narrativa, do ponto de vista da percepção do discurso musical. Um desenho de pesquisa quantitativa dividida em três experiências, foi o caminho escolhido para o presente estudo. Produziu-se uma descrição da realidade objectiva usando números, de modo a construir modelos significativos que pudessem reflectir as várias relações entre objectos ou fenómenos . Estas entidades numéricas não serão assim uma realidade em si, mas uma maneira possível de a representar . O processo de experimentação, dividido em três partes, dá-nos a oportunidade para perceber não só a existência dos efeitos de uma variável sobre a outra (visual e auditiva), mas também permite uma reflexão sobre a magnitude desses efeitos, tentando assim responder ás questões que levam a esta investigação: É possivel detectar uma narrativa no gesto percussivo? Como é que o gesto percussivo influencia a percepção do discurso musical