Beats & Units: A story-game design framework

There are no recipes or rules to develop games, any more than there are to develop stories. When we try to define a game system that not only has to create a balanced experience, but also has to tell a story, which can engage players in creating empathy and meaning, everything gets complicated. We faced this problem when we had to develop a serious game with the goal of promoting discussion and awareness among children around nutrition: FlavourGame. In this regard, we needed not only to design game mechanics that would feel complete and progressive, but also to create a narrative that provided meaning to the game experience, in order to ensure an underlying layer to the context of nutrition. For that, we had to develop a framework that could help the team in guiding the telling of the story while designing the progression of the game. In this article, we present the full framework as a story-game design approach to be employed in the creation of serious narrative games

Narrative design of sadness in Heavy Rain

Aware of all the problems videogames have faced trying to elicit sadness from its players, we decided to analyse the videogame Heavy Rain in virtue of its capabilities to induce sadness in the players. The game was studied under two different perspectives: the character’s non-verbal expressivity and the audiovisual artistic properties of the game. We have found that for the narrative design of sad interactive sequences, the game followed a three-stepped model made of: attachment, rupture, and passivity

Bodily sensation maps: Exploring a new direction for detecting emotions from user self-reported data

The ability of detecting emotions is essential in different fields such as user experience (UX), affective computing, and psychology. This paper explores the possibility of detecting emotions through user-generated bodily sensation maps (BSMs). The theoretical basis that inspires this work is the proposal by Nummenmaa et al. (2014) of BSMs for 14 emotions. To make it easy for users to create a BSM of how they feel, and convenient for researchers to acquire and classify users’ BSMs, we created a mobile app, called EmoPaint. The app includes an interface for BSM creation, and an automatic classifier that matches the created BSM with the BSMs for the 14 emotions. We conducted a user study aimed at evaluating both components of EmoPaint. First, it shows that the app is easy to use, and is able to classify BSMs consistently with the considered theoretical approach. Second, it shows that using EmoPaint increases accuracy of users’ emotion classification when compared with an adaptation of the well-known method of using the Affect Grid with the Circumplex Model, focused on the same set of 14 emotions of Nummenmaa et al. Overall, these results indicate that the novel approach of using BSMs in the context of automatic emotion detection is promising, and encourage further developments and studies of BSM-based methods

Emotion-driven interactive storytelling.

Interactive storytelling has attracted plenty of research interest in recent years. Most current interactive storytelling systems follow a goal-oriented approach to story representation, i.e. the user is engaged with the story through fulfilling a number of goals rather than empathising with the characters and experiencing anenriched emotional experience (Pizzi and Cavazza 2007). This fails to satisfy potential users who are oriented to traditional media, such as movies (Louchart et al. 2008) and demographic groups who are interested in attractive and challenging stories (Duh et al. 2010). Given this consideration, an emotion-driven interactive storytelling approach is proposed in this research. In contrast to the goal-oriented interactive storytelling approach, emotion-driven interactive storytelling attempts to create an engaging emotional experience, and involve the user’s emotion with the characters. More importantly, the user’s emotions, evoked by empathising with the characters, determine the character’s behaviours and therefore have an impact on the whole storyline. In this sense, emotions, as a driving force, directly and explicitly contribute to storytelling and the user experience. An interactive video was made by re-editing existing TV material to interpret the concept of emotion-driven interactive storytelling. The examination of user experience of playing this interactive video revealed that non-gamers were more likely to be emotionally involved with the interactive video and empathise with the character. Participants in this group also exhibited higher enjoyment and engagement than gamers. In addition, females were found more likely to empathise with the character and satisfy with the storyline. However because the TV material used to make the interactive video was female-oriented, males failed to enjoy and engage themselves as much as females. But it is important to note that in comparison to males’ previous experience of watching TV Ugly Betty, emotion-driven interactive storytelling increased their enjoyment and engagement. Therefore, emotion-driven interactive storytelling enriches the approach to developing interactive storytelling systems and has the potential to provide an engaging user experience to some types of users. Future research possibilities are discussed with respect to a wider population and research where materials suitable for both genders are presented

Inferência de estados afetivos em ambientes educacionais : proposta de um modelo híbrido baseado em informações cognitivas e físicas

Orientador: Prof. Dr. Andrey Ricardo PimentelDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 11/12/2018Inclui referências: p.119-127Área de concentração: Ciência da ComputaçãoResumo: Na comunidade científica é comum o entendimento de que os softwares educacionais precisam evoluir para garantir um suporte mais efetivo ao processo de aprendizagem. Uma limitação recorrente destes softwares refere-se à falta de funcionalidades de adaptação às reações afetivas dos estudantes. Esta limitação torna-se relevante pois as emoções têm influencia direta no processo de aprendizagem. Reconhecer as emoções dos estudantes é o primeiro passo em direção a construção de software educativos sensíveis ao afeto. Trabalhos correlatos reportam relativo sucesso na tarefa de reconhecimento automático das emoções dos estudantes. No entanto, grande parte dos trabalhos correlatos utiliza sensores pouco práticos, intrusivos e caros que normalmente monitoram apenas reações físicas. No contexto educacional o conjunto de emoções a ser considerado no processo de reconhecimento deve observar as singularidades deste domínio. Sendo assim, neste trabalho a inferência é realizada utilizando uma abordagem de quadrantes formadas pelas dimensões valência e ativação. Estes quadrantes representam situações relevantes para a aprendizagem e podem ser utilizados para embasar adaptações no ambiente computacional. Diante disto, esta pesquisa apresenta a proposta de um modelo híbrido de inferência de emoções de estudantes durante o uso softwares educacionais. Este modelo tem como principal característica a utilização simultânea de informações oriundas de reações físicas (expressões faciais) e cognitivas (eventos no software educacional). Esta abordagem fundamenta-se na perspectiva teórica de que as emoções humanas são fortemente relacionadas com reações físicas, mas também são influenciadas por processos racionais ou cognitivos. A combinação de expressões faciais e informações sobre os eventos do software educacional permite a construção de uma solução de baixo custo e intrusividade. Além disso, esta solução apresenta viabilidade de utilização em larga escala e em ambientes reais de ensino. Experimentos realizados com estudantes em um ambiente real de ensino demonstraram a viabilidade desta proposta. Este fato é importante, considerando-se que a abordagem proposta neste trabalho é pouco explorada na comunidade científica e requer a fusão de informações bastante distintas. Nestes experimentos, foram obtidas acurácia e índice Cohen Kappa próximas de 66% e 0,55, respectivamente, na tarefa de inferência de cinco classes de emoções. Embora esses resultados sejam promissores quando comparados a trabalhos correlatos, entende-se que eles podem ser aprimorados no futuro, incorporando-se novos dados ao modelo proposto. Palavras-chave: Inferência de Emoção, Emoção Relacionada à Aprendizagem, Tutoria Afetiva, Computação Afetiva.Abstract: In the scientific community there is a common understanding that educational software must evolve to ensure more effective support to the learning process. A common limitation of these software refers to the lack of adaptive features to students' affective reactions. This limitation becomes relevant because the emotions have a direct influence on the learning process. Recognizing students' emotions is the first step toward building affect-sensitive educational software. Related work reports relatively successful in the task of automatically recognize students' emotions. However, most studies use impractical, intrusive and expensive sensors that typically monitor only physical reactions. In the educational context the set of emotions to be considered in the recognition process must observe the singularities of this domain. Thus, in this work the inference is performed using a quadrant approach formed by the valence and activation dimensions. These quadrants represent situations relevant to learning and can be used to support adaptations in the computational environment. So, this research presents a proposal of a hybrid model to infer emotions of students while using educational software. This model has as its main feature the simultaneous use of information coming from physical reactions (facial expressions) and cognitive (events in the educational software). This approach is based on the theoretical perspective that human emotions are strongly related with physical reactions, but are also influenced by rational or cognitive processes. Combining facial expressions and information about the events of educational software allows the construction of a low-cost and intrusiveness solution. In addition, this solution presents feasibility for use in large scale in real learning environments. Experiments with students in a real classroom demonstrated the feasibility of this proposal. This is important, considering that the approach proposed in this work is little explored in the scientific community and requires the fusion of quite different information. In these experiments, accuracy and Cohen Kappa index close to 66% and 0,55, respectively, were obtained in the inference of five emotion classes. Although these results are promising when compared to related works, it is understood that they can be improved in the future by incorporating new data into the proposed model. Keywords: Emotion inference, Learning related emotion, Affective tutoring, Affective Computing

Convergência entre o cinema e a realidade virtual

Doutoramento em Ciências e Tecnologias da ComunicaçãoEsta tese tem como objectivos centrais apontar hipóteses sobre a origem dos problemas de criação e desenvolvimento emocional em plataformas de entretenimento interactivo e na sequência da comprovação dessas hipóteses apontar possíveis soluções. Nesse sentido este projecto assume relevância na discussão centrada sobre o desenvolvimento de elementos emocionais para narrativas interactivas estruturadas em ambientes virtuais. A forma de enquadrar o conhecimento a trabalhar foi concretizado no estudo sobre aspectos da resposta emocional na convergência entre o Cinema e a Realidade Virtual (RV). Procurou-se definir similaridades e diferenças não apenas nos artefactos criativos mas também nos aspectos comunicacionais entre o artefacto e os receptores. Foi a partir desta primeira abordagem que desenvolvemos a nossa hipótese principal: que a RV sofre de um deficit de estimulação de diversidade emocional nos seus receptores quando comparada com o Cinema. Com esta hipótese foi necessário proceder a estudos empíricos no sentido de determinar a veracidade das nossas conjecturas teóricas. Assim testámos 14 artefactos de RV em termos emocionais e depois comparámos os resultados com testes prévios realizados sobre segmentos fílmicos. Deste estudo empírico retirámos a confirmação da nossa hipótese mas determinámos também a (provável) dificuldade que as nossas hipóteses teriam de enfrentar. A um nível emocional mais profundo, descobrimos que a provável causa da falta de diversidade deriva de um paradoxo, entre a interactividade necessária ao artefacto e a inactividade necessária ao receptor para que este possa sentir determinadas emoções. Estamos assim perante um paradoxo da emoção inter(activa). Partindo deste paradoxo, desenvolvemos um estudo de análise de conteúdos sobre um grupo de sequências fílmicas previamente testadas por nós e em outros testes psicológicos, em termos da especificidade das emoções despoletadas por cada sequência. A análise foi realizada com base em critérios da linguagem formal fílmica mas em vez de realizar um estudo subjectivo de cada sequência, desenvolvemos uma abordagem empírica tendo como grupo alvo, realizadores e argumentistas de cinema. A análise dos excertos produziu uma base de dados de técnicas estilísticas que serviriam ao desenvolvimento da nossa metodologia de design de emoção geral assim como para responder ao caso específico levantado pelo paradoxo anterior. Para este caso específico desenvolvemos uma nova técnica de interacção a que chamámos de toque físico-virtual. Com os estudos empíricos realizados e as metodologias de design de emoção geral e específica estabelecidas, avançámos para a primeira aproximação à prototipagem, ou seja para a conversão das semânticas fílmicas em semânticas RV e por fim procedeu-se à implementação do protótipo. O protótipo foi depois submetido a um conjunto de testes para determinar da sua suficiência, usabilidade e facilidade de uso. Os resultados obtidos indicam que foi atingindo um nível satisfatório na estimulação da diversidade emocional.This thesis has as its central objectives the identification of the source of problems in the elicitation of emotion within interactive platforms, followed by the advancing of possible solutions to the hypotheses thus arrived at. In this regard, the dissertation seeks to make a valid contribution to discussions related with the development of emotional elements in interactive narratives built into virtual environments. The conceptual framework has been grounded in research on aspects of emotional response in the convergence between Film and Virtual Reality, looking for similarities and differences not only in creative artefacts but also in the communication between those artefacts and their consumers. From this first approach, we have stated our main hypothesis: that VR suffers from a deficit in the emotional diversity elicited in receivers when compared with that of Film. With this hypothesis, it was necessary to proceed first with empirical studies in order to ascertain the veracity of our theoretical conjectures. So we have tested 14 VR pieces in emotional terms through self-reports and then we compared the results with previous studies carried out in the analysis of film. From this empirical study we find our hypothesis confirmed but we have also determined the (probable) main difficulty with the reliability of the hypotheses. At a deeper emotional level, we have discovered that the probable cause of lack of diversity derives from a paradox, between the interactivity needed by the artefact and the inactivity needed by the user to feel certain emotions. We are therefore faced with a paradox of emotion inter(activity). Taking our lead from this paradox, we developed a content-analysis study of a selected group of film sequences previously tested by us and in other psychological studies, in terms of the specificity of the emotions elicited by each film clip. The analysis was made following objective film stylistics criteria but instead of doing a subjective analysis of each sequence, we developed an empirical approach grounded in the ideas of a group of filmmakers and film writers, analysing our excerpts in order to produce a database of possible formal film techniques. The database would serve as our main design methodology for emotional diversity and for dealing with the specific problematic areas we had found previously in our interactive artefacts study. For this specific problem, we developed a new interactive technique that we named virtual body touch. With the empirical studies carried out and the methodologies for the general and specific design of emotional diversity established, we moved on to the first stages in prototyping, mainly the conversion of film semantics methodologies into VR semantics. The prototyping then used these converted interactive categories and parameters to produce results in respect of scenery and characterization. Finally, we have submitted the prototype to a series of validation tests to determine the sufficiency, usability and easiness of the tool. This validation process obtained results which indicate a satisfactory level of appraisal by the experimenters. Finally, we present possible suggestions to be taken up and explored in further research work

4º Congresso da Associação Portuguesa de Ciências da Comunicação: “Repensar os Media: Novos Contextos da Comunicação e da Informação”: livro de actas

Todas as comunicações incluídas neste livro correspondem a trabalho apresentado no 4º Congresso da Associação Portuguesa de Ciências da Comunicação 4º SOPCOM, Realizado em Aveiro, Portugal, 20 e 21 de Outubro de 2005. As comunicações reflectem a opinião dos seus autores.or ganizado por : Universidade de Aveiro SOPCOM Associação Portuguesa de Ciências da Comunicação patr ocinador es: Fundação para a Ciência e a Tecnologia Instituto de Engenharia Electrónica e Telemática de Aveiro Universidade Lusófona de Humanidades e Tecnologias copatr ocinador es técnicos: Media XXI Revista de Comunicação e Sociedade da Informação Caixa Geral de Depósitos Delta Caféspublishe