Affective Medicine: a review of Affective Computing efforts in Medical Informatics

Background: Affective computing (AC) is concerned with emotional interactions performed with and through computers. It is defined as “computing that relates to, arises from, or deliberately influences emotions”. AC enables investigation and understanding of the relation between human emotions and health as well as application of assistive and useful technologies in the medical domain. Objectives: 1) To review the general state of the art in AC and its applications in medicine, and 2) to establish synergies between the research communities of AC and medical informatics. Methods: Aspects related to the human affective state as a determinant of the human health are discussed, coupled with an illustration of significant AC research and related literature output. Moreover, affective communication channels are described and their range of application fields is explored through illustrative examples. Results: The presented conferences, European research projects and research publications illustrate the recent increase of interest in the AC area by the medical community. Tele-home healthcare, AmI, ubiquitous monitoring, e-learning and virtual communities with emotionally expressive characters for elderly or impaired people are few areas where the potential of AC has been realized and applications have emerged. Conclusions: A number of gaps can potentially be overcome through the synergy of AC and medical informatics. The application of AC technologies parallels the advancement of the existing state of the art and the introduction of new methods. The amount of work and projects reviewed in this paper witness an ambitious and optimistic synergetic future of the affective medicine field

Museums and the Metaverse: Emerging Technologies to Promote Inclusivity and Engagement

Over the past two decades, museums have increasingly sought to build connections with the community and increase inclusivity of visitors. At the same time, emerging technologies, such as extended reality (XR) and virtual museums (VM) are increasingly adopted to engage with different generational expectations but also for the purposes of supporting inclusivity and neurodiverse populations. First such technologies were adopted to augment exhibitions in the physical museum space for edutainment. Since then, XR has expanded from room-size environments (CAVEs) and augmented exhibitions to the creation of entire virtual museums, such as The Museum of Pure Form and The Virtual Museum of Sculpture. Digital twins of museums are increasingly common, along with UNESCO World Heritage Sites. Such virtual experiences can be leveraged to prepare neurodiverse visitors prior to visiting a museum. This chapter will outline how existing approaches to social stories and sensory maps may be combined with XR experiences to support neurodiverse visitors and their families. While onsite, immersive technologies can be used both for engagement and to provide accommodations for greater inclusivity and diversity

Contributos para o desenvolvimento de aplicações para crianças com problemas de comunicação e integração

Communication, to express our needs and ideas, is a fundamental piece of our daily lives, enabling our interaction with others and contributing to establish social bonds. When this ability is affected, whether by contextual issues (e.g., noisy environment) or intrinsic conditions (e.g., speech articulation difficulties) it creates a challenging scenario for both the individuals and those surrounding them. Children diagnosed with Austism Spectrum Disorders (ASD) are a notable example of an audience facing these challenges given their frequent difficulties to communicate and socially interact with others. In this context, their challenges not only affect their ability to integrate with family and friends, but can also influence how they perform at school, an important environment for their development. In recent years, efforts have been made to propose technological solutions that contribute to support these children in their communication, but most of the work in the literature mainly focuses on the child entailing a support that does not truly addresses integration. Considering this context, we argue that these challenges need to be addressed considering the needs and motivations of children with ASD, but also keeping in mind the motivations of those around them, helping parents, teachers and friends to take part in the integration. Profiting from previous work on the characterization of the needs and motivations of children with ASD, their families, and teachers, the work carried out explores different communication-related scenarios, at school and at home, and proposes a set of requirements that should be considered to serve them. Considering these requirements, and adopting an iterative user-centered design approach, a first proof-of-concept application is proposed. AMICA, Adaptive Multimodal Integrated Communication for All, is a multi-platform, multiuser and multimodal application that aims to illustrate how some issues that affect children with ASD and those surrounding them can be addressed in order to their lives easier. At its current stage, AMICA already encompasses support for several communication features adapted to serve the child, family, friends, and teachers providing them a space for interaction. The modular multi-platform nature of the work carried out enables AMICA as a testbed for further research on different multimodal ways of communication for children with ASD (e.g., gaze or pictograms), but, also, by allowing the participation of different users, it fosters first studies on the dynamic nature of such communication.A comunicação, para expressar as nossas necessidades e ideias, é uma peça fundamental do nosso dia a dia, permitindo a nossa interação com os outros e contribuindo para estabelecer laços sociais. Quando essa capacidade é afetada, seja por questões contextuais (por exemplo, ambiente ruidoso) ou condições intrínsecas (por exemplo, dificuldades de articulação de fala), ela cria um cenário desafiante tanto para os indivíduos como para os que os rodeiam. As crianças diagnosticadas com Perturbação do Espetro do Autismo (PEA) são um exemplo notável de uma audiência que enfrenta esses desafios, devido às suas frequentes dificuldades para comunicar e interagir socialmente com os outros. Neste contexto, os seus desafios não afetam apenas a sua capacidade de integração com a família e os amigos, podendo, também, influenciar o seu desempenho na escola, um ambiente importante para o seu desenvolvimento. Nos últimos anos, têm sido feitos esforços para propor soluções tecnológicas que contribuam para apoiar essas crianças na sua comunicação, mas a maior parte do trabalho na literatura concentra-se principalmente no uso, pela criança, de apoios que não abordam verdadeiramente a integração. Considerando esse contexto, argumentamos que esses desafios precisam de ser abordados considerando as necessidades e motivações das crianças com PEA, mas também tendo em mente as motivações das pessoas ao seu redor, ajudando pais, professores e amigos a participarem na integração. Aproveitando os trabalhos anteriores sobre a caracterização das necessidades e motivações das crianças com PEA, as suas famílias e professores, o trabalho realizado explora diferentes cenários relacionados à comunicação, na escola e em casa, e propõe um conjunto de requisitos que devem ser considerados para os servir. Considerando esses requisitos e adotando uma abordagem de projeto iterativa, centrada no utilizador, uma primeira aplicação de prova de conceito é proposta. AMICA, Adaptive Multimodal Integrated Communication for All, é uma aplicação multi-plataforma, multi-utilizador e multimodal que visa ilustrar como algumas questões que afetam as crianças com PEA e aqueles que as rodeiam podem ser abordadas para tentar facilitar as suas vidas. No estado atual, a AMICA já engloba o suporte a vários recursos de comunicação adaptados para atender a criança, a família, os amigos e os professores, proporcionando-lhes um espaço de interação. A natureza modular multi-plataforma do trabalho realizado permite que AMICA seja uma base para mais pesquisas sobre diferentes formas multimodais de comunicação para crianças com PEA (por exemplo, olhar ou pictogramas), mas, também, permitindo a participação de diferentes utilizadores, promove primeiros estudos sobre a natureza dinâmica de tal comunicação.Mestrado em Engenharia de Computadores e Telemátic

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

Circus in Motion: A Multimodal Exergame Supporting Vestibular Therapy for Children with Autism

Exergames are serious games that involve physical exertion and are thought of as a form of exercise by using novel input models. Exergames are promising in improving the vestibular differences of children with autism but often lack of adaptation mechanisms that adjust the difficulty level of the exergame. In this paper, we present the design and development of Circus in Motion, a multimodal exergame supporting children with autism with the practice of non-locomotor movements. We describe how the data from a 3D depth camera enables the tracking of non-locomotor movements allowing children to naturally interact with the exergame . A controlled experiment with 12 children with autism shows Circus in Motion excels traditional vestibular therapies in increasing physical activation and the number of movements repetitions. We show how data from real-time usage of Circus in Motion could be used to feed a fuzzy logic model that can adjust the difficulty level of the exergame according to each childs motor performance. We close discussing open challenges and opportunities of multimodal exergames to support motor therapeutic interventions for children with autism in the long-term

Visualization and Interaction Technologies in Serious and Exergames for Cognitive Assessment and Training: A Survey on Available Solutions and Their Validation

Exergames and serious games, based on standard personal computers, mobile devices and gaming consoles or on novel immersive Virtual and Augmented Reality techniques, have become popular in the last few years and are now applied in various research fields, among which cognitive assessment and training of heterogeneous target populations. Moreover, the adoption of Web based solutions together with the integration of Artificial Intelligence and Machine Learning algorithms could bring countless advantages, both for the patients and the clinical personnel, as allowing the early detection of some pathological conditions, improving the efficacy and adherence to rehabilitation processes, through the personalisation of training sessions, and optimizing the allocation of resources by the healthcare system. The current work proposes a systematic survey of existing solutions in the field of cognitive assessment and training. We evaluate the visualization and interaction technologies commonly adopted and the measures taken to fulfil the need of the pathological target populations. Moreover, we analyze how implemented solutions are validated, i.e. The chosen experimental designs, data collection and analysis. Finally, we consider the availability of the applications and raw data to the large community of researchers and medical professionals and the actual application of proposed solutions in the standard clinical practice. Despite the potential of these technologies, research is still at an early stage. Although the recent release of accessible immersive virtual reality headsets and the increasing interest on vision-based techniques for tracking body and hands movements, many studies still rely on non-immersive virtual reality (67.2%), mainly mobile and personal computers, and standard gaming tools for interactions (41.5%). Finally, we highlight that although the interest of research community in this field is increasingly higher, the sharing of dataset (10.6%) and implemented applications (3.8%) should be promoted and the number of healthcare structures which have successfully introduced the new technological approaches in the treatment of their host patients is limited (10.2%)