Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI

As an emerging interaction paradigm, physiological computing is increasingly being used to both measure and feed back information about our internal psychophysiological states. While most applications of physiological computing are designed for individual use, recent research has explored how biofeedback can be socially shared between multiple users to augment human-human communication. Reflecting on the empirical progress in this area of study, this paper presents a systematic review of 64 studies to characterize the interaction contexts and effects of social biofeedback systems. Our findings highlight the importance of physio-temporal and social contextual factors surrounding physiological data sharing as well as how it can promote social-emotional competences on three different levels: intrapersonal, interpersonal, and task-focused. We also present the Social Biofeedback Interactions framework to articulate the current physiological-social interaction space. We use this to frame our discussion of the implications and ethical considerations for future research and design of social biofeedback interfaces.Comment: [Accepted version, 32 pages] Clara Moge, Katherine Wang, and Youngjun Cho. 2022. Shared User Interfaces of Physiological Data: Systematic Review of Social Biofeedback Systems and Contexts in HCI. In CHI Conference on Human Factors in Computing Systems (CHI'22), ACM, https://doi.org/10.1145/3491102.351749

Computer Mediated Reality Technologies: A Conceptual Framework and Survey of the State of the Art in Healthcare Intervention

The trend of an ageing and growing world population, particularly in developed countries, is expected to continue for decades to come causing an increase in demand for healthcare resources and services. Consequently, demand is growing faster than rises in funding. The UK government, in partnership with the European Commission’s Vision for 2020, propose a paradigm shift towards the delivery of more patient-centred self-care interventions, facilitated by novel ubiquitous computer mediated reality technology applications, as a key strategy to overcome the scarcity of health resources gap. If this vision is to become a reality, it is crucial that state of the art research focuses efforts on the development of applications that support the delivery of patient-centred self-care interventions

A System for the Measurement of the Subjective Visual Vertical Using a Virtual Reality Device

The Subjective Visual Vertical (SVV) is a common test for evaluating the perception of verticality. Altered verticality has been connected with disorders in the otolithic, visual or proprioceptive systems, caused by stroke, Parkinson’s disease or multiple sclerosis, among others. Currently, this test is carried out using a variety of specific, mostly homemade apparatuses that include moving planes, buckets, hemispheric domes or a line projected in a screen. Our aim is to develop a flexible, inexpensive, user-friendly and easily extensible system based on virtual reality for the measurement of the SVV and several related visual diagnostic tests, and validate it through an experimental evaluation. Our evaluation showed that the proposed system is suitable for the measurement of SVV in healthy subjects. The next step is to perform a more elaborated experimentation on patients and compare the results with the measurements obtained from traditional methods

Pengembangan Media Pembelajaran Tata Surya berbasis Virtual Reality untuk Siswa Kelas 6 Sekolah Dasar dengan Evaluasi Kepuasan Pengguna terhadap Elemen Multimedia

Dalam studi ini, kami mengembangkan aplikasi virtual reality untuk mempelajari tata surya di tingkat sekolah dasar. Tujuan pembuatan aplikasi ini untuk menyediakan media pembelajaran berbasis multimedia bagi siswa agar dapat memahami konsep tata surya. Multimedia Development Life Cycle (MDLC) adalah tahap pengembangan sistem yang digunakan untuk membangun aplikasi virtual reality. MDLC terdiri dari tahapan konsep manufaktur, desain, pengumpulan bahan, perakitan, pengujian, dan distribusi. Hasil tes penerimaan pengguna yang dilakukan oleh satu orang guru pengampu menunjukkan hasil 81,25%, sedangkan yang dilakukan oleh 26 siswa menunjukkan hasil 88,63%. Berdasarkan hasil tes penerimaan oleh guru diperoleh saran perbaikan aplikasi pada sisi interaktifitas pengguna. Evaluasi kepuasan pengguna terhadap aplikasi dilakukan dengan kuesioner berdasarkan empat elemen multimedia: teks, interaktivitas, animasi, dan gambar grafis. Hasil evaluasi penggunaan teks memiliki nilai 3,57, grafik bernilai 3,52, animasi bernilai 3,54, dan interaktivitas memiliki nilai 3,51. Berdasarkan hasil tes, dapat disimpulkan bahwa responden puas dengan penggunaan elemen multimedia pada aplikasi tersebut, dan aplikasi tersebut dapat membantu mereka untuk memahami topik pembelajaran lebih baik daripada metode pembelajaran dan pengajaran konvensional. AbstractIn this study, we developed a virtual reality application for learning the solar system at the elementary school. The purpose of making this application is to provide multimedia-based learning media for students to be able to understand the concept of the solar system. Multimedia Development Life Cycle is a development stage of the system used to build virtual reality applications. MDLC consists of stages of the manufacturing concept, design, material collecting, assembly, testing, and distribution. Results of user acceptance test conducted by one teacher show the results of 81.25%, while that is done by 26 student shows the results of 88.63%. Based on the acceptance test results by the teacher, there are suggestions to improve the application on the user interactivity aspect. Evaluation of user satisfaction of the applications is done by a questionnaire based on the four elements of multimedia: text, interactivity, animation, and a graphical image. The result of the evaluation of the use of text has a value of 3.57, the graphic has a value of 3.52, animation has a value of 3.54, and interactivity has a value of 3.51. Based on the test results, it can be concluded that the respondents are satisfied with the use of multimedia elements on the application, and the application can help them to understand the learning topic better than conventional methods of learning dan teaching

NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas

Virtual Reality for Pain Management in Cancer:A Comprehensive Review

Virtual Reality is a computer-simulated 3-Dimensional technology in which the user interacts via different senses: visual, auditory, tactile, and/or olfactory. In the past decades, it has been argued that Virtual Reality as a technique could be applied in the clinical environment to successfully manage pain. This article provides a systematic review of research on Virtual Reality and pain management for patients who are suffering from cancer. More specifically, this article focuses on all types of Virtual Reality technologies (Non-Immersive, Semi-Immersive, Fully-Immersive) which has been developed and released to manage the pain which evokes from the treatment of cancer. An exhaustive search identified 23 relevant studies from 2010 to 2020. Overall, the identified studies indicated that Virtual Reality can improve the experience of pain for patients who are suffering from cancer. It was also found that, if Virtual Reality is appropriately designed, the pain which is arising from cancer treatments can be reduced. Even though some positive outcomes have been reported, overall, the results are inconclusive and studies that examine specifically the treatment of pain in cancer patients are limited. Further research needs to be conducted, to articulate clearly, under what circumstances Virtual Reality is an effective tool for cancer patients, and under what factors Virtual Reality can be the solution to the pain patients are experiencing.</p

Exploring and Designing Embodied Mindfulness-based Interactive Technologies For Mental Wellbeing

Mindfulness practices have become increasingly popular due to their significant benefits for mental wellbeing. However, the required skills to sustain and regulate attention are challenging to develop. Both the benefits and the challenges of mindfulness training have attracted a growing Human-Computer Interaction (HCI) interest in designing interactive technologies for mindfulness. Most mindfulness-based technologies employ metaphorical mappings of mindfulness states to support awareness of such states and facilitate attention regulation, although the rationale for such mappings is seldom articulated. This thesis addresses this shortcoming, drawing on first-person experiences and embodied metaphors of mindfulness practices to design novel multi-sensory technologies for mental wellbeing. This work concentrates on focused attention mindfulness (FAM) practices which involve sustaining selective attention moment by moment on a chosen object, and are considered the most widely accessible among novices. In particular, the research presented in this thesis is structured around two distinct FAM practices, in 6 different but interrelated studies. On the one hand, mandala colouring illustrates a nonstatic FAM practice with an external object of attention. On the other hand, sitting meditation represents a static FAM practice with an internal object of attention. The first and second study investigate in-depth mandala colouring, respectively, by interviewing 21 people who regularly engage with this practice and through an autoethnographic and heuristic evaluation of the 14 top-ranked mandala colouring apps. Findings informed the development of Anima, a peripheral colour palette materialising EEG-based mindfulness states onto colours for mandala colouring. In the third study, Anima’s design solutions were evaluated with 12 experienced participants. The fourth study follows a research-through design and material speculation approach to understand the role of body during meditation with 24 experts. This is extended in the fifth study, in which the 16 most popular meditation apps are evaluated through auto-ethnography. An identified opportunity of designing for the physical sensations that arise during meditation was considered through the design of the WarmMind, that provide warmth patterns on the upper body. The WarmMind prototype was evaluated in the fifth study, through a user study with 10 participants experiencing thermal metaphors for mapping meditation states on the upper-body. Besides the technological design contributions of Anima and WarmMind, the contributions of this thesis include the concepts of intricate confines to scaffold mindful movement to support non-static FAM practices with an external object of attention, and interoceptive interaction to facilitate focusing inwards during meditation to support static FAM practices with an internal object of attention; as well as a framework built on embodied metaphor theories to designing meditation technologies. Collectively, these studies highlight the value of the human body in mindfulness-based technologies in HCI, as well as theoretical and methodological contributions to designing around mindfulness for mental wellbeing in HCI

A Design Exploration of Affective Gaming

Physiological sensing has been a prominent fixture in games user research (GUR) since the late 1990s, when researchers began to explore its potential to enhance and understand experience within digital game play. Since these early days, it has been widely argued that “affective gaming”—in which gameplay is influenced by a player’s emotional state—can enhance player experience by integrating physiological sensors into play. In this thesis, I conduct a design exploration of the field of affective gaming by first, systematically exploring the field and creating a framework (the affective game loop) to classify existing literature; and second by presenting two design probes, to probe and explore the design space of affective games contextualized within the affective game loop: In the Same Boat and Commons Sense. The systematic review explored this unique design space of affective gaming, opening up future avenues for exploration. The affective game loop was created as a way to classify the physiological signals and sensors most commonly used in prior literature within the context of how they are mapped into the gameplay itself. Findings suggest that the physiological input mappings can be more action-based (e.g., affecting mechanics in the game such as the movement of the character) or more context-based (e.g., affecting things like environmental or difficulty variables in the game). Findings also suggested that while the field has been around for decades, there is still yet to be any commercial successes, so does physiological interaction really heighten player experience? This question instigated the design of the two probes, exploring ways to implement these mappings and effectively heighten player experience. In the Same Boat (Design Probe One) is an embodied mirroring game designed to promote an intimate interaction, using players’ breathing rate and facial expressions to control movement of a canoe down a river. Findings suggest that playing In the Same Boat fostered the development of affiliation between the players, and that while embodied controls were less intuitive, people enjoyed them more, indicating the potential of embodied controls to foster social closeness in synchronized play over a distance. Commons Sense (Design Probe Two) is a communication modality intended to heighten audience engagement and effectively capture and communicate the audience experience, using a webcam-based heart rate detection software that takes an average of each spectator’s heart rate as input to affect in-game variables such as lighting and sound design, and game difficulty. Findings suggest that Commons Sense successfully facilitated the communication of audience response in an online entertainment context—where these social cues and signals are inherently diminished. In addition, Commons Sense is a communication modality that can both enhance a play experience while offering a novel way to communicate. Overall, findings from this design exploration shows that affective games offer a novel way to deliver a rich gameplay experience for the player

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with