A Systematic Review of Social Presence: Definition, Antecedents, and Implications

Social presence, or the feeling of being there with a “real” person, is a crucial component of interactions that take place in virtual reality. This paper reviews the concept, antecedents, and implications of social presence, with a focus on the literature regarding the predictors of social presence. The article begins by exploring the concept of social presence, distinguishing it from two other dimensions of presence—telepresence and self-presence. After establishing the definition of social presence, the article offers a systematic review of 233 separate findings identified from 152 studies that investigate the factors (i.e., immersive qualities, contextual differences, and individual psychological traits) that predict social presence. Finally, the paper discusses the implications of heightened social presence and when it does and does not enhance one's experience in a virtual environment

A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Survey on virtual coaching for older adults

Virtual coaching has emerged as a promising solution to extend independent living for older adults. A virtual coach system is an always-attentive personalized system that continuously monitors user's activity and surroundings and delivers interventions - that is, intentional messages - in the appropriate moment. This article presents a survey of different approaches in virtual coaching for older adults, from the less technically supported tools to the latest developments and future avenues for research. It focuses on the technical aspects, especially on software architectures, user interaction and coaching personalization. Nevertheless, some aspects from the fields of personality/social psychology are also presented in the context of coaching strategies. Coaching is considered holistically, including matters such as physical and cognitive training, nutrition, social interaction and mood.The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 769830

Extended Reality for the Clinical, Affective, and Social Neurosciences

Article highlighting the promise of extended reality platforms for greater ecological validity in the clinical, affective, and social neurosciences. This article belongs to the Special Issue: A Decade of Brain Sciences

Ambient hues and audible cues: An approach to automotive user interface design using multi-modal feedback

The use of touchscreen interfaces for in-vehicle information, entertainment, and for the control of comfort settings is proliferating. Moreover, using these interfaces requires the same visual and manual resources needed for safe driving. Guided by much of the prevalent research in the areas of the human visual system, attention, and multimodal redundancy the Hues and Cues design paradigm was developed to make touchscreen automotive user interfaces more suitable to use while driving. This paradigm was applied to a prototype of an automotive user interface and evaluated with respects to driver performance using the dual-task, Lane Change Test (LCT). Each level of the design paradigm was evaluated in light of possible gender differences. The results of the repeated measures experiment suggests that when compared to interfaces without both the Hues and the Cues paradigm applied, the Hues and Cues interface requires less mental effort to operate, is more usable, and is more preferred. However, the results differ in the degradation in driver performance with interfaces that only have visual feedback resulting in better task times and significant gender differences in the driving task with interfaces that only have auditory feedback. Overall, the results reported show that the presentation of multimodal feedback can be useful in design automotive interfaces, but must be flexible enough to account for individual differences

Novel supports to the assessment of cognitive functions through the combined use of technologies and subjective and objective measurements

Tesis por compendio[ES] Las funciones cognitivas incluyen todos los procesos a través de los cuales un individuo percibe, registra, mantiene, manipula, usa y expresa información que está involucrada en cualquier actividad cotidiana. Las principales herramientas estandarizadas se pueden dividir en tres grupos principales: escalas cortas de pruebas de seguimiento cognitivo: cuestionarios, baterías neuropsicológicas generales y pruebas específicas. Estas herramientas están bien validadas y son confiables, pero, en la última década, varias investigaciones han demostrado que algunos pacientes pueden realizar bien estas pruebas neuropsicológicas, incluso cuando tienen dificultades significativas para adaptar sus comportamientos a las actividades de la vida diaria. De acuerdo con esto, más recientemente, un nuevo enfoque ha aumentado sustancialmente, lo que podría proporcionar una mayor validez ecológica en la evaluación de las capacidades cognitivas funcionales que el enfoque estandarizado: el uso de sistemas tecnológicos avanzados para la evaluación neuropsicológica (STAEN). STAEN se refiere a un conjunto de dispositivos y aplicaciones de software tales como pruebas computarizadas, juegos divertidos e interactivos de fantasía (JS) y / o sistemas de realidad virtual simulada (RV) y / o aumentada (RA) que van más allá de las pruebas de evaluación tradicionales y que Brindar la posibilidad de entregar estímulos controlados y dinámicos, en entornos ecológicamente válidos y seguros. Partiendo de estas premisas, el objetivo principal de la tesis era diseñar, desarrollar y validar un SG 2D no inmersivo versus un JS 3D inmersivo y una actividad de la vida diaria en un entorno 3D RV inmersivo versus un RA para la evaluación de funciones cognitivas, comparando la eficacia y efectividad de ellos. El primer estudio 2D incluyó 354 sujetos sanos y se encontraron correlaciones entre el juego y los métodos tradicionales, lo que sugiere que el juego podría ser una herramienta válida para evaluar las funciones cognitivas en adultos. El segundo estudio, comparó la versión 2D versus una versión 3D STAEN, involucró a 94 sujetos sanos y mostró que la versión 3D fue capaz de generar tiempos más bajos y respuestas correctas más altas que la 2D, lo que sugiere evidencia inicial de la eficacia de un sistema más inmersivo en comparación con un sistema no-inmersivo. Aunque este resultado destaca una posible limitación en el uso de diferentes sistemas tecnológicos debido a las diferencias en los dos métodos de interacción (el sistema 2D aplicó el mouse y el teclado; los controladores de dos manos virtuales 3D) y el registro de datos de latencia de hardware y software. Con respecto a la variabilidad individual en edad, género y educación, los hallazgos mostraron consistencia con la literatura de referencia. Específicamente, los más jóvenes mostraron un mayor rendimiento que los mayores; niveles educativos más altos reflejados en una mejor puntuación y sobre género, los resultados mostraron un panorama más compuesto. Además, para mejorar la validez ecológica de la evaluación, el último estudio de esta tesis comparó el rendimiento conductual y las respuestas fisiológicas, durante una tarea de cocina ecológica, entre un sistema virtual y un sistema aumentado en 50 sujetos sanos. La tarea de cocinar consistió en 4 niveles que aumentaron en dificultad. A medida que el nivel aumentó, aparecieron actividades adicionales. Los resultados de comportamiento mostraron que los tiempos son siempre más bajos en realidad virtual que en RA, aumentando constantemente de acuerdo con la dificultad de las tareas. Con respecto a las respuestas fisiológicas, los hallazgos mostraron que la condición RA produjo más excitación y activación individual que la realidad virtual. Para concluir, STAEN está demostrando ser herramientas confiables y efectivas para la evaluación de las funciones cognitivas en adultos, proporcionando más validez ec[CA] Les funcions cognitives inclouen tots els processos a través dels quals un individu percep, registra, manté, manipula, usa i expressa informació que està involucrada en qualsevol activitat quotidiana. Les principals ferramentes estandarditzades es poden dividir en tres grups principals: escales curtes de proves de seguiment cognitiu: qüestionaris, bateries neuropsicológiques generals i proves específiques. Estes ferramentes estan ben validades i són confiables, però, en l'última dècada, diverses investigacions han demostrat que alguns pacients poden realitzar bé estes proves neuropsicológiques, inclús quan tenen dificultats significatives per a adaptar els seus comportaments a les activitats de la vida diària. D'acord amb açò, més recentment, un nou enfocament ha augmentat substancialment, la qual cosa podria proporcionar una major validesa ecològica en l'avaluació de les capacitats cognitives funcionals que l'enfocament estandarditzat: l'ús de sistemes tecnològics avançats per a l'avaluació neuropsicológica (STAEN). STAEN es referix a un conjunt de dispositius i aplicacions de software com ara proves computaritzades, jocs divertits i interactius de fantasia (JS) i / o sistemes de realitat virtual simulada (RV) i / o augmentada (RA) que van més enllà de les proves d'avaluació tradicionals i que brinden la possibilitat de presentar estímuls controlats i dinàmics, en entorns ecològicament vàlids i segurs. Partint d'estes premisses, l'objectiu principal de la tesi era dissenyar, desenrotllar i validar un SG 2D no inmersiu versus un JS 3D inmersiu i una activitat de la vida diària en un entorn 3D RV inmersiu versus un RA per a l'avaluació de funcions cognitives, comparant l'eficàcia i efectivitat d'ells. El primer estudi 2D va incloure 354 subjectes sans i es van trobar correlacions entre el joc i els mètodes tradicionals, la qual cosa suggerix que el joc podria ser una ferramenta vàlida per a avaluar les funcions cognitives en adults. El segon estudi, va comparar la versió 2D versus una versió 3D STAEN, va involucrar a 94 subjectes sans i va mostrar que la versió 3D va ser capaç de generar temps més baixos i respostes correctes més altes que la 2D, la qual cosa suggerix evidència inicial de l'eficàcia d'un sistema més inmersiu en comparació amb un sistema no-inmersiu. Encara que este resultat destaca una possible limitació en l'ús de diferents sistemes tecnològics a causa de les diferències en els dos mètodes d'interacció (el sistema 2D va aplicar el ratolí i el teclat; els controladors de dos mans virtuals 3D) i el registre de dades de latència de hardware i software. Respecte a la variabilitat individual en edat, gènere i educació, les troballes van mostrar consistència amb la literatura de referència. Específicament, els més jóvens van mostrar un major rendiment que els majors; nivells educatius més alts reflectits en una millor puntuació i sobre gènere, els resultats van mostrar un panorama més compost. A més, per a millorar la validesa ecològica de l'avaluació, l'últim estudi d'esta tesi va comparar el rendiment conductual i les respostes fisiològiques, durant una tasca de cuina ecològica, entre un sistema virtual i un sistema augmentat en 50 subjectes sans. La tasca de cuinar va consistir en 4 nivells que van augmentar en dificultat. A mesura que el nivell va augmentar, van aparéixer activitats addicionals. Els resultats de comportament van mostrar que els temps són sempre més baixos en realitat virtual que en RA, augmentant constantment d'acord amb la dificultat de les tasques. Respecte a les respostes fisiològiques, les troballes van mostrar que la condició RA va produir més excitació i activació individual que la realitat virtual. Per a concloure, STAEN està demostrant ser ferramentes confiables i efectives per a l'avaluació de les funcions cognitives en adults, proporcionant més validesa ecològica i objectivitat que els mètodes tradicio[EN] Cognitive functions include all the processes through which an individual perceives, records, maintains, manipulates, uses and expresses information that are involved in any everyday activity. The main standardized tools can be divided in three main groups: short scales of cognitive tracking tests - questionnaires, general neuropsychological batteries, and specific tests. These tools are well-validated and reliable but, in the last decade, several research have shown that some patients can perform these neuropsychological tests well, even when they have significant difficulties in adapting their behaviours to daily life activities. According to this, more recently, a new approach has substantially increased, potentially providing a higher ecological validity in functional cognitive abilities assessment than standardized approach: the use of advanced technological systems for neuropsychological assessment (ATSNA). ATSNA refer to a set of devices and software applications such as computerized tests, fun and interactive fantasy serious games (SG), and/or simulated virtual (VR) and/or augmented (AR) reality systems that go beyond traditional assessment tests and that supply the possibility to deliver controlled and dynamic stimuli, in ecologically valid, and secure environments. Starting from these premises, the main objective of the thesis was to design, develop, and validate a non-immersive 2D SG versus an immersive 3D SG and a daily life activity in an immersive 3D VR environment versus an AR for the assessment of cognitive functions, comparing the efficacy and effectiveness of them. The first 2D study involved 354 healthy subjects and correlations were found between the game and traditional methods, suggesting that the game could be a valid tool for assessing cognitive functions in adults. The second study, compared 2D versus a 3D ATSNA version, it involved 94 healthy subjects and showed that 3D version was able to generate lower times and higher correct answers that the 2D, suggesting initial evidence of efficacy of a more immersive system compared to a non-immersive system. Although this result highlights a potential limitation on using different technological systems due to the differences on the two interaction methods (the 2D system applied mouse and keyboard; the 3D two virtual hands' controllers) and hardware and software latency data recording. Regarding individual variability on age, gender, and education, the findings showed consistency with the reference literature. Specifically, younger showed higher performance that older; higher educational levels reflected on a better score and about gender, results showed a more composite panorama. Furthermore, to enhance the ecological validity of assessment, the last study of this thesis compared the behavioural performance and physiological responses, during an ecological cooking task, between a virtual and an augmented system on 50 healthy subjects. The cooking task consisted of 4 levels that increased in difficulty. As the level increased, additional activities appeared. The behavioural results showed that times are always lower in VR than in AR, increasing constantly in accordance with the difficulty of the tasks. Regarding physiological responses, the findings showed that AR condition produced more individual excitement and activation than VR. To conclude, ATSNA are proving to be reliable and effective tools for the assessment of cognitive functions in adults, providing more ecological validity and objectivity than traditional methods of assessment.Chicchi Giglioli, IAM. (2020). Novel supports to the assessment of cognitive functions through the combined use of technologies and subjective and objective measurements [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139075TESISCompendi

A Review of Augmented Reality Applications for History Education and Heritage Visualisation

Augmented reality is a field with a versatile range of applications used in many fields including recreation and education. Continually developing technology spanning the last decade has drastically improved the viability for augmented reality projects now that most of the population possesses a mobile device capable of supporting the graphic rendering systems required for them. Education in particular has benefited from these technological advances as there are now many fields of research branching into how augmented reality can be used in schools. For the purposes of Holocaust education however, there has been remarkable little research into how Augmented Reality can be used to enhance its delivery or impact. The purpose of this study is to speculate regarding the following questions: How is augmented reality currently being used to enhance history education? Does the usage of augmented reality assist in developing long-term memories? Is augmented reality capable of conveying the emotional weight of historical events? Will augmented reality be appropriate for teaching a complex field such as the Holocaust? To address these, multiple studies have been analysed for their research methodologies and how their findings may assist with the development of Holocaust education

Tools in and out of sight : an analysis informed by Cultural-Historical Activity Theory of audio-haptic activities involving people with visual impairments supported by technology

The main purpose of this thesis is to present a Cultural-Historical Activity Theory (CHAT) based analysis of the activities conducted by and with visually impaired users supported by audio-haptic technology.This thesis covers several studies conducted in two projects. The studies evaluate the use of audio-haptic technologies to support and/or mediate the activities of people with visual impairment. The focus is on the activities involving access to two-dimensional information, such as pictures or maps. People with visual impairments can use commercially available solutions to explore static information (raised lined maps and pictures, for example). Solu-tions for dynamic access, such as drawing a picture or using a map while moving around, are more scarce. Two distinct projects were initiated to remedy the scarcity of dynamic access solutions, specifically focusing on two separate activities.The first project, HaptiMap, focused on pedestrian outdoors navigation through audio feedback and gestures mediated by a GPS equipped mobile phone. The second project, HIPP, focused on drawing and learning about 2D representations in a school setting with the help of haptic and audio feedback. In both cases, visual feedback was also present in the technology, enabling people with vision to take advantage of that modality too.The research questions addressed are: How can audio and haptic interaction mediate activ-ities for people with visual impairment? Are there features of the programming that help or hinder this mediation? How can CHAT, and specifically the Activity Checklist, be used to shape the design process, when designing audio haptic technology together with persons with visual impairments?Results show the usefulness of the Activity Checklist as a tool in the design process, and provide practical application examples. A general conclusion emphasises the importance of modularity, standards, and libre software in rehabilitation technology to support the development of the activities over time and to let the code evolve with them, as a lifelong iterative development process. The research also provides specific design recommendations for the design of the type of audio haptic systems involved

Ultrasonic mid-air haptic technology in context of science communication

This dissertation charts opportunities and challenges of engaging people with learning about science through the use of mid-air haptic technology. To that end, research has been carried out at the intersection of three multidisciplinary fields: Science Communication, Haptics, and Human-Computer Interaction. For science communication to be effective, different tools are required for different audiences. For example, when a child pours cold milk into hot tea and sips a warm beverage, we can raise awareness of the haptic experience; triggering interest and facilitating learning about thermal equilibrium. Not every scientific concept may be explained through changing temperature, and not everybody likes tea, but the principle of haptic experience facilitated public engagement with science remains a valid basis to examine. Science communicators seek new technological solutions and innovative modalities of communication, some of which include haptic technology and touch interaction. Ultrasonic mid-air haptic technology is a novel tool, which enables the creation of programable, invisible, cutaneous tactile sensations on an airborne interface between humans and the digital world. Mid-air haptic sensations may bring many benefits when used in science communication, but these have not yet been systematically studied