From ‘hands up’ to ‘hands on’: harnessing the kinaesthetic potential of educational gaming

Traditional approaches to distance learning and the student learning journey have focused on closing the gap between the experience of off-campus students and their on-campus peers. While many initiatives have sought to embed a sense of community, create virtual learning environments and even build collaborative spaces for team-based assessment and presentations, they are limited by technological innovation in terms of the types of learning styles they support and develop. Mainstream gaming development – such as with the Xbox Kinect and Nintendo Wii – have a strong element of kinaesthetic learning from early attempts to simulate impact, recoil, velocity and other environmental factors to the more sophisticated movement-based games which create a sense of almost total immersion and allow untethered (in a technical sense) interaction with the games’ objects, characters and other players. Likewise, gamification of learning has become a critical focus for the engagement of learners and its commercialisation, especially through products such as the Wii Fit. As this technology matures, there are strong opportunities for universities to utilise gaming consoles to embed levels of kinaesthetic learning into the student experience – a learning style which has been largely neglected in the distance education sector. This paper will explore the potential impact of these technologies, to broadly imagine the possibilities for future innovation in higher education

A framework study on the use of immersive XR technologies in the cultural heritage domain

Most cultural promotion and dissemination are nowadays performed through the digitization of heritage sites and museums, a necessary requirement to meet the new needs of the public. Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) have the potential to improve the experience quality and educational effect of these sites by stimulating users’ senses in a more natural and vivid way. In this respect, head-mounted display (HMD) devices allow visitors to enhance the experience of cultural sites by digitizing information and integrating additional virtual cues about cultural artifacts, resulting in a more immersive experience that engages the visitor both physically and emotionally. This study contributes to the development and incorporation of AR, MR, and VR applications in the cultural heritage domain by providing an overview of relevant studies utilizing fully immersive systems, such as headsets and CAVE systems, emphasizing the advantages that they bring when compared to handheld devices. We propose a framework study to identify the key features of headset-based Extended Reality (XR) technologies used in the cultural heritage domain that boost immersion, sense of presence, and agency. Furthermore, we highlight core characteristics that favor the adoption of these systems over more traditional solutions (e.g., handheld devices), as well as unsolved issues that must be addressed to improve the guests’ experience and the appreciation of the cultural heritage. An extensive search of Google Scholar, Scopus, IEEE Xplore, ACM Digital Library, and Wiley Online Library databases was conducted, including papers published from January 2018 to September 2022. To improve review reporting, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Sixty-five papers met the inclusion criteria and were classified depending on the study’s purpose: education, entertainment, edutainment, touristic guidance systems, accessibility, visitor profiling, and management. Immersive cultural heritage systems allow visitors to feel completely immersed and present in the virtual environment, providing a stimulating and educational cultural experience that can improve the quality and learning purposes of cultural visits. Nonetheless, the analyzed studies revealed some limitations that must be faced to give a further impulse to the adoption of these technologies in the cultural heritage domain

The Perception/Action loop: A Study on the Bandwidth of Human Perception and on Natural Human Computer Interaction for Immersive Virtual Reality Applications

Virtual Reality (VR) is an innovating technology which, in the last decade, has had a widespread success, mainly thanks to the release of low cost devices, which have contributed to the diversification of its domains of application. In particular, the current work mainly focuses on the general mechanisms underling perception/action loop in VR, in order to improve the design and implementation of applications for training and simulation in immersive VR, especially in the context of Industry 4.0 and the medical field. On the one hand, we want to understand how humans gather and process all the information presented in a virtual environment, through the evaluation of the visual system bandwidth. On the other hand, since interface has to be a sort of transparent layer allowing trainees to accomplish a task without directing any cognitive effort on the interaction itself, we compare two state of the art solutions for selection and manipulation tasks, a touchful one, the HTC Vive controllers, and a touchless vision-based one, the Leap Motion. To this aim we have developed ad hoc frameworks and methodologies. The software frameworks consist in the creation of VR scenarios, where the experimenter can choose the modality of interaction and the headset to be used and set experimental parameters, guaranteeing experiments repeatability and controlled conditions. The methodology includes the evaluation of performance, user experience and preferences, considering both quantitative and qualitative metrics derived from the collection and the analysis of heterogeneous data, as physiological and inertial sensors measurements, timing and self-assessment questionnaires. In general, VR has been found to be a powerful tool able to simulate specific situations in a realistic and involving way, eliciting user\u2019s sense of presence, without causing severe cybersickness, at least when interaction is limited to the peripersonal and near-action space. Moreover, when designing a VR application, it is possible to manipulate its features in order to trigger or avoid triggering specific emotions and voluntarily create potentially stressful or relaxing situations. Considering the ability of trainees to perceive and process information presented in an immersive virtual environment, results show that, when people are given enough time to build a gist of the scene, they are able to recognize a change with 0.75 accuracy when up to 8 elements are in the scene. For interaction, instead, when selection and manipulation tasks do not require fine movements, controllers and Leap Motion ensure comparable performance; whereas, when tasks are complex, the first solution turns out to be more stable and efficient, also because visual and audio feedback, provided as a substitute of the haptic one, does not substantially contribute to improve performance in the touchless case

Virtual Heritage: new technologies for edutainment

Cultural heritage represents an enormous amount of information and knowledge. Accessing this treasure chest allows not only to discover the legacy of physical and intangible attributes of the past but also to provide a better understanding of the present. Museums and cultural institutions have to face the problem of providing access to and communicating these cultural contents to a wide and assorted audience, meeting the expectations and interests of the reference end-users and relying on the most appropriate tools available. Given the large amount of existing tangible and intangible heritage, artistic, historical and cultural contents, what can be done to preserve and properly disseminate their heritage significance? How can these items be disseminated in the proper way to the public, taking into account their enormous heterogeneity? Answering this question requires to deal as well with another aspect of the problem: the evolution of culture, literacy and society during the last decades of 20th century. To reflect such transformations, this period witnessed a shift in the museum’s focus from the aesthetic value of museum artifacts to the historical and artistic information they encompass, and a change into the museums’ role from a mere "container" of cultural objects to a "narrative space" able to explain, describe, and revive the historical material in order to attract and entertain visitors. These developments require creating novel exhibits, able to tell stories about the objects and enabling visitors to construct semantic meanings around them. The objective that museums presently pursue is reflected by the concept of Edutainment, Education + Entertainment. Nowadays, visitors are not satisfied with ‘learning something’, but would rather engage in an ‘experience of learning’, or ‘learning for fun’, being active actors and players in their own cultural experience. As a result, institutions are faced with several new problems, like the need to communicate with people from different age groups and different cultural backgrounds, the change in people attitude due to the massive and unexpected diffusion of technology into everyday life, the need to design the visit by a personal point of view, leading to a high level of customization that allows visitors to shape their path according to their characteristics and interests. In order to cope with these issues, I investigated several approaches. In particular, I focused on Virtual Learning Environments (VLE): real-time interactive virtual environments where visitors can experience a journey through time and space, being immersed into the original historical, cultural and artistic context of the work of arts on display. VLE can strongly help archivists and exhibit designers, allowing to create new interesting and captivating ways to present cultural materials. In this dissertation I will tackle many of the different dimensions related to the creation of a cultural virtual experience. During my research project, the entire pipeline involved into the development and deployment of VLE has been investigated. The approach followed was to analyze in details the main sub-problems to face, in order to better focus on specific issues. Therefore, I first analyzed different approaches to an effective recreation of the historical and cultural context of heritage contents, which is ultimately aimed at an effective transfer of knowledge to the end-users. In particular, I identified the enhancement of the users’ sense of presence in VLE as one of the main tools to reach this objective. Presence is generally expressed as the perception of 'being there', i.e. the subjective belief of users that they are in a certain place, even if they know that the experience is mediated by the computer. Presence is related to the number of senses involved by the VLE and to the quality of the sensorial stimuli. But in a cultural scenario, this is not sufficient as the cultural presence plays a relevant role. Cultural presence is not just a feeling of 'being there' but of being - not only physically, but also socially, culturally - 'there and then'. In other words, the VLE must be able to transfer not only the appearance, but also all the significance and characteristics of the context that makes it a place and both the environment and the context become tools capable of transferring the cultural significance of a historic place. The attention that users pay to the mediated environment is another aspect that contributes to presence. Attention is related to users’ focalization and concentration and to their interests. Thus, in order to improve the involvement and capture the attention of users, I investigated in my work the adoption of narratives and storytelling experiences, which can help people making sense of history and culture, and of gamification approaches, which explore the use of game thinking and game mechanics in cultural contexts, thus engaging users while disseminating cultural contents and, why not?, letting them have fun during this process. Another dimension related to the effectiveness of any VLE is also the quality of the user experience (UX). User interaction, with both the virtual environment and its digital contents, is one of the main elements affecting UX. With respect to this I focused on one of the most recent and promising approaches: the natural interaction, which is based on the idea that persons need to interact with technology in the same way they are used to interact with the real world in everyday life. Then, I focused on the problem of presenting, displaying and communicating contents. VLE represent an ideal presentation layer, being multiplatform hypermedia applications where users are free to interact with the virtual reconstructions by choosing their own visiting path. Cultural items, embedded into the environment, can be accessed by users according to their own curiosity and interests, with the support of narrative structures, which can guide them through the exploration of the virtual spaces, and conceptual maps, which help building meaningful connections between cultural items. Thus, VLE environments can even be seen as visual interfaces to DBs of cultural contents. Users can navigate the VE as if they were browsing the DB contents, exploiting both text-based queries and visual-based queries, provided by the re-contextualization of the objects into their original spaces, whose virtual exploration can provide new insights on specific elements and improve the awareness of relationships between objects in the database. Finally, I have explored the mobile dimension, which became absolutely relevant in the last period. Nowadays, off-the-shelf consumer devices as smartphones and tablets guarantees amazing computing capabilities, support for rich multimedia contents, geo-localization and high network bandwidth. Thus, mobile devices can support users in mobility and detect the user context, thus allowing to develop a plethora of location-based services, from way-finding to the contextualized communication of cultural contents, aimed at providing a meaningful exploration of exhibits and cultural or tourist sites according to visitors’ personal interest and curiosity

Design for Child-Robot Play The implications of Design Research within the field of Human-Robot Interaction studies for Children

This thesis investigates the intersections of three disciplines, that are Design Research, Human-Robot Interaction studies, and Child Studies. In particular, this doctoral research is focused on two research questions, namely, what is (or might be) the role of design research in HRI? And, how to design acceptable and desirable child-robot play applications? The first chapter introduces an overview of the mutual interest between robotics and design that is at the basis of the research. On the one hand, the interest of design toward robotics is documented through some exemplary projects from artists and designers that speculate on the human-robot coexistence condition. Vice versa, the robotics interest toward design is documented by referring to some tracks of robotic conferences, scienti c workshops and robotics journals which focused on the design-robotics relationship. Finally, a brief description of the background conditions that characterized this doctoral research are introduced, such as the fact of being a research founded by a company. The second chapter provides an overview of the state of the art of the intersections between three multidisciplinary disciplines. First, a de nition of Design Research is provided, together with its main trends and open issues. Then, the review focuses on the contribution of Design Research to the HRI eld, which can be summed up in actions focused on three aspects: artefacts, stakeholders, and contexts. This is followed by a focus on the role of Design Research within the context of children studies, in which it is possible to identify two main design-child relationships: design as a method for developing children’s learning experiences; and children as part of the design process for developing novel interactive systems. The third chapter introduces the Research through Design (RtD) approach and its relevance in conducting design research in HRI. The proposed methodology, based on this approach, is particularly characterized by the presence of design explorations as study methods. These, in turn, are developed through a common project’s methodology, also reported in this chapter. The fourth chapter is dedicated to the analysis of the scenario in which the child-robot interaction takes place. This was aimed at understanding what is edutainment robotics for children, its common features, how it relates to existing children play types, and where the interaction takes place. The chapter provides also a focus on the relationship between children and technology on a more general level, through which two themes and relative design opportunities were identi ed: physically active play and objects-to-think-with. These were respectively addressed in the two design explorations presented in this thesis: Phygital Play and Shybo. The Phygital Play project consists of an exploration of natural interaction modalities with robots, through mixed-reality, for fostering children’s active behaviours. To this end, a game platform was developed for allowing children to play with or against a robot, through body movement. Shybo, instead, is a low-anthropomorphic robot for playful learning activities with children that can be carried out in educational contexts. The robot, which reacts to properties of the physical environment, is designed to support different kinds of experiences. Then, the chapter eight is dedicated to the research outcomes, that were de ned through a process of reflection. The contribution of the research was analysed and documented by focusing on three main levels, namely: artefact, knowledge and theory. The artefact level corresponds to the situated implementations developed through the projects. The knowledge level consists of a set of actionable principles, emerged from the results and lessons learned from the projects. At the theory level, a theoretical framework was proposed with the aim of informing the future design of child- robot play applications. Thelastchapterprovidesa naloverviewofthe doctoral research, a series of limitations regarding the research, its process and its outcomes, and some indications for future research

Study and development of custom “serious games” for patients and users

Video games are part of our culture and permeate several segments of society, from casual entertainment to serious purposes. This research work focuses on the latter, more specifically on the use of Serious Games in the healthcare context. In this sense, Serious Games have been researched through clinical studies, implemented to improve several conditions and have already proved to be efficient and valid, especially due to their ability to captivate and motivate the patient. With this in mind, this research explored the use of Serious Games for physical rehabilitation of stroke impaired patients and how this approach could be implemented in the Portuguese healthcare system. The research started with a thorough literature review of the validity and effectiveness of the use of Serious Games for this type of patients. Then, through surveys and interviews with healthcare professionals and game developers, the main barriers to the implementation and development of Serious Games for physical rehabilitation were accessed. Posteriorly, the feasibility of implementing a business model for this specific niche (in Portugal) was evaluated through a market analysis, abridging social, economic, political and technological aspects that culminated in the design of several hypothetical scenarios that envisioned the possibility of making this approach work. This research allowed to achieve an understanding of what needs to be overcome in order for this technique to be successful: Serious Games are still widely unknown among healthcare professionals and there is an underlying stigma against video games that clouds the possible benefits of using Serious Games. Moreover, the social context of future users will also determine their acceptance and compliance. Nevertheless, the research has evidenced that the implementation of this paradigm is economically viable as it would indirectly allow the monetization of rehabilitation therapies by making them available to a larger slice of the disabled population, which is of utmost importance, mainly during the times we currently live in.Os videojogos fazem parte da nossa cultura e permeiam vários segmentos da sociedade, desde entretenimento casual até propósitos mais sérios. Este trabalho de investigação concentra-se na segunda opção – mais especificamente, no uso de Serious Games no contexto da saúde. Neste sentido, os Serious Games têm sido investigados através de estudos clínicos, implementados para melhorar diversas condições e patologias e já provaram ser eficientes e válidos, principalmente graças à sua capacidade de motivar e cativar o paciente. Tendo isto em conta, esta investigação explorou o uso dos Serious Games para reabilitação física de pacientes pós-AVC e de que forma é que esta abordagem poderia ser implementada no Serviço Nacional de Saúde. A investigação começou com uma revisão bibliográfica completa da validade e eficácia do uso de Serious Games para este tipo de pacientes. De seguida, através de questionários e entrevistas a profissionais de saúde e desenvolvedores de videojogos, as principais barreiras para implementação e desenvolvimento de Serious Games para reabilitação física foram levantadas. Posteriormente, a viabilidade de executar um business plan para esse nicho específico (em Portugal) foi avaliada através de uma análise de mercado, abrangendo aspetos sociais, económicos, políticos e tecnológicos, que culminou no desenho de vários cenários hipotéticos que previam a possibilidade de fazer com que este paradigma se tornasse possível e funcional. Este trabalho de investigação permitiu atingir uma compreensão daquilo que necessita de ser ultrapassado para que esta abordagem tenha sucesso: os Serious Games ainda são amplamente desconhecidos entre os profissionais de saúde e existe um estigma subjacente aos videojogos que obscurece os possíveis benefícios do seu uso. Para além disso, o contexto social dos futuros utilizadores também irá determinar a sua aceitação e adesão. No entanto, esta investigação evidenciou que a implementação deste modelo é economicamente viável, pois iria permitir, de uma forma indireta, a rentabilização de terapias de reabilitação ao fazer com que estas fossem acessíveis a uma fatia mais larga da população debilitada, o que é de extrema importância, principalmente nos dias de hoje

A quest for the holy grail: Tactile precision, natural movement and haptic feedback in 3D virtual spaces

Three-dimensional immersive spaces such as those provided by virtual worlds, give unparalleled opportunities for learners to practically engage with simulated authentic settings that may be too expensive or too dangerous to experience in the real world. The potential afforded by these environments is severely constrained by the use of a keyboard and mouse moving in two dimensions. While most technologies have evolved rapidly in the early 21st century, the mouse and keyboard as standard navigation and interaction tools have not. However, talented teams from a range of disciplines are on serious quests to address this limitation. Their Holy Grail is to develop ways to interact with 3D immersive spaces using more natural human movements with haptic feedback. Applications would include the training of surgeons and musical conductors, training elite sports people and even physical rehabilitation. This paper reports on the cutting-edge technology projects that look most likely to provide a solution for this complex problem, including the Wiimote and the Microsoft's Project Natal. © 2009 Helen Farley and Caroline Steel

The development and applications of serious games in the public services: defence and health

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London.The latest advances of Virtual Reality technologies and three-dimensional graphics, as well as the developments in Gaming Technologies in the recent years, have stemmed the proliferation of Serious Games in a broader spectrum of research applications. Among the most popular areas of application are public services such as Defence and Health, where digital technologies realise new challenges and opportunities for research and development of Serious Games and for a variety of contexts. As with all games, the user engagement is elevated and apart from the entertaining aspect, Serious Games serve as a novel and promising alternative experience to knowledge transfer. Furthermore, Serious Games bring to the end user and the overall society a series of attractive benefits. These benefits include safety, cost-effectiveness, increased motivation and personalisation. Hence, this Thesis aims to investigate novel approaches of developing Serious Games that utilise the recent advances of Virtual Reality and Gaming Technology and facilitate the aforementioned benefits. The process of design and development of the novel tools and applications follow an iterative manner and are driven by the review of the available literature as well as end-user feedbackEPSRC (Engineering and Physical Sciences Research Council ) , MOD (UK), NHS (UK

Tangibot: A tangible-mediated robot to support cognitive games for ageing people A usability study

[EN] The ever increasing elderly population requires a revision of technology to make it usable and meaningful for them. Most applications take into account their reduced physical and cognitive abilities in order to provide assistive services, but this paper focuses on building technology to improve these capacities through cognitive games. We present Tangibot, a tangible-mediated robot aimed at enabling more intuitive and appealing interactions. A usability study conducted on subjects at three different levels of cognitive impairment (none, mild, and severe) reveals that it is usable and engaging for users with no or mild cognitive impairment, and even though it is less usable for persons with severe impairment, it triggers positive emotional reactions among them, which makes it promising for their use in therapeutic activities.This work is supported by Spanish Ministry of Economy and Competitiveness and funded by the European Development Regional Fund (EDRF-FEDER) with Project TIN2014-60077-R. It is also supported by fellowship ACIF/2014/214 within the VALi+d program from Conselleria d'Educacio, Cultura i Esport (Generalitat Valenciana), and by fellowship FPU14/00136 within the FPU program from Spanish Ministry of Education, Culture, and Sport.García Sanjuan, F.; Jaén Martínez, FJ.; Nácher-Soler, VE. (2017). Tangibot: A tangible-mediated robot to support cognitive games for ageing people A usability study. Pervasive and Mobile Computing. 34:91-105. doi:10.1016/j.pmcj.2016.08.007S911053

ISMCR 1994: Topical Workshop on Virtual Reality. Proceedings of the Fourth International Symposium on Measurement and Control in Robotics

This symposium on measurement and control in robotics included sessions on: (1) rendering, including tactile perception and applied virtual reality; (2) applications in simulated medical procedures and telerobotics; (3) tracking sensors in a virtual environment; (4) displays for virtual reality applications; (5) sensory feedback including a virtual environment application with partial gravity simulation; and (6) applications in education, entertainment, technical writing, and animation