Collaborative educational environments incorporating mixed reality technologies: a systematic mapping study

In this paper, we report findings from a systematic mapping study, conducted to review the existing literature on collaborative educational environments incorporating mixed reality technologies. There is increasing interest in mixed reality technologies in education, especially with the introduction of new Over Head Mounted Displays (OHMDs), such as HoloLens, Oculus Rift and HTC Vive. with the consideration of areas such as education, dynamic technology and complex environments, a research area is identified. We carried out an extensive review of the literature from 2007 to 2017 and conducted an analysis of the works on mixed reality technologies and its subcategories applied to collaborative education environments. Results highlighted the lack of research across the mixed reality spectrum, especially in the augmented virtuality subcategory, as well as technical limitations such as response time in the development of mixed reality technologies for collaborative environments. Furthermore, the difficulty of teaching professionals to replicate mixed reality experiments in real environments, due to the technical skills required, was identified. The main contribution of this article is the discussion of the current works with visualization of the present state of the area, which is aimed to encourage educators to develop mixed reality artefacts and conduct further research to support collaborative educational environments

The Augmented Learner : The pivotal role of multimedia enhanced learning within a foresight-based learning model designed to accelerate the delivery of higher levels of learner creativity

The central theme for this dissertation lies at the intersection of multisensory technology enhanced learning, the field of foresight and transformative pedagogy and their role in helping to develop greater learner creativity. These skills will be key to meeting the needs of the projected growing role of the creative class within the emerging global workforce structure and the projected growth in R&D and the advancement of human-machine resource management. Over the past two decades, we have traversed from the Industrial Age through the Information Age into what we now call postnormal times, manifested partly in Industry 4.0. It is widely considered that the present education system in countries with developed economies is not optimised for delivering the much-needed creative skills, which are prominent amongst the critical 21st C skills required by the creative class, (also known as creatives), which will be increasingly dominant in terms of near future employability. Consequently, there will be a potential shortfall of creatives unless this issue is rapidly addressed. To ensure that the creative skills I aimed to enhance were relevant and aligned with emerging demands of the changing landscape, I deconstructed the critical dimensions, context, and concept of creativity in postnormal times as well as undertaking in-depth research on the potential future workscape and the future of education and learning, applying a comprehensive foresight approach to the latter using a 2030-2040 horizon. Based upon the outcomes of these studies I designed an experimental integrative learning system that I have applied, researched, and evolved over the past 4 years with over 150 students at PhD and master’s level. The system is aimed at generating higher levels of creative engagement and development through a focus on increased immersion and creativity-inducing approaches. The system, which I call the Living Learning System, is based upon eight integrated elements, supported by course development pillars aimed at optimizing learner future skill competencies and levels of creativity for which I apply severalevaluation techniques and metrics. Accordingly, as the central hypothesis of this dissertation, I argue that by integrating the critical elements of the Living Learning System, such as emerging multisensory technology enhanced learning coupled with optimised transformative and experiential learning approaches, framed within the field of foresight, with its futures focus and decentralised thinking approaches, students increase their ability to be creative. This increased ability is based on the student attaining a richer level of personal ambience through deeper immersion generated through higher incidence of self-direction, constructivism-based blended pedagogy, futures literacy, and a balance of decentralised and systems-based thinking, as well as cognitive and social platforms aimed at optimizing learner creative achievement. This dissertation demonstrates how the application of the combined elements of the Living Learning System, with its futures focus and its ensuing transdisciplinary curricula and courses, can provide a clear path towards significantly increased learner creativity. The findings of the quantitative, questionnaire-based research set out in detail in Chapter 9, together with the performance and creativity evaluation models applied against the selected case studies of student projects substantiate the validity of the hypothesis that the application of the Living Learning System with its futures focus leads to increased creativity in line with the needs of the postnormal era.publishedVersio

Designing and evaluating avatar biosignal visualization techniques in social virtual reality

Social VR is the application of virtual reality that supports remote social interaction in virtual spaces. Users communicate and interact with others in the social VR environment through avatars, which are virtual anthropomorphic characters that aim to represent humans in virtual worlds. In addition, the development of the HMD and commercially available motion capture systems enable the avatars in the virtual environment to detect and reflect the real-time motions, even facial expressions of people. However, the avatars still lack an indication of biofeedback - e.g., body temperature, breathing, heart rate, muscle contraction -, which serves as social cues for communication in reality. While some features, for example, emojis, supports users to express their feeling or emotions for richer communication, the missing information often results in miscommunication in the virtual space. It remains a barrier to a fully immersed experience in the social VR space. This project proposes a concept of visualizing biosignals of the avatars in the social virtual reality space for a richer-level interaction in virtual reality. With the technologies available to capture and reflect accurate biofeedback in real-time, we would like to explore ways and possibilities to map the bio states of the users in reality to avatars in the virtual world. The project starts with conducting user researches to understand the current user behaviors in the social VR spaces and their perspectives on sharing biosignals. Based on the requirements gathered from the user study, the scope of the project is narrowed down to a ‘watching entertainment’ scenario, and the ways to visualize biosignals on avatars were explored through a co-design session with designers. After that, four biosignal visualization techniques in two biosignals - heart rate and breathing rate - are prototyped under the VR jazz bar setting. Finally, the user study is conducted with 16 pairs (32 participants in total) to test and compare the effects of each biosignal visualization technique in watching entertainment scenarios with a companion. As a result, the embodied visualizations are the most understandable and least distracting visualization method among the four methods. Furthermore, the limitations of the research, recommendations on biosignal visualizations, and recommendations on conducting design research are provided

Personality, presence, and the virtual self: A five-factor model approach to behavioral analysis within a virtual environment

For several decades, researchers have explored the existence of the virtual self, or digital embodiment of self found within an avatar. It was surmised that this new component of one’s overall identity not only existed in conjunction with the public and private persona, but was replete with the necessary physical and psychological characteristics that facilitate a broad range of cognitive, cultural, and socio-emotional outcomes found within a virtual environment (e.g., Second Life, World of Warcraft). However, little is known with regard to whether these characteristics do indeed impact behavioral outcomes. For this reason, this study employed an observational assessment method to explore the virtual self as more than a set of characteristics attributed to an avatar, but rather as a relationship between personality (i.e., individual and avatar) and actualized behavior exhibited within a virtual environment. Further, presence measures were introduced to better understand whether feelings of immersion impact this relationship. Results indicated a burgeoning virtual self, linking personality with behavior along the domain of agreeableness. In other words, behavior is not solely the product of the environment but also is influenced by participant predispositions. Findings also suggest that the construct presence may now need to incorporate variables that account for this virtual self. Implications for educators, instructional designers, and psychologists are discussed

Computational Commensality: from theories to computational models for social food preparation and consumption in HCI

Food and eating are inherently social activities taking place, for example, around the dining table at home, in restaurants, or in public spaces. Enjoying eating with others, often referred to as “commensality,” positively affects mealtime in terms of, among other factors, food intake, food choice, and food satisfaction. In this paper we discuss the concept of “Computational Commensality,” that is, technology which computationally addresses various social aspects of food and eating. In the past few years, Human-Computer Interaction started to address how interactive technologies can improve mealtimes. However, the main focus has been made so far on improving the individual's experience, rather than considering the inherently social nature of food consumption. In this survey, we first present research from the field of social psychology on the social relevance of Food- and Eating-related Activities (F&EA). Then, we review existing computational models and technologies that can contribute, in the near future, to achieving Computational Commensality. We also discuss the related research challenges and indicate future applications of such new technology that can potentially improve F&EA from the commensality perspective

Tangible user interfaces and social interaction in children with autism

Tangible User Interfaces (TUIs) offer the potential for new modes of social interaction for children with Autism Spectrum Conditions (ASC). Familiar objects that are embedded with digital technology may help children with autism understand the actions of others by providing feedback that is logical and predictable. Objects that move, playback sound or create sound – thus repeating programmed effects – offer an exciting way for children to investigate objects and their effects. This thesis presents three studies of children with autism interacting with objects augmented with digital technology. Study one looked at Topobo, a construction toy augmented with kinetic memory. Children played with Topobo in groups of three of either Typically Developing (TD) or ASC children. The children were given a construction task, and were also allowed to play with the construction sets with no task. Topobo in the task condition showed an overall significant effect for more onlooker, cooperative, parallel, and less solitary behaviour. For ASC children significantly less solitary and more parallel behaviour was recorded than other play states. In study two, an Augmented Knights Castle (AKC) playset was presented to children with ASC. The task condition was extended to allow children to configure the playset with sound. A significant effect in a small sample was found for configuration of the AKC, leading to less solitary behaviour, and more cooperative behaviour. Compared to non-digital play, the AKC showed reduction of solitary behaviour because of augmentation. Qualitative analysis showed further differences in learning phase, user content, behaviour oriented to other children, and system responsiveness. Tangible musical blocks (‘d-touch’) in study three focused on the task. TD and ASC children were presented with a guided/non-guided task in pairs, to isolate effects of augmentation. Significant effects were found for an increase in cooperative symbolic play in the guided condition, and more solitary functional play was found in the unguided condition. Qualitative analysis highlighted differences in understanding blocks and block representation, exploratory and expressive play, understanding of shared space and understanding of the system. These studies suggest that the structure of the task conducted with TUIs may be an important factor for children’s use. When the task is undefined, play tends to lose structure and the benefits of TUIs decline. Tangible technology needs to be used in an appropriately structured manner with close coupling (the distance between digital housing and digital effect), and works best when objects are presented in familiar form

Affective Umbrella – A Wearable System to Visualize Heart and Electrodermal Activity, towards Emotion Regulation through Somaesthetic Appreciation

In this paper, we introduce Affective Umbrella, a novel system to record, analyze and visualize physiological data in real time via an umbrella handle. We implement a biofeedback loop design in the system that triggers visualization changes to reflect and regulate emotions through somaesthetic appreciation. We report the methodology, processes, and results of data reliability and visual feedback impact on emotions. We evaluated the system using a real-life user study (n=21) in rainy weather at night. The statistical results demonstrate the potential of applying the visualization of biofeedback to regulate emotional arousal with a significantly higher (p=.0022) score, a lower (p=.0277) dominance than baseline from self-reported SAM Scale, and physiological arousal, which was shown to be significantly increased (p<.0001) with biofeedback in terms of pNN50 and a significant difference in terms of RMSSD. There was no significant difference in terms of emotional valence changes from SAM scale. Furthermore, we compared the difference between two biofeedback patterns (mirror and inversion). The mirror effect was with a significantly higher emotional arousal than the inversion effect (p=.0277) from SAM results and was with a significantly lower RMSSD performance than the inversion effect (p<.0001). This work demonstrates the potential for capturing physiological data using an umbrella handle and using this data to influence a user’s emotional state via lighting effects

Cross-Reality Re-Rendering: Manipulating between Digital and Physical Realities

The advent of personalized reality has arrived. Rapid development in AR/MR/VR enables users to augment or diminish their perception of the physical world. Robust tooling for digital interface modification enables users to change how their software operates. As digital realities become an increasingly-impactful aspect of human lives, we investigate the design of a system that enables users to manipulate the perception of both their physical realities and digital realities. Users can inspect their view history from either reality, and generate interventions that can be interoperably rendered cross-reality in real-time. Personalized interventions can be generated with mask, text, and model hooks. Collaboration between users scales the availability of interventions. We verify our implementation against our design requirements with cognitive walkthroughs, personas, and scalability tests.Comment: updated. arXiv admin note: text overlap with arXiv:2204.0373

THE USE OF GAMIFICATION TO ENRICH THE PARK EXPERIENCE FOR THE VISITORS: ISTANBUL ATATÜRK URBAN FOREST PARK CASE STUDY

The excessive population growth leading to urbanization and, subsequent to it, urban sprawl, increases the size and number of urban settlements. Consequently, to fulfill the land needed for this expansion, humans encroach the public open areas, including highly crucial urban green spaces. Generally, any kind of greenness within the city, known as urban green spaces, benefits the ecosystem and the inhabitants. Hence, the green infrastructure, universally, should be preserved. Urban parks, specifically urban forest parks, serving as recreational green public spaces, are beneficial for environmental issues and human’s mental and physical health, in other words, well-being, by enabling them to interact with nature. To appropriately preserve the urban forest parks, being aware of the motivational factors for the visitors have a key role. While the willingness factors vary amongst people, it is important to know whether different people can experience the same park diversely, which is almost attractive in terms of general park features. Hence, the study refers to the concept of sense of place, capable of turning spaces into the desired places, framing the users’ behavior and transmitting cultural meaning. Augmentation occurs using Information and Communication Technologies (ICTs) in urban environments. It can be as simple as telecommunication technologies or very complicated ones. With their everyday use in peoples’ lifestyles, they have penetrated humans’ real lives, fading away the border detaching the real world from the world of computer-mediated services. Nowadays, a holistic technological perspective dominates the urban areas rather than individual ICT projects, which can reshape people’s interaction with their surroundings. Hence, the ICT-enriched spaces result in augmented spaces that convey a sense of place, supporting meaning-making and behavior framing. With its recent definition, gamification, which is the process of making activities more game-like, is a sample of augmentation. While triggering motivation in the users, it can support inducing new behaviors or reshaping the existing ones. Currently, gamification is spread out to various domains, both in academia and amongst practitioners. However, while gamification may involve any type of content, it is called a serious game in the case of transmitting serious educational content besides being entertaining. They are able to engage the player with the education of a certain topic or enhancement of a specific skill. The recent covid-19 outbreak and consequent confinement periods affected people's everyday lifestyles. While suddenly, people lost the freedom of being outdoor, experiencing new places, and interacting with each other, the technology rescued them from such a phenomenon. Virtual environment served them as a mediator to be in the same place, interact, and experience their daily activities in a novel form. Additionally, gamification changed even the sense of their virtual places. This thesis aims to enrich the park visitors’ experience via entertainingly introducing the Atatürk urban forest park to them, accompanied with educational content, by means of a locative serious game. The game benefits from the public awareness reinforcement to serve urban green space preservation. It generates a sense of placeness in its in situ mobile version and desktop version, played remotely, to frame the players’ behavior for the sake of its aim. The thesis is divided into six chapters to achieve its intended aim: The first chapter, the introduction, describes the excessive population growth leading to urbanization and its subsequent, urban sprawl, which increases the size and number of urban settlements and decreases the area of highly crucial urban green spaces. Discussing the green infrastructure benefits, this part emphasizes the importance of its preservation. After a brief debate around the possible ways of this conservation, this chapter continues with the thesis's purpose, method, and content. The literature review part, located in the second and the third chapter, introduces the types, definitions, and the benefits of the urban green spaces, instanced as parks and urban forest parks. This chapter continues with a description of the park experience, followed by the concept of the sense of place. Augmentation and the use of ICTs for behavior framing are described, followed by the idea of gamification, serious games, related case studies, and software. Next, in the fourth chapter, the thesis continues with the research part. Firstly, it introduces the selected area, Atatürk Urban Forest Park, with its detailed fauna and flora information. Then, it proceeds with the game development part, introducing the operated 360° panoramic field photography with a smartphone and using 3dvista Virtual Tour PRO software as the game development tool to provide a locative serious game. For the evaluation of the developed game’s success, in the fifth chapter, the thesis conducts an online survey, including a Google form questionnaire, requested to be filled after the gameplay, focused on the remote version, inspired by the pandemic situation. Finally, in the last chapter, the thesis results are concluded, and the discussions around the future possibilities of the provided method and its capabilities are made to enlighten the field for further studies