Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Toward emotional interactive videogames for children with autism spectrum disorder

Technology and videogames have been proven as motivating tools for working attention and complex communication skills, especially in children with autism spectrum disorder (ASD). In this work, we present two experiences that used interactive games for promoting communication and attention. The first game considers emotions in order to measure children’s attention, concentration and satisfaction, while the second uses tangible tabletops for fostering cognitive planning. The analysis of the results obtained allows to propose a new study integrating both, in which the tangible interactive game is complemented with the emotional trainer in a way that allows identifying and classifying children’s emotion with ASD when they collaborate to solve cognitively significant and contextualized challenges. The first application proposed is an emotional trainer application in which the child can work out the seven basic emotions (happiness, sadness, fear, disgust, anger, surprise and neutral). Further, a serious videogame is proposed: a 3D maze where the emotions can be captured. The second case study was carried out in a Special Education Center, where a set of activities for working cognitive planning was proposed. In this case, a tangible interactive tabletop was used to analyze, in students with ASD, how the communication processes with these interfaces affect to the attention, memory, successive and simultaneous processing that compose cognitive planning from the PASS model. The results of the first study, suggest that the autistic children did not act with previous planning, but they used their perception to adjust their actions a posteriori (that explains the higher number of collisions). On the second case study, the successive processing was not explored. The inclusion of the mazes of case study 1 to a semantic rich scenario could allow us to measure the prior planning and the emotions involved in the maze game. The new physiological sensors will also help to validate the emotions felt by the children. The first study has as objective the capability to imitate emotions and resolve a maze without semantic context. The second study organized all the actions from a semantic context close to users. The attention results presented by the second study are coherent with the first study and complement it showing that attention can be receptive or selective. In the first study case, the receptive attention was the focus of analysis. In the second case, both contributed to explain and understand how it can be developed from a videogame

The Augmented Learner : The pivotal role of multimedia enhanced learning within a foresight-based learning model designed to accelerate the delivery of higher levels of learner creativity

The central theme for this dissertation lies at the intersection of multisensory technology enhanced learning, the field of foresight and transformative pedagogy and their role in helping to develop greater learner creativity. These skills will be key to meeting the needs of the projected growing role of the creative class within the emerging global workforce structure and the projected growth in R&D and the advancement of human-machine resource management. Over the past two decades, we have traversed from the Industrial Age through the Information Age into what we now call postnormal times, manifested partly in Industry 4.0. It is widely considered that the present education system in countries with developed economies is not optimised for delivering the much-needed creative skills, which are prominent amongst the critical 21st C skills required by the creative class, (also known as creatives), which will be increasingly dominant in terms of near future employability. Consequently, there will be a potential shortfall of creatives unless this issue is rapidly addressed. To ensure that the creative skills I aimed to enhance were relevant and aligned with emerging demands of the changing landscape, I deconstructed the critical dimensions, context, and concept of creativity in postnormal times as well as undertaking in-depth research on the potential future workscape and the future of education and learning, applying a comprehensive foresight approach to the latter using a 2030-2040 horizon. Based upon the outcomes of these studies I designed an experimental integrative learning system that I have applied, researched, and evolved over the past 4 years with over 150 students at PhD and master’s level. The system is aimed at generating higher levels of creative engagement and development through a focus on increased immersion and creativity-inducing approaches. The system, which I call the Living Learning System, is based upon eight integrated elements, supported by course development pillars aimed at optimizing learner future skill competencies and levels of creativity for which I apply severalevaluation techniques and metrics. Accordingly, as the central hypothesis of this dissertation, I argue that by integrating the critical elements of the Living Learning System, such as emerging multisensory technology enhanced learning coupled with optimised transformative and experiential learning approaches, framed within the field of foresight, with its futures focus and decentralised thinking approaches, students increase their ability to be creative. This increased ability is based on the student attaining a richer level of personal ambience through deeper immersion generated through higher incidence of self-direction, constructivism-based blended pedagogy, futures literacy, and a balance of decentralised and systems-based thinking, as well as cognitive and social platforms aimed at optimizing learner creative achievement. This dissertation demonstrates how the application of the combined elements of the Living Learning System, with its futures focus and its ensuing transdisciplinary curricula and courses, can provide a clear path towards significantly increased learner creativity. The findings of the quantitative, questionnaire-based research set out in detail in Chapter 9, together with the performance and creativity evaluation models applied against the selected case studies of student projects substantiate the validity of the hypothesis that the application of the Living Learning System with its futures focus leads to increased creativity in line with the needs of the postnormal era.publishedVersio

Hybridizing 3-dimensional multiple object tracking with neurofeedback to enhance preparation, performance, and learning

Le vaste domaine de l’amélioration cognitive traverse les applications comportementales, biochimiques et physiques. Aussi nombreuses sont les techniques que les limites de ces premières : des études de pauvre méthodologie, des pratiques éthiquement ambiguës, de faibles effets positifs, des effets secondaires significatifs, des couts financiers importants, un investissement de temps significatif, une accessibilité inégale, et encore un manque de transfert. L’objectif de cette thèse est de proposer une méthode novatrice d’intégration de l’une de ces techniques, le neurofeedback, directement dans un paradigme d’apprentissage afin d’améliorer la performance cognitive et l’apprentissage. Cette thèse propose les modalités, les fondements empiriques et des données à l’appui de ce paradigme efficace d’apprentissage ‘bouclé’. En manipulant la difficulté dans une tâche en fonction de l’activité cérébrale en temps réel, il est démontré que dans un paradigme d’apprentissage traditionnel (3-dimentional multiple object tracking), la vitesse et le degré d’apprentissage peuvent être améliorés de manière significative lorsque comparés au paradigme traditionnel ou encore à un groupe de contrôle actif. La performance améliorée demeure observée même avec un retrait du signal de rétroaction, ce qui suggère que les effets de l’entrainement amélioré sont consolidés et ne dépendent pas d’une rétroaction continue. Ensuite, cette thèse révèle comment de tels effets se produisent, en examinant les corrélés neuronaux des états de préparation et de performance à travers les conditions d’état de base et pendant la tâche, de plus qu’en fonction du résultat (réussite/échec) et de la difficulté (basse/moyenne/haute vitesse). La préparation, la performance et la charge cognitive sont mesurées via des liens robustement établis dans un contexte d’activité cérébrale fonctionnelle mesurée par l’électroencéphalographie quantitative. Il est démontré que l’ajout d’une assistance- à-la-tâche apportée par la fréquence alpha dominante est non seulement appropriée aux conditions de ce paradigme, mais influence la charge cognitive afin de favoriser un maintien du sujet dans sa zone de développement proximale, ce qui facilite l’apprentissage et améliore la performance. Ce type de paradigme d’apprentissage peut contribuer à surmonter, au minimum, un des limites fondamentales du neurofeedback et des autres techniques d’amélioration cognitive : le manque de transfert, en utilisant une méthode pouvant être intégrée directement dans le contexte dans lequel l’amélioration de la performance est souhaitée.The domain of cognitive enhancement is vast, spanning behavioral, biochemical and physical applications. The techniques are as numerous as are the limitations: poorly conducted studies, ethically ambiguous practices, limited positive effects, significant side-effects, high financial costs, significant time investment, unequal accessibility, and lack of transfer. The purpose of this thesis is to propose a novel way of integrating one of these techniques, neurofeedback, directly into a learning context in order to enhance cognitive performance and learning. This thesis provides the framework, empirical foundations, and supporting evidence for a highly efficient ‘closed-loop’ learning paradigm. By manipulating task difficulty based on a measure of cognitive load within a classic learning scenario (3-dimentional multiple object tracking) using real-time brain activity, results demonstrate that over 10 sessions, speed and degree of learning can be substantially improved compared with a classic learning system or an active sham-control group. Superior performance persists even once the feedback signal is removed, which suggests that the effects of enhanced training are consolidated and do not rely on continued feedback. Next, this thesis examines how these effects occur, exploring the neural correlates of the states of preparedness and performance across baseline and task conditions, further examining correlates related to trial results (correct/incorrect) and task difficulty (slow/medium/fast speeds). Cognitive preparedness, performance and load are measured using well-established relationships between real-time quantified brain activity as measured by quantitative electroencephalography. It is shown that the addition of neurofeedback-based task assistance based on peak alpha frequency is appropriate to task conditions and manages to influence cognitive load, keeping the subject in the zone of proximal development more often, facilitating learning and improving performance. This type of learning paradigm could contribute to overcoming at least one of the fundamental limitations of neurofeedback and other cognitive enhancement techniques : a lack of observable transfer effects, by utilizing a method that can be directly integrated into the context in which improved performance is sought

Computer-based cognitive rehabilitation: the CoRe system

This work aims at providing a tool for supporting cognitive rehabilitation. This is a wide field, that includes a variety of diseases and related clinical pictures; for this reason the need arises to have a tool available that overcomes the difficulties entailed by what currently is the most common approach, that is, the so-called pen and paper rehabilitation. Methods: We first organized a big number of stimuli in an ontology that represents concepts, attributes and a set of relationships among concepts. Stimuli may be words, sounds, 2D and 3D images. Then, we developed an engine that automatically generates exercises by exploiting that ontology. The design of exercises has been carried on in synergy with neuropsychologists and speech therapists. Solutions have been devised aimed at personalizing the exercises according to both patients’ preferences and performance. Results: Exercises addressed to rehabilitation of executive functions and aphasia-related diseases have been implemented. The system has been tested on both healthy volunteers (n 1/4 38) and patients (n 1/4 9), obtaining a favourable rating and suggestions for improvements. Conclusions: We created a tool able to automate the execution of cognitive rehabilitation tasks. We hope the variety and personalization of exercises will allow to increase compliance, particularly from elderly people, usually neither familiar with technology nor particularly willing to rely on it. The next step involves the creation of a telerehabilitation tool, to allow therapy sessions to be undergone from home, thus guaranteeing continuity of care and advantages in terms of time and costs for the patients and the National Healthcare System (NHS).Postprint (published version

Collaborative design methodologies and social dynamics: a portuguese social and public health case study

This research project addresses the understanding of collaborative action among disciplines from distinct branches of science with particular focus on social action. It also aims to evaluate the impact of this collaborative action on responding to the needs of different highly vulnerable communities and on other participants of these same processes. The integration of design in the social sector is a growing tendency, albeit to a certain extent characterised by novelty and resistance. Hence, and within this framework, the present thesis seeks to respond to the hypothesis “Collaborative design methodologies improve the effectiveness of social services’ practice”, and is organized into six phases, namely: the theoretical framework and the main research question, the construction of the model structure of the research and identification of the hypothesis, a review of the collaborative models and Social Design framework through case studies, the application of an in-depth case, the collection, processing and analysis of data and conclusions. The aim of this research is to ascertain the influence of Design on collaborative action, namely in the generation of more effective results and in obtaining information, and its impact on other sectors (intervention, assistance, solidarity, community action and social development). Moreover, it has served to highlight the potential for future alliances with prospective partners who, in light of the developed process, have manifested an interest in establishing co-partnerships of this nature. This study sheds light upon the potential of integrating Design and Designers in organizations in general, and in particular in those that aim to respond to the needs of communities who are most exposed to social impacts. By increasing the capacity to respond to their vulnerabilities, the risk of social exclusion and isolation will be significantly reduced. Additionally, awareness on the part of social partners, of the added value of Design and its contribution to the maturity and sustainability of social processes will be enhanced

Individuality and the collective in AI agents: Explorations of shared consciousness and digital homunculi in the metaverse for cultural heritage

The confluence of extended reality (XR) technologies, including augmented and virtual reality, with large language models (LLM) marks a significant advancement in the field of digital humanities, opening uncharted avenues for the representation of cultural heritage within the burgeoning metaverse. This paper undertakes an examination of the potentialities and intricacies of such a convergence, focusing particularly on the creation of digital homunculi or changelings. These virtual beings, remarkable for their sentience and individuality, are also part of a collective consciousness, a notion explored through a thematic comparison in science fiction with the Borg and the Changelings in the Star Trek universe. Such a comparison offers a metaphorical framework for discussing complex phenomena such as shared consciousness and individuality, illuminating their bearing on perceptions of self and awareness. Further, the paper considers the ethical implications of these concepts, including potential loss of individuality and the challenges inherent to accurate representation of historical figures and cultures. The latter necessitates collaboration with cultural experts, underscoring the intersectionality of technological innovation and cultural sensitivity. Ultimately, this chapter contributes to a deeper understanding of the technical aspects of integrating large language models with immersive technologies and situates these developments within a nuanced cultural and ethical discourse. By offering a comprehensive overview and proposing clear recommendations, the paper lays the groundwork for future research and development in the application of these technologies within the unique context of cultural heritage representation in the metaverse