Closing the loop in exergaming - Health benefits of biocybernetic adaptation in senior adults

Exergames help senior players to get physically active by promoting fun and enjoyment while exercising. However, most exergames are not designed to produce recommended levels of exercise that elicit adequate physical responses for optimal training in the aged population. In this project, we developed physiological computing technologies to overcome this issue by making real-time adaptations in a custom exergame based on recommendations for targeted heart rate (HR) levels. This biocybernetic adaptation was evaluated against conventional cardiorespiratory training in a group of active senior adults through a floor-projected exergame and a smartwatch to record HR data. Results showed that the physiologically-augmented exergame leads players to exert around 40% more time in the recommended HR levels, compared to the conventional training, avoiding over exercising and maintaining good enjoyment levels. Finally, we made available our biocybernetic adaptation software tool to enable the creation of physiological adaptive videogames, permitting the replication of our study.info:eu-repo/semantics/publishedVersio

Exploring the dynamics of the biocybernetic loop in physiological computing

Physiological computing is a highly multidisciplinary emerging field in which the spread of results across several application areas and disciplines creates a challenge of combining the lessons learned from various studies. The thesis comprises diverse publications that together create a privileged position for contributing to a common understanding of the roles and uses of physiological computing systems, generalizability of results across application areas, the theoretical grounding of the field (as with the various ways the psychophysiological states of the user can be modeled), and the emerging data analysis approaches from the domain of machine learning. The core of physiological computing systems has been built around the concept of biocybernetic loop, aimed at providing real-time adaptation to the cognitions, motivations, and emotions of the user. However, the traditional concept of the biocybernetic loop has been both self-regulatory and immediate; that is, the system adapts to the user immediately. The thesis presents an argument that this is too narrow a view of physiological computing, and it explores scenarios wherein the physiological signals are used not only to adapt to the user but to aid system developers in designing better systems, as well as to aid other users of the system. The thesis includes eight case studies designed to answer three research questions: 1) what are the various dynamics the biocybernetic loop can display, 2) how do the changes in loop dynamics affect the way the user is represented and modeled, and 3) how do the choices of loop dynamics and user representations affect the selection of machine learning methods and approaches? To answer these questions, an analytical model for physiological computing is presented that divides each of the physiological computing systems into five separate layers. The thesis presents three main findings corresponding to the three research questions: Firstly, the case studies show that physiological computing extends beyond the simple real-time self-regulatory loop. Secondly, the selected user representations seem to correlate with the type of loop dynamics. Finally, the case studies show that the machine learning approaches are implemented at the level of feature generation and are used when the loop diverges from the traditional real-time and self-regulatory dynamics into systems where the adaptation happens in the future.Perinteinen ihmisen ja tietokoneen vuorovaikutus on hyvin epäsymmetristä: tietokone voi esittää ihmiselle monimutkaista audiovisuaalista informaatiota kun taas ihmisen kommunikaatio koneen suuntaan on rajattu näppäimistöön ja hiireen. Samoin, vaikka ihmisellä on mahdollisuus saada informaatiota tietokoneen sisäisestä tilasta, kuten muistin ja prosessorin käyttöasteesta, ei tietokoneella ole vastaavaa mahdollisuutta tutkia ihmisen sisäisiä tiloja kuten tunteita. Mittaamalla reaaliajassa ihmisen fysiologisia signaaleja nämä molemmat ongelmat voidaan ratkaista: näppäimistön ja hiiren lisäksi tietokone saa suuren määrän informaatiota ihmisen kognitiivisista ja affektiivisista tiloista. Esimerkiksi mittaamalla ihmisen sykettä tai ihon sähkönjohtavuutta voi tietokone päätellä onko käyttäjä juuri nyt kiihtynyt tai rentoutunut. Tällaista fysiologisten signaalien reaaliaikaista hyödyntämistä ihmisen ja koneen vuorovaikutuksessa on tutkittu onnistuneesti monessa eri yhteyksissä: autonkuljettajien väsymystä voidaan mitata ja tarvittaessa varoittaa ajajaa, tietokonepelaajia mittaamalla on mahdollista säätää pelin vaikeustasoa sopivaksi ja älykello voi reagoida käyttäjän stressiin ehdottamalla rentoutumisharjoitusta. Näitä tapauksia yhdistää se, että käyttäjän fysiologisia signaaleja käytetään reaaliajassa sopeuttamaan järjestelmä käyttäjän itsensä tarpeisiin. Tällaista järjestelmän sopeuttamista reaaliajassa käyttäjän fysiologisten signaalien perusteella kutsutaan “biokyberneettiseksi silmukaksi” (biocybernetic loop). Biokyberneettisen silmukka on perinteisesti määritelty systeemin sopeuttamiseen yksittäisen käyttäjän sen hetkisen fysiologisen vasteen mukaan. Väitöskirjan tarkoitus on tutkia kuinka biokyberneettisen silmukan dynamiikkaa voidaan laajentaa sekä tilassa (voiko silmukka käsittää useita käyttäjiä) ja ajassa (voiko silmukan idea toimia myös ei-reaaliajassa). Erityisesti keskitytään tutkimaan kuinka muutokset silmukan dynamiikassa vaikuttavat silmukan toteutuksen yksityiskohtiin: kannattaako käyttäjää mallintaa eri tavoin ja ovatko tietyn tyyppiset silmukat soveltuvampia koneoppimiseen verrattuna ns. käsintehtyyn ratkaisuun. Väitöskirja sisältää kahdeksan käyttäjätutkimusta, jotka peilaavat biokyberneettisen silmukan käyttäytymistä erilaisissa konteksteissa. Tutkimukset osoittavat, että biokyberneettistä silmukkaa voidaan käyttää myös osana järjestelmän suunnittelua kun fysiologisten mittausten tulokset ohjataan järjestelmän kehittelijöille, ja järjestelmän muiden käyttäjien auttamiseen suosittelujärjestelmissä, joissa käyttäjän antamaa implisiittistä palautetta käytetään hyväksi suositeltaessa tuotteita toisille käyttäjille

Emotion recognition techniques using physiological signals and video games –Systematic review–

Emotion recognition systems from physiological signals are innovative techniques that allow studying the behavior and reaction of an individual when exposed to information that may evoke emotional reactions through multimedia tools, for example, video games. This type of approach is used to identify the behavior of an individual in different fields, such as medicine, education, psychology, etc., in order to assess the effect that the content has on the individual that is interacting with it. This article shows a systematic review of articles that report studies on emotion recognition with physiological signals and video games, between January 2010 and April 2016. We searched in eight databases, and found 15 articles that met the selection criteria. With this systematic review, we found that the use of video games as emotion stimulation tools has become an innovative field of study, due to their potential to involve stories and multimedia tools that can interact directly with the person in fields like rehabilitation. We detected clear examples where video games and physiological signal measurement became an important approach in rehabilitation processes, for example, in Posttraumatic Stress Disorder (PTSD) treatments

Exploring the Biocybernetic loop: Classifying Psychophysiological Responses to Cultural Artefacts using Physiological Computing

The aim of this research project was to provide a bio-sensing component for a real-time adaptive technology in the context of cultural heritage. The proposed system was designed to infer the interest or intention of the user and to augment elements of the cultural heritage experience interactively through implicit interaction. Implicit interaction in this context is the process whereby the system observes the user while they interact with artefacts; recording psychophysiological responses to cultural heritage artefacts or materials and acting upon these responses to drive adaptations in content in real-time.Real-time biocybernetic control is the central component of physiological computing wherein physiological data are converted into a control input for a technological system. At its core the bio-sensing component is a biocybernetic control loop that utilises an inference of user interest as its primary driver. A biocybernetic loop is composed of four main stages: inference, classification, adaptation and interaction. The programme of research described in this thesis is concerned primarily with exploration of the inference and classification elements of the biocybernetic loop but also encompasses an element of adaptation and interaction. These elements are explored first through literature review and discussion (presented in chapters 1-5) and then through experimental studies (presented in chapters 7-11)

Emotions and cognitive workload in economic decision processes - A NeuroIS Approach

The influence of cognitive and emotions on decision processes have been recently highlighted. Emotions interplay with the process of cognition, and determine decision processes. In this work, the role of external and internal influences on economic decision processes are studied. A NeuroIS method is applied for measuring emotions and cognitive workload. The lack of a suitable experimental platform for performing NeuroIS studies was recognized and the platform Brownie was developed and evaluated

Towards a Biocybernetic Approach for Serious Games Real-time Psychophysiological Inferences for Adaptive Agents in Serious Games

AbstractIn this article we discuss the possible use of real-time psychophysiology towards the design and implementation of an adaptive affective agent for Serious Games. This work is still in its early stages and this position article aims to present a reflection on the technical and methodological conditions to be met so as to conduct successfully this research. Our hypothesis is that, through the combination of psycho-physiological measurements and in-game behaviour/achievement monitoring, we should be able to identify whether or content is contributing to an efficient and engaging educational experience

Evaluating social and cognitive effects of video games using electroencephalography.

Games User Research (GUR) is an area of evaluative player research and human-computer interaction (HCI) that aims to improve games, focused on a player???s understanding and on their experience when playing games. In this field, techniques are available to measure and understand user experience. These techniques each have their own strengths and weaknesses. To improve and extend GUR methodology, this thesis explores ways that electroencephalography (EEG) can be used as an evaluative measure as part of a mixed methodology. The thesis aims to improve the accuracy and richness of GUR results obtained using EEG. Hemispheric Frontal Alpha Asymmetry (HFAA) is reviewed in depth as a useful EEG technique to measure arousal in real time. HFAA, the EEG methodology proposed in this thesis is used in several experimental studies reported here to show new insights into the social and cognitive factors of gaming. The research presented in this thesis shows that player experience related to the social environment of a game does not necessarily arise from gameplay, but instead relies more on the expectations of a player than the current literature suggests. Additionally, the thesis introduces a new way to investigate player understanding and learning in games, using real-time data about the player???s brain state. This is particularly useful for game designers creating introductory tutorial mechanisms for their games. The result of this research is useful for both researchers investigating the human brain immersed in the virtual world of a video game and game designers wanting to use real-time user feedback to build their games

Design and Development of a Real-Time Bio-Sensing System Assessing Student Mental Workload and Engagement

Ο εντοπισμός του επακριβούς επιπέδου προσήλωσης και εμπλοκής των μαθητών με το περιεχόμενο διδασκαλίας στην τάξη είναι ένας από τους πιο μεγαλεπήβολους στόχους των ερευνητών της εκπαιδευτικής και επιστημονικής κοινότητας. (Lang, 1995, Grossberg, 1987). Σχετικές διεπιστημονικές ερευνητικές προσπάθειες προσαύξησης ενδιαφέροντος και εντοπισμού της αποτελεσματικότητας των διδακτικών πρακτικών βασίζονται σε τυπικές μελέτες από τον χώρο της ψυχολογίας, της παιδαγωγικής, της παιδοψυχολογίας και της ψυχοφυσιολογίας. Νέες τεχνολογίες έχουν εισάγει διαγνωστικές συσκευές δανεισμένες από τον χώρο της ιατρικής με σκοπό να εκμεταλλευτούν τις δυνατότητες μετρήσεων βιολογικών σημάτων τα οποία αποτελούν επιβεβαιωμένες εκφράσεις ψυχοφυσιολογικών καταστάσεων οι οποίες μπορούν να μεταφραστούν σε εκδηλώσεις διέγερσης και διάθεσης. Οι ιατρικές συσκευές απαιτούν εργαστηριακό περιβάλλον λόγω των αναγκών χρήσης ηλεκτροδίων, κινητικών περιορισμών, συγχρονισμού και ομοιομορφίας των στοιχείων που προκύπτουν και γι’ αυτό τον λόγο δεν μπόρεσαν ποτέ να αποδόσουν μια προσιτή λύση εφαρμόσιμη ευρύτερα σε εκπαιδευτικό περιβάλλον. Στην παρούσα μελέτη, αναλύονται οι επιδόσεις μιας ειδικά κατασκευασμένης ηλεκτρονικής συσκευής, σχεδιασμένης ώστε να εξεταστούν οι δυνατότητες να εξαχθούν δείκτες ψυχοσωματικών εκφράσεων του χρήστη, με την δυνατότητα να χρησιμοποιείται εύχρηστα στην τάξη χωρίς ηλεκτρόδια και επηρεασμούς από προσαρτήσεις. Το ολοκληρωμένο σύστημα μέτρησης και αποτύπωσης συμπερασμάτων είναι βασισμένο σε μοντελοποίηση συμπεριφορών αλλαγής του καρδιακού παλμού και της ειδικής διηλεκτρικής αγωγιμότητας του δέρματος σε πραγματικό χρόνο. Η συσκευή χρησιμοποιεί οπτικούς και διηλεκτρικούς αισθητήρες επαφής και έχει μελετηθεί σε αντιπαραβολή με διαβαθμισμένα περιβάλλοντα προκλητών καταστάσεων νοητικής φόρτισης. Σειρές πειραματικών διαδικασιών εφαρμοσμένες σε διαβαθμισμένα σενάρια πρόκλησης ψυχοσωματικών διεγέρσεων έχουν ολοκληρωθεί για επικύρωση, μελέτη επιδόσεων και λειτουργία του συστήματος ακόμη και σε σύγκριση με εμπορικό προϊόν. Πειραματικά αποτελέσματα δείχνουν αξιόλογους συσχετισμούς του μοντέλου και των επιδόσεων του συστήματος με τις αναμενόμενες αποκρίσεις με ενθαρρυντικά ποσοστά ακρίβειας.Facing the challenge of improving adaptive interaction in educational technologies scientists and educators have turned their focal point in diverse areas ranging from educational, teaching and behavioural psychology to cognitive, affective and perceptual neuroscience. The introduction of digital technologies and interactive media tools in education has shown improved learning efficiency, much higher memory activation and assimilation than verbal teaching, notably due to enhancing motivation achieved by employing approaches attracting student’s attention. Excelling aspects of audio visual presentation proved highly valuable particularly in classes with multi ethnic groups of students, as for example consistency between definitions and objects which were verbally and visually defined, eliminating possible misconceptions caused by mishearing or misinterpretation by the learner. Taking it all one step further as to how an educational system could be even more efficient, a new element would be needed revealing a credible judgment of learning scores and effectiveness of the learning process instantaneously as for example inner levels of activation and satisfaction. In fact, this could be made possible using existing technologies if subconscious neurophysiological responses of a learner could be ascertained and inferred to psycho-somatic conditions as they occur. A system including bio-sensing, data analysis and processing in real time able to provide quantified markers of psychosomatic states of a learner would help enormously in next generations of educational practice. Incorporating data of student engagement and active involvement could help to deduce the interest of a learner, which is known to improve sensitisation in implicit, incidental and also in classical learning. Experimental settings used in previous studies attempting to incorporate physiological responses and interpretations into responsive educational settings have faced major obstacles. Operational issues caused by the requirements of the devices used for the acquisition of physiological signals such as electrodes and movement restrictions have reduced the progress of such settings to laboratory environments. In such settings as described above, the effects of wiring harnesses and sensory components produced an additional psychological burden on the participants. Consequently, the need to approach the physiological data acquisition from a new angle with seamless and unnoticeable operation is apparent. The challenge to design, develop and validate a system that being minimally obstructive and literally unnoticed by the user would uncover combined subconscious expressions of a learner was the primary objective of this research. Physiological data of Heart Rate and Skin Trans-Conductance (Electro-dermal Response) elected as vitally important and highly appropriate to produce the input of data required to evaluate a behavioural concept model. The behavioural assessment model entailed vector classifiers producing directional interpretations of measurements. Directional information (Gradient response) has been derived by comparison of measurements to previously measured values in real time. Assessing the effectiveness and accuracy of the adopted model to deduce attention and engagement of a learner in real time formed the second major objective. For this purpose, a series of relevant experimental methodologies have been employed. Data produced using formal personality assessments have also been investigated in conjunction with those derived from physiological responses in order to identify personality related particularities. The final part of this work has been supplemented by propositions and suggestions with regards to various applications of the system in accomplishment of the initial aims