296 research outputs found
Modeling online adaptive navigation in virtual environments based on PID control
It is well known that locomotion-dominated navigation tasks may highly
provoke cybersickness effects. Past research has proposed numerous approaches
to tackle this issue based on offline considerations. In this work, a novel
approach to mitigate cybersickness is presented based on online adaptative
navigation. Considering the Proportional-Integral-Derivative (PID) control
method, we proposed a mathematical model for online adaptive navigation
parameterized with several parameters, taking as input the users'
electro-dermal activity (EDA), an efficient indicator to measure the
cybersickness level, and providing as output adapted navigation accelerations.
Therefore, minimizing the cybersickness level is regarded as an argument
optimization problem: find the PID model parameters which can reduce the
severity of cybersickness. User studies were organized to collect non-adapted
navigation accelerations and the corresponding EDA signals. A deep neural
network was then formulated to learn the correlation between EDA and navigation
accelerations. The hyperparameters of the network were obtained through the
Optuna open-source framework. To validate the performance of the optimized
online adaptive navigation developed through the PID control, we performed an
analysis in a simulated user study based on the pre-trained deep neural
network. Results indicate a significant reduction of cybersickness in terms of
EDA signal analysis and motion sickness dose value. This is a pioneering work
which presented a systematic strategy for adaptive navigation settings from a
theoretical point
Forecasting the Onset of Cybersickness using Physiological Data
æć°æćĄïŒMachael Vallanc
Clinical Decision Support Systems with Game-based Environments, Monitoring Symptoms of Parkinsonâs Disease with Exergames
Parkinsonâs Disease (PD) is a malady caused by progressive neuronal degeneration, deriving in several physical and cognitive symptoms that worsen with time. Like many other chronic diseases, it requires constant monitoring to perform medication and therapeutic adjustments. This is due to the significant variability in PD symptomatology and progress between patients. At the moment, this monitoring requires substantial participation from caregivers and numerous clinic visits. Personal diaries and questionnaires are used as data sources for medication and therapeutic adjustments. The subjectivity in these data sources leads to suboptimal clinical decisions. Therefore, more objective data sources are required to better monitor the progress of individual PD patients. A potential contribution towards more objective monitoring of PD is clinical decision support systems. These systems employ sensors and classification techniques to provide caregivers with objective information for their decision-making. This leads to more objective assessments of patient improvement or deterioration, resulting in better adjusted medication and therapeutic plans. Hereby, the need to encourage patients to actively and regularly provide data for remote monitoring remains a significant challenge. To address this challenge, the goal of this thesis is to combine clinical decision support systems with game-based environments. More specifically, serious games in the form of exergames, active video games that involve physical exercise, shall be used to deliver objective data for PD monitoring and therapy. Exergames increase engagement while combining physical and cognitive tasks. This combination, known as dual-tasking, has been proven to improve rehabilitation outcomes in PD: recent randomized clinical trials on exergame-based rehabilitation in PD show improvements in clinical outcomes that are equal or superior to those of traditional rehabilitation. In this thesis, we present an exergame-based clinical decision support system model to monitor symptoms of PD. This model provides both objective information on PD symptoms and an engaging environment for the patients. The model is elaborated, prototypically implemented and validated in the context of two of the most prominent symptoms of PD: (1) balance and gait, as well as (2) hand tremor and slowness of movement (bradykinesia). While balance and gait affections increase the risk of falling, hand tremors and bradykinesia affect hand dexterity. We employ Wii Balance Boards and Leap Motion sensors, and digitalize aspects of current clinical standards used to assess PD symptoms. In addition, we present two dual-tasking exergames: PDDanceCity for balance and gait, and PDPuzzleTable for tremor and bradykinesia. We evaluate the capability of our system for assessing the risk of falling and the severity of tremor in comparison with clinical standards. We also explore the statistical significance and effect size of the data we collect from PD patients and healthy controls. We demonstrate that the presented approach can predict an increased risk of falling and estimate tremor severity. Also, the target population shows a good acceptance of PDDanceCity and PDPuzzleTable. In summary, our results indicate a clear feasibility to implement this system for PD. Nevertheless, long-term randomized clinical trials are required to evaluate the potential of PDDanceCity and PDPuzzleTable for physical and cognitive rehabilitation effects
Human-centric quality management of immersive multimedia applications
Augmented Reality (AR) and Virtual Reality (VR) multimodal systems are the latest trend within the field of multimedia. As they emulate the senses by means of omni-directional visuals, 360 degrees sound, motion tracking and touch simulation, they are able to create a strong feeling of presence and interaction with the virtual environment. These experiences can be applied for virtual training (Industry 4.0), tele-surgery (healthcare) or remote learning (education). However, given the strong time and task sensitiveness of these applications, it is of great importance to sustain the end-user quality, i.e. the Quality-of-Experience (QoE), at all times. Lack of synchronization and quality degradation need to be reduced to a minimum to avoid feelings of cybersickness or loss of immersiveness and concentration. This means that there is a need to shift the quality management from system-centered performance metrics towards a more human, QoE-centered approach. However, this requires for novel techniques in the three areas of the QoE-management loop (monitoring, modelling and control). This position paper identifies open areas of research to fully enable human-centric driven management of immersive multimedia. To this extent, four main dimensions are put forward: (1) Task and well-being driven subjective assessment; (2) Real-time QoE modelling; (3) Accurate viewport prediction; (4) Machine Learning (ML)-based quality optimization and content recreation. This paper discusses the state-of-the-art, and provides with possible solutions to tackle the open challenges
User experience and robustness in social virtual reality applications
Cloud-based applications that rely on emerging technologies such as social virtual reality are increasingly being deployed at high-scale in e.g., remote-learning, public safety, and healthcare. These applications increasingly need mechanisms to maintain robustness and immersive user experience as a joint consideration to minimize disruption in service availability due to cyber attacks/faults. Specifically, effective modeling and real-time adaptation approaches need to be investigated to ensure that the application functionality is resilient and does not induce undesired cybersickness levels. In this thesis, we investigate a novel âDevSecOps' paradigm to jointly tune both the robustness and immersive performance factors in social virtual reality application design/operations. We characterize robustness factors considering Security, Privacy and Safety (SPS), and immersive performance factors considering Quality of Application, Quality of Service, and Quality of Experience (3Q). We achieve âharmonized security and performance by designâ via modeling the SPS and 3Q factors in cloud-hosted applications using attack-fault trees (AFT) and an accurate quantitative analysis via formal verification techniques i.e., statistical model checking (SMC). We develop a real-time adaptive control capability to manage SPS/3Q issues affecting a critical anomaly event that induces undesired cybersickness. This control capability features a novel dynamic rule-based approach for closed-loop decision making augmented by a knowledge base for the SPS/3Q issues of individual and/or combination events. Correspondingly, we collect threat intelligence on application and network based cyber-attacks that disrupt immersiveness, and develop a multi-label K-NN classifier as well as statistical analysis techniques for critical anomaly event detection. We validate the effectiveness of our solution approach in a real-time cloud testbed featuring vSocial, a social virtual reality based learning environment that supports delivery of Social Competence Intervention (SCI) curriculum for youth. Based on our experiment findings, we show that our solution approach enables: (i) identification of the most vulnerable components that impact user immersive experience to formally conduct risk assessment, (ii) dynamic decision making for controlling SPS/3Q issues inducing undesirable cybersickness levels via quantitative metrics of user feedback and effective anomaly detection, and (iii) rule-based policies following the NIST SP 800-160 principles and cloud-hosting recommendations for a more secure, privacy-preserving, and robust cloud-based application configuration with satisfactory immersive user experience.Includes bibliographical references (pages 133-146)
Evaluation of Detecting Cybersickness via VR HMD Positional Measurements Under Realistic Usage Conditions.
With the resurgence of virtual reality, head-mounted displays (VR HMD) technologies since 2015, VR technology is becoming ever more present in people's day-to-day lives. However, one significant barrier to this progress is a condition called cybersickness, a form of motion sickness induced by the usage of VR HMDâs. It is often debilitating to sufferers, resulting in symptoms anywhere from mild discomfort to full-on vomiting. Much research effort focuses on identifying the cause of and solution to this problem, with many studies reporting various factors that influence cybersickness, such as vection and field of view. However, there is often disagreement in these studies' results and comparing the results is often complicated as stimuli used for the experiments vary wildly.
This study theorised that these results' mismatch might partially be down to the different mental loads of these tasks, which may influence cybersickness and stability-based measurement methods such as postural stability captured by the centre of pressure (COP) measurements. One recurring desire in these research projects is the idea of using the HMD device itself to capture the stability of the users head. However, measuring the heads position via the VR HMD is known to have inaccuracies meaning a perfect representation of the heads position cannot be measured.
This research took the HTC Vive headset and used it to capture the head position of multiple subjects experiencing two different VR environments under differing levels of cognitive load. The design of these test environments reflected normal VR usage. This research found that the VR HMD measurements in this scenario may be a suitable proxy for recording instability. However, the underlying method was greatly influenced by other factors, with cognitive load (5.4% instability increase between the low and high load conditions) and test order (2.4% instability decrease between first run and second run conditions) having a more significant impact on the instability recorded than the onset of cybersickness (2% instability increase between sick and well participants). Also, separating participants suffering from cybersickness from unaffected participants was not possible based upon the recorded motion alone. Additionally, attempts to capture stability data during actual VR gameplay in specific areas of possible head stability provided mixed results and failed to identify participants exhibiting symptoms of cybersickness successfully.
In conclusion, this study finds that while a proxy measurement for head stability is obtainable from an HTC Vive headset, the results recorded in no way indicate cybersickness onset. Additionally, the study proves cognitive load and test order significantly impact stability measurements recorded in this way. As such, this approach would need calibration on a case-by-case basis if used to detect cybersickness
ERUDITE: Human-in-the-Loop IoT for an Adaptive Personalized Learning System
Thanks to the rapid growth in wearable technologies and recent advancement in
machine learning and signal processing, monitoring complex human contexts
becomes feasible, paving the way to develop human-in-the-loop IoT systems that
naturally evolve to adapt to the human and environment state autonomously.
Nevertheless, a central challenge in designing many of these IoT systems arises
from the requirement to infer the human mental state, such as intention,
stress, cognition load, or learning ability. While different human contexts can
be inferred from the fusion of different sensor modalities that can correlate
to a particular mental state, the human brain provides a richer sensor modality
that gives us more insights into the required human context. This paper
proposes ERUDITE, a human-in-the-loop IoT system for the learning environment
that exploits recent wearable neurotechnology to decode brain signals. Through
insights from concept learning theory, ERUDITE can infer the human state of
learning and understand when human learning increases or declines. By
quantifying human learning as an input sensory signal, ERUDITE can provide
adequate personalized feedback to humans in a learning environment to enhance
their learning experience. ERUDITE is evaluated across participants and
showed that by using the brain signals as a sensor modality to infer the human
learning state and providing personalized adaptation to the learning
environment, the participants' learning performance increased on average by
. Furthermore, we showed that ERUDITE can be deployed on an edge-based
prototype to evaluate its practicality and scalability.Comment: It is under review in the IEEE IoT journa
The effect of visual detail on cybersickness:predicting symptom severity using spatial velocity
Abstract. In this work, we examine the effect of visual realism on the severity of cybersickness symptoms experienced by users of virtual environments. We also seek to validate a metric called spatial velocity as a predictor of cybersickness. The proposed metric combines the visual complexity of a virtual scene with the amount of movement within the scene.
To achieve this, we prepared two virtual scenes depicting the same environment with a variable level of detail. We recruited volunteers who were exposed to both scenes in two separate sessions. We obtained the sickness ratings after both sessions and saved the data required for spatial velocity calculations.
After comparing the sickness ratings between the two scenes, we found no evidence of the visual realism playing any significant role in the generation of cybersickness symptoms. The spatial velocity also proved inadequate in characterizing the difference in visual complexity and correlated poorly with all the observed sickness scores.Visuaalisen yksityiskohtaisuuden vaikutus VR-pahoinvointiin : oireiden vakavuuden ennustaminen kaÌyttaÌen SV-metriikkaa. TiivistelmĂ€. TĂ€ssĂ€ työssĂ€ tutkimme sitĂ€, millainen vaikutus virtuaalisten ympĂ€ristöjen graafisella yksityiskohtaisuudella on VR-pahoinvointiin. Pyrimme myös validoimaan "spatial velocity" -nimisen mittasuureen kyvyn ennustaa VR-pahoinvoinnin oireiden vakavuutta. Kyseisen mittasuureen etuna on, ettĂ€ se yhdistÀÀ visuaalisen kompleksisuuden ja ympĂ€ristössĂ€ koetun liikkeen yhdeksi suureeksi.
Tutkimusta varten valmistimme kaksi virtuaaliympÀristöÀ, joissa mallinnettiin Oulun yliopiston kampusaluetta. Toinen ympÀristö pyrki mahdollisimman realistiseen esitystapaan, kun taas toisessa yksityiskohtien mÀÀrÀ minimoitiin. Koetta varten vÀrvÀsimme 18 vapaaehtoista. Vapaaehtoiset altistettiin kummallekin ympÀristölle kahdessa noin kymmenen minuutin mittaisessa kokeessa. Vapaaehtoisten kokeman VR-pahoinvoinnin vakavuutta arvioitiin kunkin kokeen jÀlkeen tÀytetyillÀ kyselylomakkeilla. Kokeiden aikana tallensimme myös SV laskentaan tarvittavat tiedot.
Verrattuamme koeolosuhteiden tuloksia, emme löytÀneet todisteita siitÀ, ettÀ ympÀristön graafisten yksityiskohtien mÀÀrÀllÀ olisi merkittÀvÀÀ vaikutusta koettuun pahoinvointiin. KÀytetty SV metriikka ei myöskÀÀn kyennyt erottelemaan ympÀristöjÀ oletetulla tavalla, eivÀtkÀ lasketut arvot korreloineet merkittÀvÀsti minkÀÀn mitatun pahoinvointisuureen kanssa
EnviroScape: Coping With Stress Using Implicit Biofeedback Application
Stress has been identified by the Word Health Organization as an epidemic that has negative impacts on work productivity. It costs the American industry approximately $300 billion/year and is also the leading contributor to obesity and cardiovascular diseases. Current stress remediation tools incorporate techniques such as deep breathing, meditation and biofeedback responses. These type of exercises require a substantial amount of time and resources along with adhering to their strict system in order to see results. Most biofeedback mechanisms are repetitive and mundane and also require complex equipment to participate, in order to receive proper evaluation on stress levels. The purpose of this study is to develop an engaging relaxation technique and analyze the effects of the biofeedback mechanism on the stress levels of a user. An interactive application is developed such that the user receives subtle cues when they are in a âstressedâ state, which is determined through the physiological indicator of the userâs breathing rate (BR) signal. Unlike previous research, this biofeedback game focuses on providing a soothing natural environment with no specific objectives in order to distract them from their current stressful state. This will help analyze and discuss the effects of a non-competitive video game on a userâs stress levels, their awareness to recognize signs of stress and their ability to reduce them
Automatic cybersickness detection by deep learning of augmented physiological data from off-the-shelf consumer-grade sensors
Cybersickness is still a prominent risk factor potentially affecting the usability of virtual reality applications. Automated real-time detection of cybersickness promises to support a better general understanding of the phenomena and to avoid and counteract its occurrence. It could be used to facilitate application optimization, that is, to systematically link potential causes (technical development and conceptual design decisions) to cybersickness in closed-loop user-centered development cycles. In addition, it could be used to monitor, warn, and hence safeguard users against any onset of cybersickness during a virtual reality exposure, especially in healthcare applications. This article presents a novel real-time-capable cybersickness detection method by deep learning of augmented physiological data. In contrast to related preliminary work, we are exploring a unique combination of mid-immersion ground truth elicitation, an unobtrusive wireless setup, and moderate training performance requirements. We developed a proof-of-concept prototype to compare (combinations of) convolutional neural networks, long short-term memory, and support vector machines with respect to detection performance. We demonstrate that the use of a conditional generative adversarial network-based data augmentation technique increases detection performance significantly and showcase the feasibility of real-time cybersickness detection in a genuine application example. Finally, a comprehensive performance analysis demonstrates that a four-layered bidirectional long short-term memory network with the developed data augmentation delivers superior performance (91.1% F1-score) for real-time cybersickness detection. To encourage replicability and reuse in future cybersickness studies, we released the code and the dataset as publicly available
- âŠ