Avaliação do Virtual Reality Sickness Questionnaire (VRSQ) como preditor de cybersickness

Trabalho de conclusão de curso (graduação)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2020.Cybersickness é uma síndrome que ocorre em alguns usuários de equipamentos de reali- dade virtual (RV). Para o futuro de tecnologias RV, é importante que sejam determinadas formas eficazes de detectar e medir cybersickness, além de prevenir e mitigar seus efeitos. Este trabalho verifica se um subconjunto do Simulator Sickness Questionnaire (SSQ) chamado Virtual Reality Sickness Questionnaire (VRSQ) é suficiente para prever o resul- tado do SSQ e, assim, medir cybersickness. O SSQ ainda é o método mais comum de se medir cybersickness no meio científico, por mais que não tenha sido feito para cybersick- ness em específico. O VRSQ é um questionário mais recente que busca essa especificidade, mas ainda não foi adotado pela comunidade científica. Regressões lineares mostraram que os resultados são fortemente correlacionados (R = 0.95), e as medidas fisiológicas sele- cionadas possuem o mesmo poder preditivo para ambos os questionários (R2 = 0.99). Como as medidas fisiológicas tiveram poder preditivo baixo para os questionários (R2 próximo de 0.4), trabalhos que busquem melhores preditores de cybersickness são essen- ciais para concluir a comparação feita por meio de medidas objetivas.Cybersickness is a syndrome which besets some users of virtual reality (VR). The de- tection and measurement of this condition, as well as its prevention and mitigation, is paramount for the future of VR technology. This work verifies if a subset of the Simulator Sickness Questionnaire (SSQ) called the Virtual Reality Sickness Questionnaire (VRSQ) is sufficient to predict the results of the SSQ, therefore measuring cybersickness. Although SSQ is still the most commonly used questionnaire to measure cybersickness, VRSQ is a more modern questionnaire which was made specifically for cybersickness. Linear re- gressions found a strong correlation between the scores (R = 0.95), and analysis through physiological measurements showed strong correlation between the predictive power of the selected measurements for both questionnaires (R2 = 0.99. Since the physiological measurements only weakly correlated (R2 around 0.40) with the questionnaire scores, the search for better predictors is essential for a definitive conclusion through objective measurements

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing

Development of tools and paradigms to assess brain cortical activity during cognitive tasks

Monitoring brain cortical activity is essential to decipher and understand neurophysiological behaviour. A wide amount of tools and experimental setups has been developed to stimulate, record and analyze brain activity. The identification of quantitative metrics to assess this activity during specific tasks remains an essential requirement, as it could lead to improve diagnostics, describe objectively self-assessed condition, or track variation during long-term studies. This thesis introduces the development of tools and paradigms to assess brain cortical activity during cognitive tasks. It introduces a complete set of analyses based on EEG signals, under two main scopes, schizophrenia and postural control. The first part of the work evaluates the impact of a potential therapeutic solution for patients with schizophrenia. A longitudinal study case is introduced, where psychometrics data are compared with three types of analysis from EEG data: temporal, spectral and connectivity. The small sample size prevents us to draw definitive conclusion, however, this work reveals the interest to use EEG-based metrics to complete the standard psychometrics assessment. The second part of the work focuses on postural control, using a novel measurement setup, called BioVRSea, combining virtual reality and a moving platform. The brain cortical activity of more than 150 healthy individuals have been investigated during this experiment. A robust neurophysiological reference has been identified using power spectral density. Moreover, combining brain connectivity and microstate segmentation, network dynamics reveal a coherent brain remodeling throughout the acquisition, strengthening our current knowledge regarding complex postural control. The current work highlights the concrete benefit of using EEG signal to decipher brain cortical activity. The tools developed in this thesis are of interest to build a neurophysiological signature of specific cognitive tasks, that will be crucial for a further understanding of neurodegenerative disease.Að fylgjast með starfsemi heilaberki er nauðsynlegt til að útskýra og skilja tauga-lífeðlisfræðilega hegðun. Fjölbreytt útval af tólum og tilraunauppsetningum hefur verið þróað til að örva, vista og greina heilastarfsemi. Það er nauðsynleg krafa að finna magnmælingu til að greina þessa starfsemi í ákveðnu verkefni, því það gæti leitt að beturumbættu greiningarferli, útskýrt hlutlægu sjálfsmats ástandi, eða fylgst með breytingum í lang-tíma rannsóknum. Þessi ritgerð kynnir þróunn tóla og hugmyndafræði til að meta heilaberka starfsemi við vitræn verkefni. Ritgerðinn kynnir heilt safn af greiningum byggt á EEG merkj- um, í tvem megin sviðum, geðklofa og líkamsstöðustjórnun. Fyrsti hluti verkefnisins metur áhrifin af mögulegum meðferðarlegum lausnum fyrir sjúklinga með geðklofa. Kynnt er langtímarannsóknartilvik, þar sem þar sem sálfræðigögn eru borin saman við þrenns konar greiningar úr heilarita gögnum: tímabundnum, litrófs- og tengingum. Lítil úrtaksstærð kemur í veg fyrir að við getum dregið endanlegar ályktanir, en þessi vinna sýnir áhugann á því að nota heilalínuritaða mælikvarða til að ljúka stöðluðu sálfræðimati. Annar hluti verksins fjallar um líkamsstöðustýringu, með því að nota nýja mæling- aruppsetningu, sem kallast BioVRSea, sem sameinar sýndarveruleika og hreyfanlegan vettvang. Heilabarkarvirkni af meira en 150 heilbrigðra einstaklinga hefur verið rann- sökuð í þessari tilraun. Öflug taugalífeðlisfræðileg tilvísun hefur fundist með því að nota kraftrófsþéttleika. Þar að auki, með því að sameina heilatengingu og örstöðu- skiptingu, sýnir netverkun samfellda endurgerð heilans í gegnum tökuna, sem styrkir núverandi þekkingu okkar varðandi flókna líkamsstöðustjórnun. Þessi rannsókn undirstrikar raunverulegan ávinning af því að nota EEG merki til að ráða virkni heilabarka. Tólin sem þróuð eru í þessari ritgerð eru mikilvæg til að byggja upp taugalífeðlisfræðilega undirskrift ákveðinna vitræna verkefna, sem mikilvæg eru fyrir frekari skilning á taugahrörnunarsjúkdómum

Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems

Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various technical complexities such as multimodal data types, diverse hardware and software components, time synchronisation issues and distributed deployment configurations. Building these systems is inherently difficult and requires addressing of these complexities before the intended and purposeful functionalities can be attained. The technical issues are often common and similar among diverse applications. This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS) framework, a generic solution to address the technical complexities in building DCISs. The proposed solution is an abstract software framework that can be extended and customised to any application requirements. UbiITS includes all fundamental software components, techniques, system level layer abstractions and reference architecture as a collection to enable the systematic construction of complex DCISs. This work details four case studies to showcase the versatility and extensibility of UbiITS framework’s functionalities and demonstrate how it was employed to successfully solve a range of technical requirements. In each case UbiITS operated as the core element of each application. Additionally, these case studies are novel systems by themselves in each of their domains. Longstanding technical issues such as flexibly integrating and interoperating multimodal tools, precise time synchronisation, etc., were resolved in each application by employing UbiITS. The framework enabled establishing a functional system infrastructure in these cases, essentially opening up new lines of research in each discipline where these research approaches would not have been possible without the infrastructure provided by the framework. The thesis further presents a sample implementation of the framework on a device firmware exhibiting its capability to be directly implemented on a hardware platform. Summary metrics are also produced to establish the complexity, reusability, extendibility, implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394

Accessibility of Health Data Representations for Older Adults: Challenges and Opportunities for Design

Health data of consumer off-the-shelf wearable devices is often conveyed to users through visual data representations and analyses. However, this is not always accessible to people with disabilities or older people due to low vision, cognitive impairments or literacy issues. Due to trade-offs between aesthetics predominance or information overload, real-time user feedback may not be conveyed easily from sensor devices through visual cues like graphs and texts. These difficulties may hinder critical data understanding. Additional auditory and tactile feedback can also provide immediate and accessible cues from these wearable devices, but it is necessary to understand existing data representation limitations initially. To avoid higher cognitive and visual overload, auditory and haptic cues can be designed to complement, replace or reinforce visual cues. In this paper, we outline the challenges in existing data representation and the necessary evidence to enhance the accessibility of health information from personal sensing devices used to monitor health parameters such as blood pressure, sleep, activity, heart rate and more. By creating innovative and inclusive user feedback, users will likely want to engage and interact with new devices and their own data

WiFi-Based Human Activity Recognition Using Attention-Based BiLSTM

Recently, significant efforts have been made to explore human activity recognition (HAR) techniques that use information gathered by existing indoor wireless infrastructures through WiFi signals without demanding the monitored subject to carry a dedicated device. The key intuition is that different activities introduce different multi-paths in WiFi signals and generate different patterns in the time series of channel state information (CSI). In this paper, we propose and evaluate a full pipeline for a CSI-based human activity recognition framework for 12 activities in three different spatial environments using two deep learning models: ABiLSTM and CNN-ABiLSTM. Evaluation experiments have demonstrated that the proposed models outperform state-of-the-art models. Also, the experiments show that the proposed models can be applied to other environments with different configurations, albeit with some caveats. The proposed ABiLSTM model achieves an overall accuracy of 94.03%, 91.96%, and 92.59% across the 3 target environments. While the proposed CNN-ABiLSTM model reaches an accuracy of 98.54%, 94.25% and 95.09% across those same environments

Emotion and Stress Recognition Related Sensors and Machine Learning Technologies

This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective

Finding the Hidden: Detecting Atypical Affective States from Physiological Signals

In cognitive science, intuition is described as a strategy of processing information that relies on people's instinctive and emotional criteria. When compared with the deliberate choices made after conscious reasoning, the quick and intuitive decision making strategies can be more effective. The intuitive thinking provokes changes in human physiological responses which can be measured by sensors. Utilising physiological reactions, previous work shows that atypical patterns such as emotion expressions and image manipulations can be identified. This thesis expands the exploration to examine whether more atypical human behaviour can be recognised from physiological signals. The examined subtly atypical behaviour includes depression, doubt and deception, Depression is a serious chronic mental disease and is considered as an atypical health condition in people. Doubt is defined as a non-deliberate attempt to mislead others and is a passive form of deception, representing an atypicality from honest behaviours. Deception is a more purposeful attempt to deceive, and thus is a distinct type of atypicality than honest communication. Through examining physiological reactions from presenters who have a particular atypical behaviour or condition, and observers who view behaviours of presenters, this research aims to recognise atypicality in human behaviour. A collection of six user studies are conducted. In two user studies, presenters are asked to conduct doubting and deceiving behaviours, while the remaining user studies involve observers watching behaviours of presenters who suffer from depression, have doubt, or have conducted deception. Physiological reactions of both presenters and observers are collected, including Blood Volume Pulse, Electrodermal Activity, Skin Temperature and Pupillary Responses. Observers are also asked to explicitly evaluate whether the viewed presenters were being depressed, doubting, or deceiving. Investigations upon physiological data in this thesis finds that detectable cues corresponding with depression, doubt and deception can be found. Viewing depression provokes visceral physiological reactions in observers that can be measured. Such physiological responses can be used to derive features for machine learning models to accurately distinguish between healthy individuals and people with depression. By contrast, depression does not provoke strong conscious recognition in observers, resulting in a conscious evaluation accuracy slightly above chance level. Similar results are also found in detecting doubt and deception. People with doubt and deceit elicit consistent physiological reactions within themselves. These bodily responses can be utilised by machine learning models or deep learning models to recognise doubt or deception. The doubt and deceit in presenters can also be recognised using physiological signals in observers, with excellent recognition rates which are higher when compared with the conscious judgments from the same group of observers. The results indicate that atypicality in presenters can both be captured by physiological signals of presenters and observers. Presenters' physiological reactions contribute to higher recognition of atypicality, but observers' physiological responses can serve as a comparable alternative. The awareness of atypicality among observers happens physiologically, so can be used by machine learning models, even when they do not reach the consciousness of the person. The research findings lead to a further discussion around the implications of observers' physiological responses. Decision support applications which utilise a quantifiable measure of people's unconscious and intuitive 'gut feeling' can be developed based on the work reported here to assist people with medical diagnosis, information credibility evaluation, and criminal detection. Further research suggests exploring more atypical behaviours in the wild