189 research outputs found

    PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User Studies

    Get PDF
    The proliferation of physiological sensors opens new opportunities to explore interactions, conduct experiments and evaluate the user experience with continuous monitoring of bodily functions. Commercial devices, however, can be costly or limit access to raw waveform data, while low-cost sensors are efforts-intensive to setup. To address these challenges, we introduce PhysioKit, an open-source, low-cost physiological computing toolkit. PhysioKit provides a one-stop pipeline consisting of (i) a sensing and data acquisition layer that can be configured in a modular manner per research needs, and (ii) a software application layer that enables data acquisition, real-time visualization and machine learning (ML)-enabled signal quality assessment. This also supports basic visual biofeedback configurations and synchronized acquisition for co-located or remote multi-user settings. In a validation study with 16 participants, PhysioKit shows strong agreement with research-grade sensors on measuring heart rate and heart rate variability metrics data. Furthermore, we report usability survey results from 10 small-project teams (44 individual members in total) who used PhysioKit for 4–6 weeks, providing insights into its use cases and research benefits. Lastly, we discuss the extensibility and potential impact of the toolkit on the research community

    Feeling the Temperature of the Room: Unobtrusive Thermal Display of Engagement during Group Communication

    Full text link
    Thermal signals have been explored in HCI for emotion-elicitation and enhancing two-person communication, showing that temperature invokes social and emotional signals in individuals. Yet, extending these findings to group communication is missing. We investigated how thermal signals can be used to communicate group affective states in a hybrid meeting scenario to help people feel connected over a distance. We conducted a lab study (N=20 participants) and explored wrist-worn thermal feedback to communicate audience emotions. Our results show that thermal feedback is an effective method of conveying audience engagement without increasing workload and can help a presenter feel more in tune with the audience. We outline design implications for real-world wearable social thermal feedback systems for both virtual and in-person communication that support group affect communication and social connectedness. Thermal feedback has the potential to connect people across distances and facilitate more effective and dynamic communication in multiple contexts.Comment: In IMWUT 202

    AffectiveViz:Designing Collective Stress Related Visualization

    Get PDF

    Game AI revisited

    Get PDF
    More than a decade after the early research efforts on the use of artificial intelligence (AI) in computer games and the establishment of a new AI domain the term “game AI” needs to be redefined. Traditionally, the tasks associated with game AI revolved around non player character (NPC) behavior at different levels of control, varying from navigation and pathfinding to decision making. Commercial-standard games developed over the last 15 years and current game productions, however, suggest that the traditional challenges of game AI have been well addressed via the use of sophisticated AI approaches, not necessarily following or inspired by advances in academic practices. The marginal penetration of traditional academic game AI methods in industrial productions has been mainly due to the lack of constructive communication between academia and industry in the early days of academic game AI, and the inability of academic game AI to propose methods that would significantly advance existing development processes or provide scalable solutions to real world problems. Recently, however, there has been a shift of research focus as the current plethora of AI uses in games is breaking the non-player character AI tradition. A number of those alternative AI uses have already shown a significant potential for the design of better games. This paper presents four key game AI research areas that are currently reshaping the research roadmap in the game AI field and evidently put the game AI term under a new perspective. These game AI flagship research areas include the computational modeling of player experience, the procedural generation of content, the mining of player data on massive-scale and the alternative AI research foci for enhancing NPC capabilities.peer-reviewe

    Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems

    Get PDF
    Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various technical complexities such as multimodal data types, diverse hardware and software components, time synchronisation issues and distributed deployment configurations. Building these systems is inherently difficult and requires addressing of these complexities before the intended and purposeful functionalities can be attained. The technical issues are often common and similar among diverse applications. This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS) framework, a generic solution to address the technical complexities in building DCISs. The proposed solution is an abstract software framework that can be extended and customised to any application requirements. UbiITS includes all fundamental software components, techniques, system level layer abstractions and reference architecture as a collection to enable the systematic construction of complex DCISs. This work details four case studies to showcase the versatility and extensibility of UbiITS framework’s functionalities and demonstrate how it was employed to successfully solve a range of technical requirements. In each case UbiITS operated as the core element of each application. Additionally, these case studies are novel systems by themselves in each of their domains. Longstanding technical issues such as flexibly integrating and interoperating multimodal tools, precise time synchronisation, etc., were resolved in each application by employing UbiITS. The framework enabled establishing a functional system infrastructure in these cases, essentially opening up new lines of research in each discipline where these research approaches would not have been possible without the infrastructure provided by the framework. The thesis further presents a sample implementation of the framework on a device firmware exhibiting its capability to be directly implemented on a hardware platform. Summary metrics are also produced to establish the complexity, reusability, extendibility, implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394

    Design and Development of a Real-Time Bio-Sensing System Assessing Student Mental Workload and Engagement

    Get PDF
    Ο εντοπισμός του επακριβούς επιπέδου προσήλωσης και εμπλοκής των μαθητών με το περιεχόμενο διδασκαλίας στην τάξη είναι ένας από τους πιο μεγαλεπήβολους στόχους των ερευνητών της εκπαιδευτικής και επιστημονικής κοινότητας. (Lang, 1995, Grossberg, 1987). Σχετικές διεπιστημονικές ερευνητικές προσπάθειες προσαύξησης ενδιαφέροντος και εντοπισμού της αποτελεσματικότητας των διδακτικών πρακτικών βασίζονται σε τυπικές μελέτες από τον χώρο της ψυχολογίας, της παιδαγωγικής, της παιδοψυχολογίας και της ψυχοφυσιολογίας. Νέες τεχνολογίες έχουν εισάγει διαγνωστικές συσκευές δανεισμένες από τον χώρο της ιατρικής με σκοπό να εκμεταλλευτούν τις δυνατότητες μετρήσεων βιολογικών σημάτων τα οποία αποτελούν επιβεβαιωμένες εκφράσεις ψυχοφυσιολογικών καταστάσεων οι οποίες μπορούν να μεταφραστούν σε εκδηλώσεις διέγερσης και διάθεσης. Οι ιατρικές συσκευές απαιτούν εργαστηριακό περιβάλλον λόγω των αναγκών χρήσης ηλεκτροδίων, κινητικών περιορισμών, συγχρονισμού και ομοιομορφίας των στοιχείων που προκύπτουν και γι’ αυτό τον λόγο δεν μπόρεσαν ποτέ να αποδόσουν μια προσιτή λύση εφαρμόσιμη ευρύτερα σε εκπαιδευτικό περιβάλλον. Στην παρούσα μελέτη, αναλύονται οι επιδόσεις μιας ειδικά κατασκευασμένης ηλεκτρονικής συσκευής, σχεδιασμένης ώστε να εξεταστούν οι δυνατότητες να εξαχθούν δείκτες ψυχοσωματικών εκφράσεων του χρήστη, με την δυνατότητα να χρησιμοποιείται εύχρηστα στην τάξη χωρίς ηλεκτρόδια και επηρεασμούς από προσαρτήσεις. Το ολοκληρωμένο σύστημα μέτρησης και αποτύπωσης συμπερασμάτων είναι βασισμένο σε μοντελοποίηση συμπεριφορών αλλαγής του καρδιακού παλμού και της ειδικής διηλεκτρικής αγωγιμότητας του δέρματος σε πραγματικό χρόνο. Η συσκευή χρησιμοποιεί οπτικούς και διηλεκτρικούς αισθητήρες επαφής και έχει μελετηθεί σε αντιπαραβολή με διαβαθμισμένα περιβάλλοντα προκλητών καταστάσεων νοητικής φόρτισης. Σειρές πειραματικών διαδικασιών εφαρμοσμένες σε διαβαθμισμένα σενάρια πρόκλησης ψυχοσωματικών διεγέρσεων έχουν ολοκληρωθεί για επικύρωση, μελέτη επιδόσεων και λειτουργία του συστήματος ακόμη και σε σύγκριση με εμπορικό προϊόν. Πειραματικά αποτελέσματα δείχνουν αξιόλογους συσχετισμούς του μοντέλου και των επιδόσεων του συστήματος με τις αναμενόμενες αποκρίσεις με ενθαρρυντικά ποσοστά ακρίβειας.Facing the challenge of improving adaptive interaction in educational technologies scientists and educators have turned their focal point in diverse areas ranging from educational, teaching and behavioural psychology to cognitive, affective and perceptual neuroscience. The introduction of digital technologies and interactive media tools in education has shown improved learning efficiency, much higher memory activation and assimilation than verbal teaching, notably due to enhancing motivation achieved by employing approaches attracting student’s attention. Excelling aspects of audio visual presentation proved highly valuable particularly in classes with multi ethnic groups of students, as for example consistency between definitions and objects which were verbally and visually defined, eliminating possible misconceptions caused by mishearing or misinterpretation by the learner. Taking it all one step further as to how an educational system could be even more efficient, a new element would be needed revealing a credible judgment of learning scores and effectiveness of the learning process instantaneously as for example inner levels of activation and satisfaction. In fact, this could be made possible using existing technologies if subconscious neurophysiological responses of a learner could be ascertained and inferred to psycho-somatic conditions as they occur. A system including bio-sensing, data analysis and processing in real time able to provide quantified markers of psychosomatic states of a learner would help enormously in next generations of educational practice. Incorporating data of student engagement and active involvement could help to deduce the interest of a learner, which is known to improve sensitisation in implicit, incidental and also in classical learning. Experimental settings used in previous studies attempting to incorporate physiological responses and interpretations into responsive educational settings have faced major obstacles. Operational issues caused by the requirements of the devices used for the acquisition of physiological signals such as electrodes and movement restrictions have reduced the progress of such settings to laboratory environments. In such settings as described above, the effects of wiring harnesses and sensory components produced an additional psychological burden on the participants. Consequently, the need to approach the physiological data acquisition from a new angle with seamless and unnoticeable operation is apparent. The challenge to design, develop and validate a system that being minimally obstructive and literally unnoticed by the user would uncover combined subconscious expressions of a learner was the primary objective of this research. Physiological data of Heart Rate and Skin Trans-Conductance (Electro-dermal Response) elected as vitally important and highly appropriate to produce the input of data required to evaluate a behavioural concept model. The behavioural assessment model entailed vector classifiers producing directional interpretations of measurements. Directional information (Gradient response) has been derived by comparison of measurements to previously measured values in real time. Assessing the effectiveness and accuracy of the adopted model to deduce attention and engagement of a learner in real time formed the second major objective. For this purpose, a series of relevant experimental methodologies have been employed. Data produced using formal personality assessments have also been investigated in conjunction with those derived from physiological responses in order to identify personality related particularities. The final part of this work has been supplemented by propositions and suggestions with regards to various applications of the system in accomplishment of the initial aims

    Toward Data-Driven Digital Therapeutics Analytics: Literature Review and Research Directions

    Full text link
    With the advent of Digital Therapeutics (DTx), the development of software as a medical device (SaMD) for mobile and wearable devices has gained significant attention in recent years. Existing DTx evaluations, such as randomized clinical trials, mostly focus on verifying the effectiveness of DTx products. To acquire a deeper understanding of DTx engagement and behavioral adherence, beyond efficacy, a large amount of contextual and interaction data from mobile and wearable devices during field deployment would be required for analysis. In this work, the overall flow of the data-driven DTx analytics is reviewed to help researchers and practitioners to explore DTx datasets, to investigate contextual patterns associated with DTx usage, and to establish the (causal) relationship of DTx engagement and behavioral adherence. This review of the key components of data-driven analytics provides novel research directions in the analysis of mobile sensor and interaction datasets, which helps to iteratively improve the receptivity of existing DTx.Comment: This paper has been accepted by the IEEE/CAA Journal of Automatica Sinic

    NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

    Get PDF
    Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas
    corecore