Investigating the Role of Biofeedback and Haptic Stimulation in Mobile Paced Breathing Tools

Previous studies have shown that mindfulness meditation and paced breathing are effective tools for stress management. There are a number of mobile applications currently available that are designed to guide the breath to support these relaxation practices. However, these focus mainly on audio/visual cues and are mostly non-interactive. Our goal is to develop a mobile paced breathing tool focusing on the exploration of haptic cues and biofeedback. We conducted user studies to investigate the effectiveness of the system. This study explores the following questions: Do users prefer control of the breathing rate interval through an on-screen slider (manual mode) or through a physiological sensor (biofeedback mode)? How effective is haptic guidance on its own? And how may the addition of haptic feedback enhance audio-based guidance? Our analysis suggests that while both manual and biofeedback modes are desirable, manual control leads to a greater overall increase in relaxation. Additionally, the findings of this study support the value of haptic guidance in mobile paced breathing tools

Live Biofeedback as a User Interface Design Element: A Review of the Literature

With the advances in sensor technology and real-time processing of neurophysiological data, a growing body of academic literature has begun to explore how live biofeedback can be integrated into information systems for everyday use. While researchers have traditionally studied live biofeedback in the clinical domain, the proliferation of affordable mobile sensor technology enables researchers and practitioners to consider live biofeedback as a user interface element in contexts such as decision support, education, and gaming. In order to establish the current state of research on live biofeedback, we conducted a literature review on studies that examine self and foreign live biofeedback based on neurophysiological data for healthy subjects in an information systems context. By integrating a body of highly fragmented work from computer science, engineering and technology, information systems, medical science, and psychology, this paper synthesizes results from existing research, identifies knowledge gaps, and suggests directions for future research. In this vein, this review can serve as a reference guide for researchers and practitioners on how to integrate self and foreign live biofeedback into information systems for everyday use

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

Design and Evaluation of Biofeedback:Interfaces for Awareness and Regulation of Affect

Biofeedback interfaces enable dynamic representations of bodily data using sensors and actuators to actively control complex physiological activities. These provide individuals with access to their psychophysiological processes, help regulate bodily responses, and have been shown to have positive effects on affective health and wellbeing. Traditionally biofeedback has been provided using audiovisual modality whose understanding usually required technical input from physicians. There are still a limited number of biofeedback interfaces that have been deployed from the lab settings to everyday lives. Specifically, there is a limited focus on low-cost, non-screen based, emerging alternative technologies that could support biosensory information in different ways so that users themselves can understand it. To address these challenges, this thesis engages in the design and evaluation of low-cost, wearable smart materials and actuators to support awareness and regulation of affect. The thesis presents six studies describing them. The first exploration of smart materials and actuators helped in unpacking their material qualities. These include responsiveness, duration, rhythm, aliveness, and range, which led to the design of six wearable visual and haptic interfaces representing physiological arousal. By evaluating the six interfaces in daily life settings, the thesis' findings have shown how the material-driven qualities of the interfaces shape people’s awareness of emotions in different ways starting with reflexivity, emotion identification, and finally, its attribution. This thesis then presents the design of the ThermoPixels toolkit containing digital and physical materials. The toolkit is evaluated by involving users in the design of affective displays for arousal. Findings reveal two distinct motivations for designing physiological arousal interfaces, i.e., awareness and regulation. Analysis of both types of representations helped study their qualities and the role of colors and shapes for personalizing interfaces for awareness and regulation of arousal, i.e., awareness of increased arousal can be supported by angular shapes, warm colors, and rich patterns and regulation of high arousal can be supported by round shapes, cool colors, and light patterns. Moving forward, the thesis engages in the exploration of heart rate variability to regulate affect. It introduces a mixed-methods approach to compare and evaluate wearable heart rate variability sensors in terms of data quality and user acceptance. Following heart rate variability exploration, the thesis involves users in the design of vibrotactile and temperature patterns for affect regulation and demonstrates the value of personalized haptic patterns in regulating affect as measured by self-reported forms and heart rate variability. Interviews with the haptic group help study haptic patterns' experiential qualities and participants' experiences. Between subjects analysis indicates that subjective and objective measures of anxiety and stress decreased under haptic patterns than without and that low frequency vibration was the most effective pattern for stress regulation. The contribution of this work includes unpacking experiential qualities of high - low frequency vibration and warm - cool thermal patterns for affect regulation by engaging users in their design and guidelines for designing these patterns. Finally, two visual and haptic wearable smartwatch apps i.e., Breathe and Heart are designed for affect regulation. These utilize slow bodily rhythms of breathing and heartbeat and are evaluated in daily life under everyday life situations of high arousal negative affect. Findings show the value of technology-delivered interventions in supporting affect regulation that can augment prior strategies being implemented by individuals in their daily lives. The thesis is concluded with a discussion of research contributions and future directions

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS

Historically, communication implies the transfer of information between bodies, yet this phenomenon is constantly adapting to new technological and cultural standards. In a digital context, it’s commonplace to envision systems that revolve around verbal modalities. However, behavioural analysis grounded in psychology research calls attention to the emotional information disclosed by non-verbal social cues, in particular, actions that are involuntary. This notion has circulated heavily into various interdisciplinary computing research fields, from which multiple studies have arisen, correlating non-verbal activity to socio-affective inferences. These are often derived from some form of motion capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that occur from inside the body. This thesis proposes a motivation and methodology for using physiological sensory data as an expressive resource for technology-mediated interactions. Initialised from a thorough discussion on state-of-the-art technologies and established design principles regarding this topic, then applied to a novel approach alongside a selection of practice works to compliment this. We advocate for aesthetic experience, experimenting with abstract representations. Atypically from prevailing Affective Computing systems, the intention is not to infer or classify emotion but rather to create new opportunities for rich gestural exchange, unconfined to the verbal domain. Given the preliminary proposition of non-representation, we justify a correspondence with modern Machine Learning and multimedia interaction strategies, applying an iterative, human-centred approach to improve personalisation without the compromising emotional potential of bodily gesture. Where related studies in the past have successfully provoked strong design concepts through innovative fabrications, these are typically limited to simple linear, one-to-one mappings and often neglect multi-user environments; we foresee a vast potential. In our use cases, we adopt neural network architectures to generate highly granular biofeedback from low-dimensional input data. We present the following proof-of-concepts: Breathing Correspondence, a wearable biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance; and Anti-Social Distancing Ensemble, an installation for public space interventions, analysing physical distance to generate a collective soundscape. Key findings are extracted from the individual reports to formulate an extensive technical and theoretical framework around this topic. The projects first aim to embrace some alternative perspectives already established within Affective Computing research. From here, these concepts evolve deeper, bridging theories from contemporary creative and technical practices with the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram em torno de modalidades verbais. Contudo, a análise comportamental fundamentada na investigação psicológica chama a atenção para a informação emocional revelada por sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou fortemente em vários campos interdisciplinares de investigação na área das ciências da computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal com inferências sócio-afectivas. Estes são frequentemente derivados de alguma forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas “invisíveis” que ocorrem no interior do corpo. Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia. Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem, juntamente com uma selecção de trabalhos práticos, para complementar esta. Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente aos sistemas de Computação Afectiva predominantes, a intenção não é inferir ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual, não confinada ao domínio verbal. Dada a proposta preliminar de não representação, justificamos uma correspondência com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma abordagem iterativa e centrada no ser humano para melhorar a personalização sem o potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores demonstraram com sucesso conceitos de design fortes através de fabricações inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um, e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão. Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema de biofeedback wearable inspirado nos princípios de design somaestético; Latent Steps, um modelo autoencoder em tempo real para representar experiências corporais a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing Ensemble, uma instalação para intervenções no espaço público, analisando a distância física para gerar uma paisagem sonora colectiva. Os principais resultados são extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas perspectivas alternativas às que já estão estabelecidas no âmbito da investigação da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente, fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o avanço das tecnologias biomédicas

A REMATERIALIZAÇÃO HÁPTICA COMO RESPOSTA À DESMATERIALIZAÇÃO: UMA INTERPRETAÇÃO PELO ATUAL CONTEXTO TECNOLÓGICO

A computação aliada às redes de comunicação trouxe o referencial global para os ambientes domésticos. Neste meio, o acesso a conteúdos digitais parece se adequar aos conceitos da desmaterialização, um dos focos da sustentabilidade. Entretanto, há impacto perceptível nas relações que as pessoas tecem com o ambiente, com os recursos e também com outras pessoas. Percebe-se, dentre outros aspectos, um afastamento importante das relações táteis com os objetos. Ao mesmo tempo, observa-se em pesquisas a intenção de desenvolver informações háptica para dispositivos móveis. Com a intenção de compreender a relação entre os objetos digitais e a desmaterialização e o possível afastamento das relações táteis, exploramos o seguinte questionamento: seria a háptica uma resposta à emergente necessidade de desmaterialização? Embora não seja apresentada uma resposta definitiva, sugere-se que, se não uma relação direta, a háptica poderá auxiliar em alguns aspectos perceptivos da desmaterialização