63 research outputs found
Affective Man-Machine Interface: Unveiling human emotions through biosignals
As is known for centuries, humans exhibit an electrical profile. This profile is altered through various psychological and physiological processes, which can be measured through biosignals; e.g., electromyography (EMG) and electrodermal activity (EDA). These biosignals can reveal our emotions and, as such, can serve as an advanced man-machine interface (MMI) for empathic consumer products. However, such a MMI requires the correct classification of biosignals to emotion classes. This chapter starts with an introduction on biosignals for emotion detection. Next, a state-of-the-art review is presented on automatic emotion classification. Moreover, guidelines are presented for affective MMI. Subsequently, a research is presented that explores the use of EDA and three facial EMG signals to determine neutral, positive, negative, and mixed emotions, using recordings of 21 people. A range of techniques is tested, which resulted in a generic framework for automated emotion classification with up to 61.31% correct classification of the four emotion classes, without the need of personal profiles. Among various other directives for future research, the results emphasize the need for parallel processing of multiple biosignals
Time series morphological analysis applied to biomedical signals events detection
Dissertation submitted in the fufillment of the requirements for the Degree of Master in Biomedical EngineeringAutomated techniques for biosignal data acquisition and analysis have become increasingly powerful, particularly at the Biomedical Engineering research field. Nevertheless, it is verified the need to improve tools for signal pattern recognition and classification systems, in which the detection of specific events and the automatic signal segmentation are preliminary
processing steps.
The present dissertation introduces a signal-independent algorithm, which detects significant events in a biosignal. From a time series morphological analysis, the algorithm computes the instants when the most significant standard deviation discontinuities occur, segmenting the signal. An iterative optimization step is then applied. This assures that a minimal error is achieved when modeling these segments with polynomial regressions. The adjustment of a scale factor gives different detail levels of events detection.
An accurate and objective algorithm performance evaluation procedure was designed.
When applied on a set of synthetic signals, with known and quantitatively predefined events, an overall mean error of 20 samples between the detected and the actual events showed the high accuracy of the proposed algorithm. Its ability to perform the detection of signal activation onsets and transient waveshapes was also assessed, resulting in higher reliability than
signal-specific standard methods.
Some case studies, with signal processing requirements for which the developed algorithm can be suitably applied, were approached. The algorithm implementation in real-time, as part of an application developed during this research work, is also reported.
The proposed algorithm detects significant signal events with accuracy and significant
noise immunity. Its versatile design allows the application in different signals without previous knowledge on their statistical properties or specific preprocessing steps. It also brings added objectivity when compared with the exhaustive and time-consuming examiner analysis.
The tool introduced in this dissertation represents a relevant contribution in events detection, a particularly important issue within the wide digital biosignal processing research field
Connecting people through physiosocial technology
Social connectedness is one of the most important predictors of health and well-being. The goal of this dissertation is to investigate technologies that can support social connectedness. Such technologies can build upon the notion that disclosing emotional information has a strong positive influence on social connectedness. As physiological signals are strongly related to emotions, they might provide a solid base for emotion communication technologies. Moreover, physiological signals are largely lacking in unmediated communication, have been used successfully by machines to recognize emotions, and can be measured relatively unobtrusively with wearable sensors. Therefore, this doctoral dissertation examines the following research question: How can we use physiological signals in affective technology to improve social connectedness? First, a series of experiments was conducted to investigate if computer interpretations of physiological signals can be used to automatically communicate emotions and improve social connectedness (Chapters 2 and 3). The results of these experiments showed that computers can be more accurate at recognizing emotions than humans are. Physiological signals turned out to be the most effective information source for machine emotion recognition. One advantage of machine based emotion recognition for communication technology may be the increase in the rate at which emotions can be communicated. As expected, experiments showed that increases in the number of communicated emotions increased feelings of closeness between interacting people. Nonetheless, these effects on feelings of closeness are limited if users attribute the cause of the increases in communicated emotions to the technology and not to their interaction partner. Therefore, I discuss several possibilities to incorporate emotion recognition technologies in applications in such a way that users attribute the communication to their interaction partner. Instead of using machines to interpret physiological signals, the signals can also be represented to a user directly. This way, the interpretation of the signal is left to be done by the user. To explore this, I conducted several studies that employed heartbeat representations as a direct physiological communication signal. These studies showed that people can interpret such signals in terms of emotions (Chapter 4) and that perceiving someone's heartbeat increases feelings of closeness between the perceiver and sender of the signal (Chapter 5). Finally, we used a field study (Chapter 6) to investigate the potential of heartbeat communication mechanisms in practice. This again confirmed that heartbeat can provide an intimate connection to another person, showing the potential for communicating physiological signals directly to improve connectedness. The last part of the dissertation builds upon the notion that empathy has positive influences on social connectedness. Therefore, I developed a framework for empathic computing that employed automated empathy measurement based on physiological signals (Chapter 7). This framework was applied in a system that can train empathy (Chapter 8). The results showed that providing users frequent feedback about their physiological synchronization with others can help them to improve empathy as measured through self-report and physiological synchronization. In turn, this improves understanding of the other and helps people to signal validation and caring, which are types of communication that improve social connectedness. Taking the results presented in this dissertation together, I argue that physiological signals form a promising modality to apply in communication technology (Chapter 9). This dissertation provides a basis for future communication applications that aim to improve social connectedness
NON-VERBAL COMMUNICATION WITH PHYSIOLOGICAL SENSORS. THE AESTHETIC DOMAIN OF WEARABLES AND NEURAL NETWORKS
Historically, communication implies the transfer of information between bodies, yet this
phenomenon is constantly adapting to new technological and cultural standards. In a
digital context, it’s commonplace to envision systems that revolve around verbal modalities.
However, behavioural analysis grounded in psychology research calls attention to
the emotional information disclosed by non-verbal social cues, in particular, actions that
are involuntary. This notion has circulated heavily into various interdisciplinary computing
research fields, from which multiple studies have arisen, correlating non-verbal
activity to socio-affective inferences. These are often derived from some form of motion
capture and other wearable sensors, measuring the ‘invisible’ bioelectrical changes that
occur from inside the body.
This thesis proposes a motivation and methodology for using physiological sensory
data as an expressive resource for technology-mediated interactions. Initialised from a
thorough discussion on state-of-the-art technologies and established design principles
regarding this topic, then applied to a novel approach alongside a selection of practice
works to compliment this. We advocate for aesthetic experience, experimenting with
abstract representations. Atypically from prevailing Affective Computing systems, the
intention is not to infer or classify emotion but rather to create new opportunities for rich
gestural exchange, unconfined to the verbal domain.
Given the preliminary proposition of non-representation, we justify a correspondence
with modern Machine Learning and multimedia interaction strategies, applying an iterative,
human-centred approach to improve personalisation without the compromising
emotional potential of bodily gesture. Where related studies in the past have successfully
provoked strong design concepts through innovative fabrications, these are typically limited
to simple linear, one-to-one mappings and often neglect multi-user environments;
we foresee a vast potential. In our use cases, we adopt neural network architectures to
generate highly granular biofeedback from low-dimensional input data.
We present the following proof-of-concepts: Breathing Correspondence, a wearable
biofeedback system inspired by Somaesthetic design principles; Latent Steps, a real-time auto-encoder to represent bodily experiences from sensor data, designed for dance performance;
and Anti-Social Distancing Ensemble, an installation for public space interventions,
analysing physical distance to generate a collective soundscape. Key findings are
extracted from the individual reports to formulate an extensive technical and theoretical
framework around this topic. The projects first aim to embrace some alternative perspectives
already established within Affective Computing research. From here, these concepts
evolve deeper, bridging theories from contemporary creative and technical practices with
the advancement of biomedical technologies.Historicamente, os processos de comunicação implicam a transferência de informação
entre organismos, mas este fenómeno está constantemente a adaptar-se a novos padrões
tecnológicos e culturais. Num contexto digital, é comum encontrar sistemas que giram
em torno de modalidades verbais. Contudo, a análise comportamental fundamentada
na investigação psicológica chama a atenção para a informação emocional revelada por
sinais sociais não verbais, em particular, acções que são involuntárias. Esta noção circulou
fortemente em vários campos interdisciplinares de investigação na área das ciências da
computação, dos quais surgiram múltiplos estudos, correlacionando a actividade nãoverbal
com inferências sócio-afectivas. Estes são frequentemente derivados de alguma
forma de captura de movimento e sensores “wearable”, medindo as alterações bioeléctricas
“invisíveis” que ocorrem no interior do corpo.
Nesta tese, propomos uma motivação e metodologia para a utilização de dados sensoriais
fisiológicos como um recurso expressivo para interacções mediadas pela tecnologia.
Iniciada a partir de uma discussão aprofundada sobre tecnologias de ponta e princípios
de concepção estabelecidos relativamente a este tópico, depois aplicada a uma nova abordagem,
juntamente com uma selecção de trabalhos práticos, para complementar esta.
Defendemos a experiência estética, experimentando com representações abstractas. Contrariamente
aos sistemas de Computação Afectiva predominantes, a intenção não é inferir
ou classificar a emoção, mas sim criar novas oportunidades para uma rica troca gestual,
não confinada ao domínio verbal.
Dada a proposta preliminar de não representação, justificamos uma correspondência
com estratégias modernas de Machine Learning e interacção multimédia, aplicando uma
abordagem iterativa e centrada no ser humano para melhorar a personalização sem o
potencial emocional comprometedor do gesto corporal. Nos casos em que estudos anteriores
demonstraram com sucesso conceitos de design fortes através de fabricações
inovadoras, estes limitam-se tipicamente a simples mapeamentos lineares, um-para-um,
e muitas vezes negligenciam ambientes multi-utilizadores; com este trabalho, prevemos
um potencial alargado. Nos nossos casos de utilização, adoptamos arquitecturas de redes
neurais para gerar biofeedback altamente granular a partir de dados de entrada de baixa dimensão.
Apresentamos as seguintes provas de conceitos: Breathing Correspondence, um sistema
de biofeedback wearable inspirado nos princípios de design somaestético; Latent
Steps, um modelo autoencoder em tempo real para representar experiências corporais
a partir de dados de sensores, concebido para desempenho de dança; e Anti-Social Distancing
Ensemble, uma instalação para intervenções no espaço público, analisando a
distância física para gerar uma paisagem sonora colectiva. Os principais resultados são
extraídos dos relatórios individuais, para formular um quadro técnico e teórico alargado
para expandir sobre este tópico. Os projectos têm como primeiro objectivo abraçar algumas
perspectivas alternativas às que já estão estabelecidas no âmbito da investigação
da Computação Afectiva. A partir daqui, estes conceitos evoluem mais profundamente,
fazendo a ponte entre as teorias das práticas criativas e técnicas contemporâneas com o
avanço das tecnologias biomédicas
The Case for Public Interventions during a Pandemic
Funding Information: This work has been supported by Marie Skłodowska Curie Actions ITN AffecTech (ERC H2020 Project 1059 ID: 722022). Publisher Copyright: © 2022 by the authors.Within the field of movement sensing and sound interaction research, multi-user systems have gradually gained interest as a means to facilitate an expressive non-verbal dialogue. When tied with studies grounded in psychology and choreographic theory, we consider the qualities of interaction that foster an elevated sense of social connectedness, non-contingent to occupying one’s personal space. Upon reflection of the newly adopted social distancing concept, we orchestrate a technological intervention, starting with interpersonal distance and sound at the core of interaction. Materialised as a set of sensory face-masks, a novel wearable system was developed and tested in the context of a live public performance from which we obtain the user’s individual perspectives and correlate this with patterns identified in the recorded data. We identify and discuss traits of the user’s behaviour that were accredited to the system’s influence and construct four fundamental design considerations for physically distanced sound interaction. The study concludes with essential technical reflections, accompanied by an adaptation for a pervasive sensory intervention that is finally deployed in an open public space.publishersversionpublishe
Smart workplaces: a system proposal for stress management
Over the past last decades of contemporary society, workplaces
have become the primary source of many health issues, leading
to mental problems such as stress, depression, and anxiety.
Among the others, environmental aspects have shown to be the
causes of stress, illness, and lack of productivity. With the arrival
of new technologies, especially in the smart workplaces field,
most studies have focused on investigating the building energy
efficiency models and human thermal comfort. However, little has
been applied to occupants’ stress recognition and well-being
overall. Due to this fact, this present study aims to propose a
stress management solution for an interactive design system that
allows the adapting of comfortable environmental conditions
according to the user preferences by measuring in real-time the
environmental and biological characteristics, thereby helping to
prevent stress, as well as to enable users to cope stress when
being stressed. The secondary objective will focus on evaluating
one part of the system: the mobile application. The proposed
system uses several usability methods to identify users’ needs,
behavior, and expectations from the user-centered design
approach. Applied methods, such as User Research, Card
Sorting, and Expert Review, allowed us to evaluate the design
system according to Heuristics Analysis, resulting in improved
usability of interfaces and experience. The study presents the
research results, the design interface, and usability tests.
According to the User Research results, temperature and noise
are the most common environmental stressors among the users
causing stress and uncomfortable conditions to work in, and the
preference for physical activities over the digital solutions for
coping with stress. Additionally, the System Usability Scale (SUS)
results identified that the system’s usability was measured as
“excellent” and “acceptable” with a final score of 88 points out of
the 100. It is expected that these conclusions can contribute to
future investigations in the smart workplaces study field and their
interaction with the people placed there.Nas últimas décadas da sociedade contemporânea, o local de
trabalho tem se tornado principal fonte de muitos problemas de
saúde mental, como o stress, depressão e ansiedade. Os aspetos
ambientais têm se revelado como as causas de stress, doenças,
falta de produtividade, entre outros. Atualmente, com a chegada de
novas tecnologias, principalmente na área de locais de trabalho
inteligentes, a maioria dos estudos tem se concentrado na
investigação de modelos de eficiência energética de edifícios e
conforto térmico humano. No entanto, pouco foi aplicado ao
reconhecimento do stress dos ocupantes e ao bem-estar geral das
pessoas. Diante disso, o objetivo principal é propor um sistema de
design de gestão do stress para um sistema de design interativo que
permita adaptar as condições ambientais de acordo com as
preferências de utilizador, medindo em tempo real as características
ambientais e biológicas, auxiliando assim na prevenção de stress,
bem como ajuda os utilizadores a lidar com o stress quando estão
sob o mesmo. O segundo objetivo é desenhar e avaliar uma parte
do projeto — o protótipo da aplicação móvel através da realização
de testes de usabilidade. O sistema proposto resulta da abordagem
de design centrado no utilizador, utilizando diversos métodos de
usabilidade para identificar as necessidades, comportamentos e as
expectativas dos utilizadores. Métodos aplicados, como Pesquisa de
Usuário, Card Sorting e Revisão de Especialistas, permitiram avaliar
o sistema de design de acordo com a análise heurística, resultando
numa melhoria na usabilidade das interfaces e experiência. O
estudo apresenta os resultados da pesquisa, a interface do design e
os testes de usabilidade. De acordo com os resultados de User
Research, a temperatura e o ruído são os stressores ambientais
mais comuns entre os utilizadores, causando stresse e condições
menos favoráveis para trabalhar, igualmente existe uma preferência
por atividades físicas sobre as soluções digitais na gestão do
stresse. Adicionalmente, os resultados de System Usability Scale
(SUS) identificaram a usabilidade do sistema de design como
“excelente” e “aceitável” com pontuação final de 88 pontos em 100.
É esperado que essas conclusões possam contribuir para futuras
investigações no campo de estudo dos smart workplaces e sua
interação com os utilizadores
No-estacionariedad, multifractalidad y limpieza de ruido en señales reales
Las señales biomédicas, como el electrocardiograma, el electroencefalograma, o la señal de voz, tienen en común características de no estacionariedad y no linealidad. Aunque enmuchas aplicaciones se considera que se trata de señales estacionarias procedentes de sistemas lineales, ésta simplificación constituye una hipótesis de trabajo válida sólo como una aproximación que permite la aplicación de técnicas clásicas deanálisis de señales. Muchos trastornos que afectan a uno o varios órganos pueden ser detectados a través de un correcto análisis de las señales en cuya producción están involucrados. Sin embargo, debe atenderse al hecho de que una señal procedente de un sistema patológico se aleja aún más de las condiciones hipotéticas de estacionariedad y linealidad. Se desprende de esta circunstancia la necesidad de abordar el análisis de las señales biomédicas mediante técnicas no convencionales que permitan su tratamiento en un marco que tenga en cuenta sus características de no estacionariedad y no linealidad. Sobre la base de la experiencia del grupo de trabajo en las áreas del análisis tiempo-frecuencia/escala, análisis y modelado estadístico, análisis multifractal, complejidad y métodos guiados por los datos (adaptativos), a partir de problemas reales se han propuesto y estudiado nuevas técnicas que posibiliten su solución
Emotion and Stress Recognition Related Sensors and Machine Learning Technologies
This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective
Intelligent Biosignal Analysis Methods
This book describes recent efforts in improving intelligent systems for automatic biosignal analysis. It focuses on machine learning and deep learning methods used for classification of different organism states and disorders based on biomedical signals such as EEG, ECG, HRV, and others
- …