Supplementary Material for "Continuous Sonification Enhances Adequacy of Interactions in Peripheral Process Monitoring"

Hildebrandt T, Hermann T, Rinderle-Ma S. Supplementary Material for "Continuous Sonification Enhances Adequacy of Interactions in Peripheral Process Monitoring". Bielefeld University; 2016.As many users who are charged with process monitoring need to focus mainly on other work while performing monitoring as a secondary task, monitoring systems that purely rely on visual means are often not well suited for this purpose. Sonification, the presentation of data as (non-speech) sound, has proven in several studies that it can help in guiding the user's attention, especially in scenarios where process monitoring is performed in parallel with a different, main task. However, there are several aspects that have not been investigated in this area so far, for example if a continuous soundscape can guide the user's attention better than one that is based on auditory cues. We have developed a system that allows reproducible research to answer such questions. In this system, the participants' performance both for the main task (simulated by simple arithmetic problems) and for the secondary task (a simulation of a production process) can be measured in a more fine-grained manner than has been the case for existing research in this field. In a within-subject study (n=18), we compared three monitoring conditions - visual only, visual + auditory alerts and a condition combining the visual mode with continuous sonification of process events based on a forest soundscape. Participants showed significantly higher process monitoring performances in the continuous sonification condition, compared to the other two modes. The performance in the main task was at the same time not significantly affected by the continuous sonification

ECG sonification to support the diagnosis and monitoring of myocardial infarction

Aldana Blanco AL, Grautoff S, Hermann T. ECG Sonification to support the diagnosis and monitoring of myocardial infarction. Journal on Multimodal User Interfaces. 2020;14:207-218.This paper presents the design and evaluation of four sonification methods to support monitoring and diagnosis in Electrocardiography (ECG). In particular we focus on an ECG abnormality called ST-elevation which is an important indicator of a myocardial infarction. Since myocardial infarction represents a life-threatening condition it is of essential value to detect an ST-elevation as early as possible. As part of the evaluated sound designs, we propose two novel sonifications: (i) Polarity sonification, a continuous parameter-mapping sonification using a formant synthesizer and (ii) Stethoscope sonification, a combination of the ECG signal and a stethoscope recording. The other two designs, (iii) the water ambience sonification and the (iv) morph sonification, were presented in our previous work about ECG sonification [ref]. The study evaluates three components across the proposed sonifications (1) detection performance, meaning if participants are able to detect a transition from healthy to unhealthy states, (2) classification accuracy, that evaluates if participants can accurately classify the severity of the pathology, and (3) aesthetics and usability (pleasantness, informativeness and long-term listening). The study results show that the polarity design had the highest accuracy rates in the detection task whereas the stethoscope sonification obtained the better score in the classification assignment. Concerning aesthetics, the water ambience sonification was regarded as the most pleasant. Furthermore, we found a significant difference between sound/music experts and non-experts in terms of the error rates obtained in the detection task using the morph sonification and also in the classification task using the stethoscope sonification. Overall, the group of experts obtained lower error rates than the group of non-experts, which means that further training could improve accuracy rates and, particularly for designs that rely mainly on pitch variations, additional training is needed in the non-experts group

Sonification as Negotiation - Learning from Translation Studies.

This paper introduces a first comparison between the re-search domains of translation studies and data sonification.This contribution explores the idea of considering the prac-tice of sonification as an hermeneutic motion which entailsthe transfer of information across different media. Sonifi-cation is then envisioned as an adaptation concerned withthe transfer of incoming data into sonic forms. Transla-tion theories are used to reflect on various sonification ap-proaches: three translation perspectives are discussed andcompared to different sonification scenarios. The notionof negotiation is suggested to frame the translation of datainto sound as a process by which the designer mediates be-tween the source data and the target sound

Sonification in security operations centres: what do security practitioners think?

In Security Operations Centres (SOCs) security practitioners work using a range of tools to detect and mitigate malicious computer-network activity. Sonification, in which data is represented as sound, is said to have potential as an approach to addressing some of the unique challenges faced by SOCs. For example, sonification has been shown to enable peripheral monitoring of processes, which could aid practitioners multitasking in busy SOCs. The perspectives of security practitioners on incorporating sonification into their actual working environments have not yet been examined, however. The aim of this paper therefore is to address this gap by exploring attitudes to using sonification in SOCs. We report on the results of a study consisting of an online survey (N=20) and interviews (N=21) with security practitioners working in a range of different SOCs. Our contribution is a refined appreciation of the contexts in which sonification could aid in SOC working practice, and an understanding of the areas in which sonification may not be beneficial or may even be problematic.We also analyse the critical requirements for the design of sonification systems and their integration into the SOC setting. Our findings clarify insights into the potential benefits and challenges of introducing sonification to support work in this vital security-monitoring environment

A Formalised Approach to Designing Sonification Systems for Network-Security Monitoring

Sonification systems, in which data are represented through sound, have the potential to be useful in a number of network-security monitoring applications in Security Operations Centres (SOCs). Security analysts working in SOCs generally monitor networks using a combination of anomaly-detection techniques, Intrusion Detection Systems and data presented in visual and text-based forms. In the last two decades significant progress has been made in developing novel sonification systems to further support network-monitoring tasks, but many of these systems have not been sufficiently validated, and there is a lack of uptake in SOCs. Furthermore, little guidance exists on design requirements for the sonification of network data. In this paper, we identify the key role that sonification, if implemented correctly, could play in addressing shortcomings of traditional network-monitoring methods. Based on a review of prior research, we propose an approach to developing sonification systems for network monitoring. This approach involves the formalisation of a model for designing sonifications in this space; identification of sonification design aesthetics suitable for realtime network monitoring; and system refinement and validation through comprehensive user testing. As an initial step in this system development, we present a formalised model for designing sonifications for network-security monitoring. The application of this model is demonstrated through our development of prototype sonification systems for two different use-cases within network security monitoring

Data-to-music sonification and user engagement

The process of transforming data into sounds for auditory display provides unique user experiences and new perspectives for analyzing and interpreting data. A research study for data transformation to sounds based on musical elements, called data-to-music sonification, reveals how musical characteristics can serve analytical purposes with enhanced user engagement. An existing user engagement scale has been applied to measure engagement levels in three conditions within melodic, rhythmic, and chordal contexts. This article reports findings from a user engagement study with musical traits and states the benefits and challenges of using musical characteristics in sonifications. The results can guide the design of future sonifications of multivariable data

Human-centred design of clinical auditory alarms

Auditory alarms are commonly badly designed, providing little to no information or guidance. In the healthcare context, the poor acoustics of alarms is one contributor for the noise problem. The goal of this thesis is to propose a human-centred methodology for the design of clinical auditory alarms, by making them less disruptive and more informative, thus improving the healthcare soundscape. It implements this methodology from concept to evaluation and validation, combining psychoacoustics with usability and user experience methods. Another aim of this research consisted in understanding the limitations and possibilities offered by online tools for scientific studies. Thus, different processes and methodologies were implemented, and corresponding results were discussed. To understand the acoustic healthcare environment, field visits, interviews, and surveys were performed with healthcare professionals. Additionally, sound pressure levels and frequency analysis of several surgeries in different hospitals provided specific sound design requirements, which were added to an existent body of knowledge on clinical alarm design. A second stage consisted in prototyping very simple sounds to comprehend which temporal and spectral parameters of sound could be manipulated to communicate clinical information. Parameters such as frequency, speed, onset, and rhythm were studied, and relations between subjective perception and physical parameters were established. In parallel, and heavily influenced by the new IEC 60601-1-8 - General requirements, tests and guidance for alarm systems in medical electrical equipment and medical electrical systems, a design strategy with auditory icons was created. This strategy intended to provide as much information as possible in an auditory alarm. To do so, it involved two main components: a priority pointer indicating the priority of the alarm; an auditory icon indicating the cause of the alarm. A third component indicating increasing or decreasing tendency of the vital sign was designed, but not validated with users. After online validation of the priority pointer and auditory icon for eight categories (cardiac, drug administration, ventilation, blood pressure, perfusion, oxygen, temperature, and power down), a new library of clinical auditory alarms is proposed.Os alarmes auditivos são habitualmente mal concebidos, dando poucas informações ou orientações perante a situação que despoletou o aviso. No contexto da saúde, a má acústica dos alarmes é um dos contribuidores para o problema do ruído. O objetivo desta tese é o de melhorar a paisagem sonora em ambientes clínicos, propondo uma metodologia centrada no Humano para o design de alarmes auditivos clínicos, tornando-os menos disruptivos e mais informativos. Essa metodologia é implementada desde o conceito até a avaliação e validação, combinando métodos da psicoacústica com métodos de usabilidade e experiência do utilizador. Outro objetivo desta investigação é o de compreender as limitações e possibilidades oferecidas pelas ferramentas online para estudos científicos. Assim, diversos processos e metodologias foram implementados, e os respetivos resultados são discutidos. Para compreender o ambiente acústico clínico, foram realizadas visitas de campo, entrevistas e inquéritos com profissionais de saúde. Além disso, avaliou-se o nível de pressão sonora e frequências de várias cirurgias em diferentes hospitais. Esta atividade forneceu requisitos específicos de design de som que foram adicionados a um corpo existente de conhecimento sobre design de alarmes clínicos. Uma segunda etapa consistiu na prototipagem de sons simples para compreender que parâmetros temporais e espectrais do som poderiam ser manipulados para comunicar informações clínicas. Parâmetros como frequência, velocidade, envelope e ritmo foram estudados, e as relações entre a perceção subjetiva e os parâmetros físicos foram estabelecidas. Paralelamente, e fortemente influenciado pela nova norma IEC 60601-1-8 - Requisitos gerais, testes e orientações para sistemas de alarme em equipamentos médicos elétricos e sistemas médicos elétricos, foi criada uma estratégia de design com ícones auditivos. Essa estratégia pretendia incorporar o máximo de informações num alarme auditivo. Para isso, envolveu dois componentes principais: um ponteiro de prioridade que indica a prioridade do alarme; e um ícone auditivo que indica a causa do alarme. Um terceiro componente de tendência (aumento ou diminuição do valor do sinal vital) foi criado, mas não validado com utilizadores. Após a validação do ponteiro de prioridade e ícone auditivo para oito categorias (cardíaco, administração de medicamentos, ventilação, pressão arterial, perfusão, oxigénio, temperatura e falha de equipamento), propõe-se uma nova biblioteca de alarmes auditivos clínicos

Amplifying Actions - Towards Enactive Sound Design

Recently, artists and designers have begun to use digital technologies in order to stimulate bodily interaction, while scientists keep revealing new findings about sensorimotor contingencies, changing the way in which we understand human knowledge. However, implicit knowledge generated in artistic projects can become difficult to transfer and scientific research frequently remains isolated due to specific disciplinary languages and methodologies. By mutually enriching holistic creative approaches and highly specific scientific ways of working, this doctoral dissertation aims to set the foundation for Enactive Sound Design. It is focused on sound that engages sensorimotor experience that has been neglected within the existing design practices. The premise is that such a foundation can be best developed if grounded in transdisciplinary methods that bring together scientific and design approaches. The methodology adopted to achieve this goal is practice-based and supported by theoretical research and project analysis. Three different methodologies were formulated and evaluated during this doctoral study, based on a convergence of existing methods from design, psychology and human-computer interaction. First, a basic design approach was used to engage in a reflective creation process and to extend the existing work on interaction gestalt through hands-on activities. Second, psychophysical experiments were carried out and adapted to suit the needed shift from reception-based tests to a performance-based quantitative evaluation. Last, a set of participatory workshops were developed and conducted, within which the enactive sound exercises were iteratively tested through direct and participatory observation, questionnaires and interviews. A foundation for Enactive Sound Design developed in this dissertation includes novel methods that have been generated by extensive explorations into the fertile ground between basic design education, psychophysical experiments and participatory design. Combining creative practices with traditional task analysis further developed this basic design approach. The results were a number of abstract sonic artefacts conceptualised as the experimental apparatuses that can allow psychologists to study enactive sound experience. Furthermore, a collaboration between designers and scientists on a psychophysical study produced a new methodology for the evaluation of sensorimotor performance with tangible sound interfaces.These performance experiments have revealed that sonic feedback can support enactive learning. Finally, participatory workshops resulted in a number of novel methods focused on a holistic perspective fostered through a subjective experience of self-producing sound. They indicated the influence that such an approach may have on both artists and scientists in the future. The role of designer, as a scientific collaborator within psychological research and as a facilitator of participatory workshops, has been evaluated. Thus, this dissertation recommends a number of collaborative methods and strategies that can help designers to understand and reflectively create enactive sound objects. It is hoped that the examples of successful collaborations between designers and scientists presented in this thesis will encourage further projects and connections between different disciplines, with the final goal of creating a more engaging and a more aware sonic future.European Commission 6th Framework and European Science Foundation (COST Action

Diagnosis and Treatment of Small Bowel Disorders

Over the last few decades, remarkable progress has been made in understanding the aetiology and pathophysiology of diseases and many new theories emphasize the importance of the small bowel ‘ecosystem’ in the pathogenesis of acute and chronic illness. Emerging factors such as microbiome, stem cells, innate intestinal immunity and the enteric nervous system along with mucosal and endothelial barriers have key role in the development of gastrointestinal and extra-intestinal diseases. Therefore, the small intestine is considered key player in metabolic disease development, including diabetes mellitus, and other diet-related disorders such as celiac and non-celiac enteropathies. Another major field is drug metabolism and its interaction with microbiota. Moreover, the emergence of gut-brain, gut-liver and gut-blood barriers points toward the important role of small intestine in the pathogenesis of common disorders, such as liver disease, hypertension and neurodegenerative disease. However, the small bowel remains an organ that is difficult to fully access and assess and accurate diagnosis often poses a clinical challenge. Eventually, the therapeutic potential remains untapped. Therefore, it is due time to direct our interest towards the small intestine and unravel the interplay between small-bowel and other gastrointestinal (GI) and non-GI related maladies