7 research outputs found
Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts
This thesis’s primary goal is to investigate performance ecologies, that is the compound
of humans, artifacts and environmental elements that contribute to the result of a per-
formance. In particular, this thesis focuses on designing new interactive technologies for
sound and music. The goal of this thesis leads to the following Research Questions (RQs):
• RQ1 How can the design of interactive sonic artifacts support a joint expression
across different actors (composers, choreographers, and performers, musicians, and
dancers) in a given performance ecology?
• RQ2 How does each different actor influence the design of different artifacts, and
what impact does this have on the overall artwork?
• RQ3 How do the different actors in the same ecology interact, and appropriate an
interactive artifact?
To reply to these questions, a new framework named ARCAA has been created. In this
framework, all the Actors of a given ecology are connected to all the Artifacts throughout
three layers: Role, Context and Activity. This framework is then applied to one systematic
literature review, two case studies on music performance and one case study in dance
performance. The studies help to better understand the shaded roles of composers, per-
formers, instrumentalists, dancers, and choreographers, which is relevant to better design
interactive technologies for performances. Finally, this thesis proposes a new reflection on
the blurred distinction between composing and designing a new instrument in a context
that involves a multitude of actors.
Overall, this work introduces the following contributions to the field of interaction
design applied to music technology: 1) ARCAA, a framework to analyse the set of inter-
connected relationship in interactive (music) performances, validated through 2 music
studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de-
signing music interactive system for performance (music or dance), accounting for the
needs of the various actors and for the overlapping on music composition and design of in-
teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings;
4) Proposal of a methodological approach combining autobiographical and idiographical
design approaches in interactive performances.O objetivo principal desta tese é investigar as ecologias performativas, conjunto formado
pelos participantes humanos, artefatos e elementos ambientais que contribuem para o
resultado de uma performance. Em particular, esta tese foca-se na conceção de novas
tecnologias interativas para som e música. O objetivo desta tese originou as seguintes
questões de investigação (Research Questions RQs):
• RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con-
junta entre diferentes atores (compositores, coreógrafos e performers, músicos e
dançarinos) numa determinada ecologia performativa?
• RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso
tem no trabalho artístico global?
• RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam
de um artefato interativo?
Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA.
Nesta framework, todos os atores (Actores) de uma dada ecologia estão conectados a todos
os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework
foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre
performance musical e a um estudo de caso em performance de dança. Estes estudos aju-
daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes,
instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec-
nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre
a distinção entre compor e projetar um novo instrumento num contexto que envolve uma
multiplicidade de atores.
Este trabalho apresenta as seguintes contribuições principais para o campo do design
de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o
conjunto de relações interconectadas em performances interativas, validado através de
dois estudos de caso relacionados com a música, um estudo de caso relacionado com
a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as
necessidades dos vários atores e a sobreposição entre a composição musical e o design de
tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram
as ecologias performativas no NIME, com base numa análise sistemática da literatura
dos artigos apresentados e publicados nestas conferência; 4) Proposta de uma aborda-
gem metodológica combinando abordagens de design autobiográfico e idiográfico em
performances interativas
Self-managed Workflows for Cyber-physical Systems
Workflows are a well-established concept for describing business logics and processes in web-based applications and enterprise application integration scenarios on an abstract implementation-agnostic level. Applying Business Process Management (BPM) technologies to increase autonomy and automate sequences of activities in Cyber-physical Systems (CPS) promises various advantages including a higher flexibility and simplified programming, a more efficient resource usage, and an easier integration and orchestration of CPS devices. However, traditional BPM notations and engines have not been designed to be used in the context of CPS, which raises new research questions occurring with the close coupling of the virtual and physical worlds. Among these challenges are the interaction with complex compounds of heterogeneous sensors, actuators, things and humans; the detection and handling of errors in the physical world; and the synchronization of the cyber-physical process execution models. Novel factors related to the interaction with the physical world including real world obstacles, inconsistencies and inaccuracies may jeopardize the successful execution of workflows in CPS and may lead to unanticipated situations.
This thesis investigates properties and requirements of CPS relevant for the introduction of BPM technologies into cyber-physical domains. We discuss existing BPM systems and related work regarding the integration of sensors and actuators into workflows, the development of a Workflow Management System (WfMS) for CPS, and the synchronization of the virtual and physical process execution as part of self-* capabilities for WfMSes. Based on the identified research gap, we present concepts and prototypes regarding the development of a CPS WFMS w.r.t. all phases of the BPM lifecycle. First, we introduce a CPS workflow notation that supports the modelling of the interaction of complex sensors, actuators, humans, dynamic services and WfMSes on the business process level. In addition, the effects of the workflow execution can be specified in the form of goals defining success and error criteria for the execution of individual process steps. Along with that, we introduce the notion of Cyber-physical Consistency. Following, we present a system architecture for a corresponding WfMS (PROtEUS) to execute the modelled processes-also in distributed execution settings and with a focus on interactive process management. Subsequently, the integration of a cyber-physical feedback loop to increase resilience of the process execution at runtime is discussed. Within this MAPE-K loop, sensor and context data are related to the effects of the process execution, deviations from expected behaviour are detected, and compensations are planned and executed. The execution of this feedback loop can be scaled depending on the required level of precision and consistency. Our implementation of the MAPE-K loop proves to be a general framework for adding self-* capabilities to WfMSes. The evaluation of our concepts within a smart home case study shows expected behaviour, reasonable execution times, reduced error rates and high coverage of the identified requirements, which makes our CPS~WfMS a suitable system for introducing workflows on top of systems, devices, things and applications of CPS.:1. Introduction 15
1.1. Motivation 15
1.2. Research Issues 17
1.3. Scope & Contributions 19
1.4. Structure of the Thesis 20
2. Workflows and Cyber-physical Systems 21
2.1. Introduction 21
2.2. Two Motivating Examples 21
2.3. Business Process Management and Workflow Technologies 23
2.4. Cyber-physical Systems 31
2.5. Workflows in CPS 38
2.6. Requirements 42
3. Related Work 45
3.1. Introduction 45
3.2. Existing BPM Systems in Industry and Academia 45
3.3. Modelling of CPS Workflows 49
3.4. CPS Workflow Systems 53
3.5. Cyber-physical Synchronization 58
3.6. Self-* for BPM Systems 63
3.7. Retrofitting Frameworks for WfMSes 69
3.8. Conclusion & Deficits 71
4. Modelling of Cyber-physical Workflows with Consistency Style Sheets 75
4.1. Introduction 75
4.2. Workflow Metamodel 76
4.3. Knowledge Base 87
4.4. Dynamic Services 92
4.5. CPS-related Workflow Effects 94
4.6. Cyber-physical Consistency 100
4.7. Consistency Style Sheets 105
4.8. Tools for Modelling of CPS Workflows 106
4.9. Compatibility with Existing Business Process Notations 111
5. Architecture of a WfMS for Distributed CPS Workflows 115
5.1. Introduction 115
5.2. PROtEUS Process Execution System 116
5.3. Internet of Things Middleware 124
5.4. Dynamic Service Selection via Semantic Access Layer 125
5.5. Process Distribution 126
5.6. Ubiquitous Human Interaction 130
5.7. Towards a CPS WfMS Reference Architecture for Other Domains 137
6. Scalable Execution of Self-managed CPS Workflows 141
6.1. Introduction 141
6.2. MAPE-K Control Loops for Autonomous Workflows 141
6.3. Feedback Loop for Cyber-physical Consistency 148
6.4. Feedback Loop for Distributed Workflows 152
6.5. Consistency Levels, Scalability and Scalable Consistency 157
6.6. Self-managed Workflows 158
6.7. Adaptations and Meta-adaptations 159
6.8. Multiple Feedback Loops and Process Instances 160
6.9. Transactions and ACID for CPS Workflows 161
6.10. Runtime View on Cyber-physical Synchronization for Workflows 162
6.11. Applicability of Workflow Feedback Loops to other CPS Domains 164
6.12. A Retrofitting Framework for Self-managed CPS WfMSes 165
7. Evaluation 171
7.1. Introduction 171
7.2. Hardware and Software 171
7.3. PROtEUS Base System 174
7.4. PROtEUS with Feedback Service 182
7.5. Feedback Service with Legacy WfMSes 213
7.6. Qualitative Discussion of Requirements and Additional CPS Aspects 217
7.7. Comparison with Related Work 232
7.8. Conclusion 234
8. Summary and Future Work 237
8.1. Summary and Conclusion 237
8.2. Advances of this Thesis 240
8.3. Contributions to the Research Area 242
8.4. Relevance 243
8.5. Open Questions 245
8.6. Future Work 247
Bibliography 249
Acronyms 277
List of Figures 281
List of Tables 285
List of Listings 287
Appendices 28
The Meeting Journey: Supporting the Design of Interaction Within Co-located, Collaborative Device Ecologies
The combination of personal and Bring Your Own Device (BYOD) technologies with augmented "smart" spaces designed for collaborative work offers new design challenges for the HCI community. This thesis looks at how BYOD technologies can integrate with such spaces in the context of cross-channel collaboration, from a user experience (UX) perspective. In addition to this, the work looks at how BYOD technologies can replace smart spaces entirely and form space-agnostic collaborative device ecologies. A series of qualitative empirical studies were undertaken that led to the development of the concept of a meeting journey. The meeting journey is an abstract representation of the different steps, tools and activities undertaken by users in the context of a co- located collaborative activity aimed to inform the design of such systems. The meeting journey helped define a series of design principles for collaborative device ecologies. Whilst the longitudinal aspects of collaboration have been well covered by work in the area of Computer Supported Collaborative Work (CSCW) and supported by a number of commercial products, the UX of co-located ad hoc meetings have not. Such meetings are still characterised by difficult and awkward interactions due to different technologies, levels of computer literacy and processes. The concept proposed to improve the desirability of collaborative device ecologies following the design principles previously defined is based on a hybrid approach, built on an extensible framework nicknamed “OIL”. The sharing activities specific to ad hoc collaborative meetings are delegated to a consistent user interface, whilst the ecology retains a platform-agnostic philosophy as to which applications and devices are used by the participants. The final part of the thesis relates the development of a proof-of-concept hybrid system inspired by OIL, and its evaluation using desirability metrics. This leads to a discussion of the possibilities of extensions to the proof of concept, including support for more steps of the meeting journey, a broader set of functionalities, and a broader range of issues such as security and data ownership