Design and Real-World Evaluation of Dependable Wireless Cyber-Physical Systems

The ongoing effort for an efficient, sustainable, and automated interaction between humans, machines, and our environment will make cyber-physical systems (CPS) an integral part of the industry and our daily lives. At their core, CPS integrate computing elements, communication networks, and physical processes that are monitored and controlled through sensors and actuators. New and innovative applications become possible by extending or replacing static and expensive cable-based communication infrastructures with wireless technology. The flexibility of wireless CPS is a key enabler for many envisioned scenarios, such as intelligent factories, smart farming, personalized healthcare systems, autonomous search and rescue, and smart cities. High dependability, efficiency, and adaptivity requirements complement the demand for wireless and low-cost solutions in such applications. For instance, industrial and medical systems should work reliably and predictably with performance guarantees, even if parts of the system fail. Because emerging CPS will feature mobile and battery-driven devices that can execute various tasks, the systems must also quickly adapt to frequently changing conditions. Moreover, as applications become ever more sophisticated, featuring compact embedded devices that are deployed densely and at scale, efficient designs are indispensable to achieve desired operational lifetimes and satisfy high bandwidth demands. Meeting these partly conflicting requirements, however, is challenging due to imperfections of wireless communication and resource constraints along several dimensions, for example, computing, memory, and power constraints of the devices. More precisely, frequent and correlated message losses paired with very limited bandwidth and varying delays for the message exchange significantly complicate the control design. In addition, since communication ranges are limited, messages must be relayed over multiple hops to cover larger distances, such as an entire factory. Although the resulting mesh networks are more robust against interference, efficient communication is a major challenge as wireless imperfections get amplified, and significant coordination effort is needed, especially if the networks are dynamic. CPS combine various research disciplines, which are often investigated in isolation, ignoring their complex interaction. However, to address this interaction and build trust in the proposed solutions, evaluating CPS using real physical systems and wireless networks paired with formal guarantees of a system’s end-to-end behavior is necessary. Existing works that take this step can only satisfy a few of the abovementioned requirements. Most notably, multi-hop communication has only been used to control slow physical processes while providing no guarantees. One of the reasons is that the current communication protocols are not suited for dynamic multi-hop networks. This thesis closes the gap between existing works and the diverse needs of emerging wireless CPS. The contributions address different research directions and are split into two parts. In the first part, we specifically address the shortcomings of existing communication protocols and make the following contributions to provide a solid networking foundation: • We present Mixer, a communication primitive for the reliable many-to-all message exchange in dynamic wireless multi-hop networks. Mixer runs on resource-constrained low-power embedded devices and combines synchronous transmissions and network coding for a highly scalable and topology-agnostic message exchange. As a result, it supports mobile nodes and can serve any possible traffic patterns, for example, to efficiently realize distributed control, as required by emerging CPS applications. • We present Butler, a lightweight and distributed synchronization mechanism with formally guaranteed correctness properties to improve the dependability of synchronous transmissions-based protocols. These protocols require precise time synchronization provided by a specific node. Upon failure of this node, the entire network cannot communicate. Butler removes this single point of failure by quickly synchronizing all nodes in the network without affecting the protocols’ performance. In the second part, we focus on the challenges of integrating communication and various control concepts using classical time-triggered and modern event-based approaches. Based on the design, implementation, and evaluation of the proposed solutions using real systems and networks, we make the following contributions, which in many ways push the boundaries of previous approaches: • We are the first to demonstrate and evaluate fast feedback control over low-power wireless multi-hop networks. Essential for this achievement is a novel co-design and integration of communication and control. Our wireless embedded platform tames the imperfections impairing control, for example, message loss and varying delays, and considers the resulting key properties in the control design. Furthermore, the careful orchestration of control and communication tasks enables real-time operation and makes our system amenable to an end-to-end analysis. Due to this, we can provably guarantee closed-loop stability for physical processes with linear time-invariant dynamics. • We propose control-guided communication, a novel co-design for distributed self-triggered control over wireless multi-hop networks. Self-triggered control can save energy by transmitting data only when needed. However, there are no solutions that bring those savings to multi-hop networks and that can reallocate freed-up resources, for example, to other agents. Our control system informs the communication system of its transmission demands ahead of time so that communication resources can be allocated accordingly. Thus, we can transfer the energy savings from the control to the communication side and achieve an end-to-end benefit. • We present a novel co-design of distributed control and wireless communication that resolves overload situations in which the communication demand exceeds the available bandwidth. As systems scale up, featuring more agents and higher bandwidth demands, the available bandwidth will be quickly exceeded, resulting in overload. While event-triggered control and self-triggered control approaches reduce the communication demand on average, they cannot prevent that potentially all agents want to communicate simultaneously. We address this limitation by dynamically allocating the available bandwidth to the agents with the highest need. Thus, we can formally prove that our co-design guarantees closed-loop stability for physical systems with stochastic linear time-invariant dynamics.:Abstract Acknowledgements List of Abbreviations List of Figures List of Tables 1 Introduction 1.1 Motivation 1.2 Application Requirements 1.3 Challenges 1.4 State of the Art 1.5 Contributions and Road Map 2 Mixer: Efficient Many-to-All Broadcast in Dynamic Wireless Mesh Networks 2.1 Introduction 2.2 Overview 2.3 Design 2.4 Implementation 2.5 Evaluation 2.6 Discussion 2.7 Related Work 3 Butler: Increasing the Availability of Low-Power Wireless Communication Protocols 3.1 Introduction 3.2 Motivation and Background 3.3 Design 3.4 Analysis 3.5 Implementation 3.6 Evaluation 3.7 Related Work 4 Feedback Control Goes Wireless: Guaranteed Stability over Low-Power Multi-Hop Networks 4.1 Introduction 4.2 Related Work 4.3 Problem Setting and Approach 4.4 Wireless Embedded System Design 4.5 Control Design and Analysis 4.6 Experimental Evaluation 4.A Control Details 5 Control-Guided Communication: Efficient Resource Arbitration and Allocation in Multi-Hop Wireless Control Systems 5.1 Introduction 5.2 Problem Setting 5.3 Co-Design Approach 5.4 Wireless Communication System Design 5.5 Self-Triggered Control Design 5.6 Experimental Evaluation 6 Scaling Beyond Bandwidth Limitations: Wireless Control With Stability Guarantees Under Overload 6.1 Introduction 6.2 Problem and Related Work 6.3 Overview of Co-Design Approach 6.4 Predictive Triggering and Control System 6.5 Adaptive Communication System 6.6 Integration and Stability Analysis 6.7 Testbed Experiments 6.A Proof of Theorem 4 6.B Usage of the Network Bandwidth for Control 7 Conclusion and Outlook 7.1 Contributions 7.2 Future Directions Bibliography List of Publication

On Age-of-Information Aware Resource Allocation for Industrial Control-Communication-Codesign

Unter dem Überbegriff Industrie 4.0 wird in der industriellen Fertigung die zunehmende Digitalisierung und Vernetzung von industriellen Maschinen und Prozessen zusammengefasst. Die drahtlose, hoch-zuverlässige, niedrig-latente Kommunikation (engl. ultra-reliable low-latency communication, URLLC) – als Bestandteil von 5G gewährleistet höchste Dienstgüten, die mit industriellen drahtgebundenen Technologien vergleichbar sind und wird deshalb als Wegbereiter von Industrie 4.0 gesehen. Entgegen diesem Trend haben eine Reihe von Arbeiten im Forschungsbereich der vernetzten Regelungssysteme (engl. networked control systems, NCS) gezeigt, dass die hohen Dienstgüten von URLLC nicht notwendigerweise erforderlich sind, um eine hohe Regelgüte zu erzielen. Das Co-Design von Kommunikation und Regelung ermöglicht eine gemeinsame Optimierung von Regelgüte und Netzwerkparametern durch die Aufweichung der Grenze zwischen Netzwerk- und Applikationsschicht. Durch diese Verschränkung wird jedoch eine fundamentale (gemeinsame) Neuentwicklung von Regelungssystemen und Kommunikationsnetzen nötig, was ein Hindernis für die Verbreitung dieses Ansatzes darstellt. Stattdessen bedient sich diese Dissertation einem Co-Design-Ansatz, der beide Domänen weiterhin eindeutig voneinander abgrenzt, aber das Informationsalter (engl. age of information, AoI) als bedeutenden Schnittstellenparameter ausnutzt. Diese Dissertation trägt dazu bei, die Echtzeitanwendungszuverlässigkeit als Folge der Überschreitung eines vorgegebenen Informationsalterschwellenwerts zu quantifizieren und fokussiert sich dabei auf den Paketverlust als Ursache. Anhand der Beispielanwendung eines fahrerlosen Transportsystems wird gezeigt, dass die zeitlich negative Korrelation von Paketfehlern, die in heutigen Systemen keine Rolle spielt, für Echtzeitanwendungen äußerst vorteilhaft ist. Mit der Annahme von schnellem Schwund als dominanter Fehlerursache auf der Luftschnittstelle werden durch zeitdiskrete Markovmodelle, die für die zwei Netzwerkarchitekturen Single-Hop und Dual-Hop präsentiert werden, Kommunikationsfehlerfolgen auf einen Applikationsfehler abgebildet. Diese Modellierung ermöglicht die analytische Ableitung von anwendungsbezogenen Zuverlässigkeitsmetriken wie die durschnittliche Dauer bis zu einem Fehler (engl. mean time to failure). Für Single-Hop-Netze wird das neuartige Ressourcenallokationsschema State-Aware Resource Allocation (SARA) entwickelt, das auf dem Informationsalter beruht und die Anwendungszuverlässigkeit im Vergleich zu statischer Multi-Konnektivität um Größenordnungen erhöht, während der Ressourcenverbrauch im Bereich von konventioneller Einzelkonnektivität bleibt. Diese Zuverlässigkeit kann auch innerhalb eines Systems von Regelanwendungen, in welchem mehrere Agenten um eine begrenzte Anzahl Ressourcen konkurrieren, statistisch garantiert werden, wenn die Anzahl der verfügbaren Ressourcen pro Agent um ca. 10 % erhöht werden. Für das Dual-Hop Szenario wird darüberhinaus ein Optimierungsverfahren vorgestellt, das eine benutzerdefinierte Kostenfunktion minimiert, die niedrige Anwendungszuverlässigkeit, hohes Informationsalter und hohen durchschnittlichen Ressourcenverbrauch bestraft und so das benutzerdefinierte optimale SARA-Schema ableitet. Diese Optimierung kann offline durchgeführt und als Look-Up-Table in der unteren Medienzugriffsschicht zukünftiger industrieller Drahtlosnetze implementiert werden.:1. Introduction 1 1.1. The Need for an Industrial Solution . . . . . . . . . . . . . . . . . . . 3 1.2. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Related Work 7 2.1. Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2. Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3. Codesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1. The Need for Abstraction – Age of Information . . . . . . . . 11 2.4. Dependability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3. Deriving Proper Communications Requirements 17 3.1. Fundamentals of Control Theory . . . . . . . . . . . . . . . . . . . . 18 3.1.1. Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2. Performance Requirements . . . . . . . . . . . . . . . . . . . 21 3.1.3. Packet Losses and Delay . . . . . . . . . . . . . . . . . . . . . 22 3.2. Joint Design of Control Loop with Packet Losses . . . . . . . . . . . . 23 3.2.1. Method 1: Reduced Sampling . . . . . . . . . . . . . . . . . . 23 3.2.2. Method 2: Markov Jump Linear System . . . . . . . . . . . . . 25 3.2.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3. Focus Application: The AGV Use Case . . . . . . . . . . . . . . . . . . 31 3.3.1. Control Loop Model . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.2. Control Performance Requirements . . . . . . . . . . . . . . . 33 3.3.3. Joint Modeling: Applying Reduced Sampling . . . . . . . . . . 34 3.3.4. Joint Modeling: Applying MJLS . . . . . . . . . . . . . . . . . 34 3.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4. Modeling Control-Communication Failures 43 4.1. Communication Assumptions . . . . . . . . . . . . . . . . . . . . . . 43 4.1.1. Small-Scale Fading as a Cause of Failure . . . . . . . . . . . . 44 4.1.2. Connectivity Models . . . . . . . . . . . . . . . . . . . . . . . 46 4.2. Failure Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.1. Single-hop network . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.2. Dual-hop network . . . . . . . . . . . . . . . . . . . . . . . . 51 4.3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.1. Mean Time to Failure . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.2. Packet Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.3. Average Number of Assigned Channels . . . . . . . . . . . . . 57 4.3.4. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 57 4.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5. Single Hop – Single Agent 61 5.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 61 5.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.3. Erroneous Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 67 5.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6. Single Hop – Multiple Agents 71 6.1. Failure Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.1. Admission Control . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.2. Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . 73 6.1.3. Computational Complexity . . . . . . . . . . . . . . . . . . . 74 6.1.4. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 75 6.2. Illustration Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3.1. Verification through System-Level Simulation . . . . . . . . . 78 6.3.2. Applicability on the System Level . . . . . . . . . . . . . . . . 79 6.3.3. Comparison of Admission Control Schemes . . . . . . . . . . 80 6.3.4. Impact of the Packet Loss Tolerance . . . . . . . . . . . . . . . 82 6.3.5. Impact of the Number of Agents . . . . . . . . . . . . . . . . . 84 6.3.6. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 84 6.3.7. Channel Saturation Ratio . . . . . . . . . . . . . . . . . . . . 86 6.3.8. Enforcing Full Channel Saturation . . . . . . . . . . . . . . . 86 6.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7. Dual Hop – Single Agent 91 7.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 91 7.2. Optimization Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.3.1. Extensive Simulation . . . . . . . . . . . . . . . . . . . . . . . 96 7.3.2. Non-Integer-Constrained Optimization . . . . . . . . . . . . . 98 7.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8. Conclusions and Outlook 105 8.1. Key Results and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 105 8.2. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 A. DC Motor Model 111 Bibliography 113 Publications of the Author 127 List of Figures 129 List of Tables 131 List of Operators and Constants 133 List of Symbols 135 List of Acronyms 137 Curriculum Vitae 139In industrial manufacturing, Industry 4.0 refers to the ongoing convergence of the real and virtual worlds, enabled through intelligently interconnecting industrial machines and processes through information and communications technology. Ultrareliable low-latency communication (URLLC) is widely regarded as the enabling technology for Industry 4.0 due to its ability to fulfill highest quality-of-service (QoS) comparable to those of industrial wireline connections. In contrast to this trend, a range of works in the research domain of networked control systems have shown that URLLC’s supreme QoS is not necessarily required to achieve high quality-ofcontrol; the co-design of control and communication enables to jointly optimize and balance both quality-of-control parameters and network parameters through blurring the boundary between application and network layer. However, through the tight interlacing, this approach requires a fundamental (joint) redesign of both control systems and communication networks and may therefore not lead to short-term widespread adoption. Therefore, this thesis instead embraces a novel co-design approach which keeps both domains distinct but leverages the combination of control and communications by yet exploiting the age of information (AoI) as a valuable interface metric. This thesis contributes to quantifying application dependability as a consequence of exceeding a given peak AoI with the particular focus on packet losses. The beneficial influence of negative temporal packet loss correlation on control performance is demonstrated by means of the automated guided vehicle use case. Assuming small-scale fading as the dominant cause of communication failure, a series of communication failures are mapped to an application failure through discrete-time Markov models for single-hop (e.g, only uplink or downlink) and dual-hop (e.g., subsequent uplink and downlink) architectures. This enables the derivation of application-related dependability metrics such as the mean time to failure in closed form. For single-hop networks, an AoI-aware resource allocation strategy termed state-aware resource allocation (SARA) is proposed that increases the application reliability by orders of magnitude compared to static multi-connectivity while keeping the resource consumption in the range of best-effort single-connectivity. This dependability can also be statistically guaranteed on a system level – where multiple agents compete for a limited number of resources – if the provided amount of resources per agent is increased by approximately 10 %. For the dual-hop scenario, an AoI-aware resource allocation optimization is developed that minimizes a user-defined penalty function that punishes low application reliability, high AoI, and high average resource consumption. This optimization may be carried out offline and each resulting optimal SARA scheme may be implemented as a look-up table in the lower medium access control layer of future wireless industrial networks.:1. Introduction 1 1.1. The Need for an Industrial Solution . . . . . . . . . . . . . . . . . . . 3 1.2. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Related Work 7 2.1. Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2. Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3. Codesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1. The Need for Abstraction – Age of Information . . . . . . . . 11 2.4. Dependability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3. Deriving Proper Communications Requirements 17 3.1. Fundamentals of Control Theory . . . . . . . . . . . . . . . . . . . . 18 3.1.1. Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2. Performance Requirements . . . . . . . . . . . . . . . . . . . 21 3.1.3. Packet Losses and Delay . . . . . . . . . . . . . . . . . . . . . 22 3.2. Joint Design of Control Loop with Packet Losses . . . . . . . . . . . . 23 3.2.1. Method 1: Reduced Sampling . . . . . . . . . . . . . . . . . . 23 3.2.2. Method 2: Markov Jump Linear System . . . . . . . . . . . . . 25 3.2.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3. Focus Application: The AGV Use Case . . . . . . . . . . . . . . . . . . 31 3.3.1. Control Loop Model . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.2. Control Performance Requirements . . . . . . . . . . . . . . . 33 3.3.3. Joint Modeling: Applying Reduced Sampling . . . . . . . . . . 34 3.3.4. Joint Modeling: Applying MJLS . . . . . . . . . . . . . . . . . 34 3.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4. Modeling Control-Communication Failures 43 4.1. Communication Assumptions . . . . . . . . . . . . . . . . . . . . . . 43 4.1.1. Small-Scale Fading as a Cause of Failure . . . . . . . . . . . . 44 4.1.2. Connectivity Models . . . . . . . . . . . . . . . . . . . . . . . 46 4.2. Failure Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.1. Single-hop network . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.2. Dual-hop network . . . . . . . . . . . . . . . . . . . . . . . . 51 4.3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.1. Mean Time to Failure . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.2. Packet Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.3. Average Number of Assigned Channels . . . . . . . . . . . . . 57 4.3.4. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 57 4.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5. Single Hop – Single Agent 61 5.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 61 5.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.3. Erroneous Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 67 5.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6. Single Hop – Multiple Agents 71 6.1. Failure Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.1. Admission Control . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.2. Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . 73 6.1.3. Computational Complexity . . . . . . . . . . . . . . . . . . . 74 6.1.4. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 75 6.2. Illustration Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3.1. Verification through System-Level Simulation . . . . . . . . . 78 6.3.2. Applicability on the System Level . . . . . . . . . . . . . . . . 79 6.3.3. Comparison of Admission Control Schemes . . . . . . . . . . 80 6.3.4. Impact of the Packet Loss Tolerance . . . . . . . . . . . . . . . 82 6.3.5. Impact of the Number of Agents . . . . . . . . . . . . . . . . . 84 6.3.6. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 84 6.3.7. Channel Saturation Ratio . . . . . . . . . . . . . . . . . . . . 86 6.3.8. Enforcing Full Channel Saturation . . . . . . . . . . . . . . . 86 6.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7. Dual Hop – Single Agent 91 7.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 91 7.2. Optimization Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.3.1. Extensive Simulation . . . . . . . . . . . . . . . . . . . . . . . 96 7.3.2. Non-Integer-Constrained Optimization . . . . . . . . . . . . . 98 7.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8. Conclusions and Outlook 105 8.1. Key Results and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 105 8.2. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 A. DC Motor Model 111 Bibliography 113 Publications of the Author 127 List of Figures 129 List of Tables 131 List of Operators and Constants 133 List of Symbols 135 List of Acronyms 137 Curriculum Vitae 13

Synchronous Transmissions in Low-Power Wireless: A Survey of Communication Protocols and Network Services

Low-power wireless communication is a central building block of Cyber-physical Systems and the Internet of Things. Conventional low-power wireless protocols make avoiding packet collisions a cornerstone design choice. The concept of synchronous transmissions challenges this view. As collisions are not necessarily destructive, under specific circumstances, commodity low-power wireless radios are often able to receive useful information even in the presence of superimposed signals from different transmitters. We survey the growing number of protocols that exploit synchronous transmissions for higher robustness and efficiency as well as unprecedented functionality and versatility compared to conventional designs. The illustration of protocols based on synchronous transmissions is cast in a conceptional framework we establish, with the goal of highlighting differences and similarities among the proposed solutions. We conclude the paper with a discussion on open research questions in this field.Comment: Submitted to ACM Computing Survey

Fehlertolerante Mehrkernprozessoren für gemischt-kritische Echtzeitsysteme

Current and future computing systems must be appropriately designed to cope with random hardware faults in order to provide a dependable service and correct functionality. Dependability has many facets to be addressed when designing a system and that is specially challenging in mixed-critical real-time systems, where safety standards play an important role and where responding in time can be as important as responding correctly or even responding at all. The thesis addresses the dependability of mixed-critical real-time systems, considering three important requirements: integrity, resilience and real-time. More specifically, it looks into the architectural and performance aspects of achieving dependability, concentrating its scope on error detection and handling in hardware -- more specifically in the Network-on-Chip (NoC), the backbone of modern MPSoC -- and on the performance of error handling and recovery in software. The thesis starts by looking at the impacts of random hardware faults on the NoC and on the system, with special focus on soft errors. Then, it addresses the uncovered weaknesses in the NoC by proposing a resilient NoC for mixed-critical real-time systems that is able to provide a highly reliable service with transparent protection for the applications. Formal communication time analysis is provided with common ARQ protocols modeled for NoCs and including a novel ARQ-based protocol optimized for DMAs. After addressing the efficient use of ARQ-based protocols in NoCs, the thesis proposes the Advanced Integrity Q-service (AIQ), a low-overhead mechanism to achieve integrity and real-time guarantees of NoC transactions on an End-to-End (E2E) basis. Inspired by transactions in distributed systems, the mechanism differs from the previous approach in that it does not provide error recovery in hardware but delegates the task to software, making use of existing functionality in cross-layer fault-tolerance solutions. Finally, the thesis addresses error handling in software as seen in cross-layer approaches. It addresses the performance of replicated software execution in many-core platforms. Replicated software execution provides protection to the system against random hardware faults. It relies on hardware-supported error detection and error handling in software. The replica-aware co-scheduling is proposed to achieve high performance with replicated execution, which is not possible with standard real-time schedulers.Um einen zuverlässigen Betrieb und korrekte Funktionalität zu gewährleisten, müssen aktuelle und zukünftige Computersysteme so ausgelegt werden, dass sie mit diesen Fehlern umgehen können. Zuverlässigkeit hat viele Aspekte, die bei der Entwicklung eines Systems berücksichtigt werden müssen. Das gilt insbesondere für Echtzeitsysteme mit gemischter Kritikalität, bei denen Sicherheitsstandards, die ein korrektes und rechtzeitiges Verhalten fordern, eine wichtige Rolle spielen. Diese Dissertation befasst sich mit der Zuverlässigkeit von gemischt-kritischen Echtzeitsystemen unter Berücksichtigung von drei wichtigen Anforderungen: Integrität, Resilienz und Echtzeit. Genauer gesagt, behandelt sie Architektur- und Leistungsaspekte die notwendig sind um Zuverlässigkeit zu erreichen, wobei der Schwerpunkt auf der Fehlererkennung und -behandlung in der Hardware – genauer gesagt im Network-on-Chip (NoC), dem Rückgrat des modernen MPSoC – und auf der Leistung der Fehlerbehandlung und -behebung in der Software liegt. Die Arbeit beginnt mit der Untersuchung der Auswirkung von zufälligen Hardwarefehlern auf das NoC und das System, wobei der Schwerpunkt auf weichen Fehler (soft errors) liegt. Anschließend werden die aufgedeckten Schwachstellen im NoC behoben, indem ein widerstandsfähiges NoC für gemischt-kritische Echtzeitsysteme vorgeschlagen wird, das in der Lage ist, einen höchst zuverlässigen Betrieb mit transparentem Schutz für die Anwendungen zu bieten. Nach der Auseinandersetzung mit der effizienten Nutzung von ARQ-basierten Protokolle in NoCs, wird der Advanced Integrity Q-Service (AIQ) vorgestellt, der ein Mechanismus mit geringem Overhead ist, um Integrität und Echtzeit-Garantien von NoC-Transaktionen auf Ende-zu-Ende (E2E)-Basis zu erreichen. Inspiriert von Transaktionen in verteilten Systemen unterscheidet sich der Mechanismus vom bisherigen Konzept dadurch, dass er keine Fehlerbehebung in der Hardware vorsieht, sondern diese Aufgabe an die Software delegiert. Schließlich befasst sich die Dissertation mit der Fehlerbehandlung in Software, wie sie in schichtübergreifenden Methoden zu sehen ist. Sie behandelt die Leistung der replizierten Software-Ausführung in Many-Core-Plattformen. Es setzt auf hardwaregestützte Fehlererkennung und Fehlerbehandlung in der Software. Das Replika-bewusste Co-Scheduling wird vorgeschlagen, um eine hohe Performance bei replizierter Ausführung zu erreichen, was mit Standard-Echtzeit-Schedulern nicht möglich ist

Synchronous and Concurrent Transmissions for Consensus in Low-Power Wireless

With the emergence of the Internet of Things, autonomous vehicles and the Industry 4.0, the need for dependable yet adaptive network protocols is arising. Many of these applications build their operations on distributed consensus. For example, UAVs agree on maneuvers to execute, and industrial systems agree on set-points for actuators.Moreover, such scenarios imply a dynamic network topology due to mobility and interference, for example. Many applications are mission- and safety-critical, too.Failures could cost lives or precipitate economic losses.In this thesis, we design, implement and evaluate network protocols as a step towards enabling a low-power, adaptive and dependable ubiquitous networking that enables consensus in the Internet of Things. We make four main contributions:- We introduce Orchestra that addresses the challenge of bringing TSCH (Time Slotted Channel Hopping) to dynamic networks as envisioned in the Internet of Things. In Orchestra, nodes autonomously compute their local schedules and update automatically as the topology evolves without signaling overhead. Besides, it does not require a central or distributed scheduler. Instead, it relies on the existing network stack information to maintain the schedules.- We present A2 : Agreement in the Air, a system that brings distributed consensus to low-power multihop networks. A2 introduces Synchrotron, a synchronous transmissions kernel that builds a robust mesh by exploiting the capture effect, frequency hopping with parallel channels, and link-layer security. A2 builds on top of this layer and enables the two- and three-phase commit protocols, and services such as group membership, hopping sequence distribution, and re-keying.- We present Wireless Paxos, a fault-tolerant, network-wide consensus primitive for low-power wireless networks. It is a new variant of Paxos, a widely used consensus protocol, and is specifically designed to tackle the challenges of low-power wireless networks. By utilizing concurrent transmissions, it provides a dependable low-latency consensus.- We present BlueFlood, a protocol that adapts concurrent transmissions to Bluetooth. The result is fast and efficient data dissemination in multihop Bluetooth networks. Moreover, BlueFlood floods can be reliably received by off-the-shelf Bluetooth devices such as smartphones, opening new applications of concurrent transmissions and seamless integration with existing technologies

Architecture, Services and Protocols for CRUTIAL

This document describes the complete specification of the architecture, services and protocols of the project CRUTIAL. The CRUTIAL Architecture intends to reply to a grand challenge of computer science and control engineering: how to achieve resilience of critical information infrastructures (CII), in particular in the electrical sector. In general lines, the document starts by presenting the main architectural options and components of the architecture, with a special emphasis on a protection device called the CRUTIAL Information Switch (CIS). Given the various criticality levels of the equipments that have to be protected, and the cost of using a replicated device, we define a hierarchy of CIS designs incrementally more resilient. The different CIS designs offer various trade offs in terms of capabilities to prevent and tolerate intrusions, both in the device itself and in the information infrastructure. The Middleware Services, APIs and Protocols chapter describes our approach to intrusion tolerant middleware. The CRUTIAL middleware comprises several building blocks that are organized on a set of layers. The Multipoint Network layer is the lowest layer of the middleware, and features an abstraction of basic communication services, such as provided by standard protocols, like IP, IPsec, UDP, TCP and SSL/TLS. The Communication Support layer features three important building blocks: the Randomized Intrusion-Tolerant Services (RITAS), the CIS Communication service and the Fosel service for mitigating DoS attacks. The Activity Support layer comprises the CIS Protection service, and the Access Control and Authorization service. The Access Control and Authorization service is implemented through PolyOrBAC, which defines the rules for information exchange and collaboration between sub-modules of the architecture, corresponding in fact to different facilities of the CII’s organizations. The Monitoring and Failure Detection layer contains a definition of the services devoted to monitoring and failure detection activities. The Runtime Support Services, APIs, and Protocols chapter features as a main component the Proactive-Reactive Recovery service, whose aim is to guarantee perpetual correct execution of any components it protects.Project co-funded by the European Commission within the Sixth Frame-work Programme (2002-2006

A survey on vehicular communication for cooperative truck platooning application

Platooning is an application where a group of vehicles move one after each other in close proximity, acting jointly as a single physical system. The scope of platooning is to improve safety, reduce fuel consumption, and increase road use efficiency. Even if conceived several decades ago as a concept, based on the new progress in automation and vehicular networking platooning has attracted particular attention in the latest years and is expected to become of common implementation in the next future, at least for trucks.The platoon system is the result of a combination of multiple disciplines, from transportation, to automation, to electronics, to telecommunications. In this survey, we consider the platooning, and more specifically the platooning of trucks, from the point of view of wireless communications. Wireless communications are indeed a key element, since they allow the information to propagate within the convoy with an almost negligible delay and really making all vehicles acting as one. Scope of this paper is to present a comprehensive survey on connected vehicles for the platooning application, starting with an overview of the projects that are driving the development of this technology, followed by a brief overview of the current and upcoming vehicular networking architecture and standards, by a review of the main open issues related to wireless communications applied to platooning, and a discussion of security threats and privacy concerns. The survey will conclude with a discussion of the main areas that we consider still open and that can drive future research directions.(c) 2022 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)

Hardware-Aware Algorithm Designs for Efficient Parallel and Distributed Processing

The introduction and widespread adoption of the Internet of Things, together with emerging new industrial applications, bring new requirements in data processing. Specifically, the need for timely processing of data that arrives at high rates creates a challenge for the traditional cloud computing paradigm, where data collected at various sources is sent to the cloud for processing. As an approach to this challenge, processing algorithms and infrastructure are distributed from the cloud to multiple tiers of computing, closer to the sources of data. This creates a wide range of devices for algorithms to be deployed on and software designs to adapt to.In this thesis, we investigate how hardware-aware algorithm designs on a variety of platforms lead to algorithm implementations that efficiently utilize the underlying resources. We design, implement and evaluate new techniques for representative applications that involve the whole spectrum of devices, from resource-constrained sensors in the field, to highly parallel servers. At each tier of processing capability, we identify key architectural features that are relevant for applications and propose designs that make use of these features to achieve high-rate, timely and energy-efficient processing.In the first part of the thesis, we focus on high-end servers and utilize two main approaches to achieve high throughput processing: vectorization and thread parallelism. We employ vectorization for the case of pattern matching algorithms used in security applications. We show that re-thinking the design of algorithms to better utilize the resources available in the platforms they are deployed on, such as vector processing units, can bring significant speedups in processing throughout. We then show how thread-aware data distribution and proper inter-thread synchronization allow scalability, especially for the problem of high-rate network traffic monitoring. We design a parallelization scheme for sketch-based algorithms that summarize traffic information, which allows them to handle incoming data at high rates and be able to answer queries on that data efficiently, without overheads.In the second part of the thesis, we target the intermediate tier of computing devices and focus on the typical examples of hardware that is found there. We show how single-board computers with embedded accelerators can be used to handle the computationally heavy part of applications and showcase it specifically for pattern matching for security-related processing. We further identify key hardware features that affect the performance of pattern matching algorithms on such devices, present a co-evaluation framework to compare algorithms, and design a new algorithm that efficiently utilizes the hardware features.In the last part of the thesis, we shift the focus to the low-power, resource-constrained tier of processing devices. We target wireless sensor networks and study distributed data processing algorithms where the processing happens on the same devices that generate the data. Specifically, we focus on a continuous monitoring algorithm (geometric monitoring) that aims to minimize communication between nodes. By deploying that algorithm in action, under realistic environments, we demonstrate that the interplay between the network protocol and the application plays an important role in this layer of devices. Based on that observation, we co-design a continuous monitoring application with a modern network stack and augment it further with an in-network aggregation technique. In this way, we show that awareness of the underlying network stack is important to realize the full potential of the continuous monitoring algorithm.The techniques and solutions presented in this thesis contribute to better utilization of hardware characteristics, across a wide spectrum of platforms. We employ these techniques on problems that are representative examples of current and upcoming applications and contribute with an outlook of emerging possibilities that can build on the results of the thesis