Scheduling event-triggered and time-triggered applications with optimal reliability and predictability on networked multi-core chips

Multi-core processors are gaining increasing importance in safety-relevant embedded realtime systems, where temporal guarantees must be ensured despite the sharing of on-chip resources such as processor cores and networks-on-chip (NoC). At the same time, many applications comprise workloads with different timing models including time triggered and event-triggered communication. The first contribution is a scheduling model based on Mixed Integer Linear Programming(MILP) supporting the allocation of computational jobs to processing cores as well as the scheduling of messages and the selection of paths on NoC. The model supports dependencies between computational jobs and it combines both time-triggered and event-triggered messages. Phase alignment of time-triggered messages is performed while avoiding collisions between time-triggered messages and satisfying bandwidth constraints for event-triggered messages. Example scenarios are solved optimally using the IBM CPLEX optimizer yielding minimal computational and communication latencies. Real-time communication and reliability are two important requirements in the development of safety-critical embedded systems, which benefit from the inherent fault isolation and temporal predictability of time-triggered networks. These systems depend on redundant communication schedules that contain global time-based information of message transmissions with conflict-free paths through the switches. In these systems, the use of redundancy to handle communication errors requires the preallocation of communication resources. The second contribution introduces a novel scheduler for redundant time-triggered networks that assigns messages to redundant paths. The scheduler considers the link reliability along with physical and logical models and produces a schedule where each message is assigned to two different paths along the switches. We also discuss and validate the approach with results from a prototype implementation. SoS consist of complex interconnections of large numbers of networked embedded systems that are characterized by operational and managerial independence of constituent systems, geographical separation, and emergent behavior in a constantly changing environment. The support for real-time communication is crucial for many SoS application areas such as medical, business, and military systems. The third contribution is a conceptual model and a scheduling algorithm for supporting real-time requirements in SoS. The search for a feasible schedule is computed incrementally upon the introduction of new applications in the SoS. The distributed computation of the schedule using the different constituent systems considers the lack of global knowledge and control in the SoS, while also reducing the overall scheduling time. Concurrent scheduling activities are supported to deal with the uncoordinated and possibly simultaneous introduction of multiple applications. The dissertation introduces also a simulation framework with real-time support of SoS that supports high-level scheduling as well as low-level scheduling for each constituent system. A time-triggered Ethernet (TTEthernet) simulation framework was extended by adding a scheduler layer to perform incremental scheduling among Constituent System Managers (CSMs). The simulation framework enabled the evaluation of the proposed algorithms in terms of schedulability, run-time, and worst-case latency for time-triggered and rate-constrained messages.Mehrkernprozessoren gewinnen zunehmend an Bedeutung in sicherheitsrelevanten eingebetteten Echtzeitsystemen, bei denen trotz der gemeinsamen Nutzung von On-Chip-Ressourcen wie Prozessorkernen und On-Chip-Netzwerken zeitliche Garantien gewährleistet sein müssen. Gleichzeitig umfassen viele Anwendungen Arbeitsbelastungen mit unterschiedlichen Timing- Modellen, einschließlich zeitgesteuerter und ereignisgesteuerter Kommunikation. Der erste Beitrag der Dissertation ist ein Planungsmodell, das auf der gemischt-ganzzahligen linearen Programmierung basiert und die Zuweisung von Rechenaufträgen an Prozessorkerne sowie die Planung von Nachrichten und die Auswahl von Wegen auf NoCs unterstützt. Das Modell unterstützt Abhängigkeiten zwischen Rechenjobs und kombiniert sowohl zeitgesteuerte als auch ereignisgesteuerte Nachrichten. Die Phasenausrichtung zeitgesteuerter Nachrichten wird durchgeführt, während Kollisionen zwischen zeitgesteuerten Nachrichten und die Verletzung von Bandbreitenbeschränkungen für ereignisgesteuerte Nachrichten vermieden werden. Beispielszenarien werden optimal mit dem IBM CPLEX-Optimierer gelöst, wobei minimale Rechen- und Kommunikationslatenzen garantiert werden. Echtzeitkommunikation und Zuverlässigkeit sind zwei wichtige Anforderungen bei der Entwicklung sicherheitskritischer eingebetteter Systeme, die von der inhärenten Fehlerisolierung und zeitlichen Vorhersagbarkeit zeitgesteuerter Netzwerke profitieren. Als Grundlage für Fehlertoleranz benötigen diese Systeme außerdem redundante Kommunikationspläne, die globale zeitbasierte Informationen von mehrfachen Nachrichtenübertragungen mit konfliktfreien Pfaden durch die Switches enthalten. In diesen Systemen erfordert die Verwendung von Redundanz zur Behandlung von Kommunikationsfehlern die Vorbelegung von Kommunikationsressourcen. Der zweite Beitrag der Dissertation stellt einen neuartigen Scheduler für redundante zeitgesteuerte Netzwerke vor, der Nachrichten redundanten Pfaden zuweist. Der Scheduler berücksichtigt die Verbindungszuverlässigkeit zusammen mit physischen und logischen Modellen und erstellt einen Zeitplan, bei dem jede Nachricht zwei verschiedenen Pfaden entlang der Switches zugewiesen wird. Wir diskutieren und validieren den Ansatz mit den Ergebnissen einer Prototypimplementierung. Systeme von Systemen (SoS) bestehen aus komplexen Zusammenschaltungen einer großen Anzahl von vernetzten eingebetteten Systemen, die durch betriebliche Unabhängigkeit von Teilsystemen, geografische Trennung und emergentes Verhalten in einer sich ständig verändernden Umgebung gekennzeichnet sind. Die Unterstützung für Echtzeitkommunikation ist für viele Anwendungsbereiche wie medizinische, geschäftliche und militärische Systeme von entscheidender Bedeutung. Der dritte Beitrag der Dissertation ist ein konzeptionelles Modell und ein Planungsalgorithmus zur Unterstützung von Echtzeitanforderungen in SoS. Die Suche nach einem realisierbaren Zeitplan wird schrittweise nach der Einführung neuer Anwendungen im SoS berechnet. Die verteilte Berechnung des Zeitplans unter Verwendung der verschiedenen Teilsysteme berücksichtigt den Mangel an globalem Wissen und Kontrolle im SoS, während gleichzeitig die Gesamtplanungszeit verringert wird. Gleichzeitige Terminierungsaktivitäten werden unterstützt, um die unkoordinierte und möglicherweise gleichzeitige Einführung mehrerer Anwendungen zu bewältigen. Die Dissertation stellt auch ein Simulationsframework mit Echtzeit-Unterstützung von SoS vor, das sowohl die High-Level-Planung als auch die Low-Level-Planung für jedes Teilsystem unterstützt. Ein Simulationsframework für zeitgesteuertes Ethernet (TTEthernet) wurde um eine Scheduler-Schicht erweitert, um eine inkrementelle Planung unter Constituent System Managern (CSMs) durchzuführen. Das Simulationsframework ermöglichte die Evaluierung der vorgeschlagenen Algorithmen hinsichtlich der Planbarkeit, der Laufzeit und der Worst-Case-Latenz für zeitgesteuerte und ratenbeschränkte Nachrichten

lLTZVisor: a lightweight TrustZone-assisted hypervisor for low-end ARM devices

Dissertação de mestrado em Engenharia Eletrónica Industrial e ComputadoresVirtualization is a well-established technology in the server and desktop space and has recently been spreading across different embedded industries. Facing multiple challenges derived by the advent of the Internet of Things (IoT) era, these industries are driven by an upgrowing interest in consolidating and isolating multiple environments with mixed-criticality features, to address the complex IoT application landscape. Even though this is true for majority mid- to high-end embedded applications, low-end systems still present little to no solutions proposed so far. TrustZone technology, designed by ARM to improve security on its processors, was adopted really well in the embedded market. As such, the research community became active in exploring other TrustZone’s capacities for isolation, like an alternative form of system virtualization. The lightweight TrustZone-assisted hypervisor (LTZVisor), that mainly targets the consolidation of mixed-criticality systems on the same hardware platform, is one design example that takes advantage of TrustZone technology for ARM application processors. With the recent introduction of this technology to the new generation of ARM microcontrollers, an opportunity to expand this breakthrough form of virtualization to low-end devices arose. This work proposes the development of the lLTZVisor hypervisor, a refactored LTZVisor version that aims to provide strong isolation on resource-constrained devices, while achieving a low-memory footprint, determinism and high efficiency. The key for this is to implement a minimal, reliable, secure and predictable virtualization layer, supported by the TrustZone technology present on the newest generation of ARM microcontrollers (Cortex-M23/33).Virtualização é uma tecnologia já bem estabelecida no âmbito de servidores e computadores pessoais que recentemente tem vindo a espalhar-se através de várias indústrias de sistemas embebidos. Face aos desafios provenientes do surgimento da era Internet of Things (IoT), estas indústrias são guiadas pelo crescimento do interesse em consolidar e isolar múltiplos sistemas com diferentes níveis de criticidade, para atender ao atual e complexo cenário aplicativo IoT. Apesar de isto se aplicar à maioria de aplicações embebidas de média e alta gama, sistemas de baixa gama apresentam-se ainda com poucas soluções propostas. A tecnologia TrustZone, desenvolvida pela ARM de forma a melhorar a segurança nos seus processadores, foi adoptada muito bem pelo mercado dos sistemas embebidos. Como tal, a comunidade científica começou a explorar outras aplicações da tecnologia TrustZone para isolamento, como uma forma alternativa de virtualização de sistemas. O "lightweight TrustZone-assisted hypervisor (LTZVisor)", que tem sobretudo como fim a consolidação de sistemas de criticidade mista na mesma plataforma de hardware, é um exemplo que tira vantagem da tecnologia TrustZone para os processadores ARM de alta gama. Com a recente introdução desta tecnologia para a nova geração de microcontroladores ARM, surgiu uma oportunidade para expandir esta forma inovadora de virtualização para dispositivos de baixa gama. Este trabalho propõe o desenvolvimento do hipervisor lLTZVisor, uma versão reestruturada do LTZVisor que visa em proporcionar um forte isolamento em dispositivos com recursos restritos, simultâneamente atingindo um baixo footprint de memória, determinismo e alta eficiência. A chave para isto está na implementação de uma camada de virtualização mínima, fiável, segura e previsível, potencializada pela tecnologia TrustZone presente na mais recente geração de microcontroladores ARM (Cortex-M23/33)

Distributed scheduling algorithms for LoRa-based wide area cyber-physical systems

Low Power Wide Area Networks (LPWAN) are a class of wireless communication protocols that work over long distances, consume low power and support low datarates. LPWANs have been designed for monitoring applications, with sparse communication from nodes to servers and sparser from servers to nodes. Inspite of their initial design, LPWANs have the potential to target applications with higher and stricter requirements like those of Cyber-Physical Systems (CPS). Due to their long-range capabilities, LPWANs can specifically target CPS applications distributed over a wide-area, which is referred to as Wide-Area CPS (WA-CPS). Augmenting WA-CPSs with wireless communication would allow for more flexible, low-cost and easily maintainable deployment. However, wireless communications come with problems like reduced reliability and unpredictable latencies, making them harder to use for CPSs. With this intention, this thesis explores the use of LPWANs, specifically LoRa, to meet the communication and control requirements of WA-CPSs. The thesis focuses on using LoRa due to its high resilience to noise, several communication parameters to choose from and a freely modifiable communication stack and servers making it ideal for research and deployment. However, LoRaWAN suffers from low reliability due to its ALOHA channel access method. The thesis posits that "Distributed algorithms would increase the protocol's reliability allowing it to meet the requirements of WA-CPSs". Three different application scenarios are explored in this thesis that leverage unexplored aspects of LoRa to meet their requirements. The application scenarios are delay-tolerant vehicular networks, multi-stakeholder WA-CPS deployments and water distribution networks. The systems use novel algorithms to facilitate communication between the nodes and gateways to ensure a highly reliable system. The results outperform state-of-art techniques to prove that LoRa is currently under-utilised and can be used for CPS applications.Open Acces

Synchronous Transmissions in Low-Power Wireless: A Survey of Communication Protocols and Network Services

Low-power wireless communication is a central building block of Cyber-physical Systems and the Internet of Things. Conventional low-power wireless protocols make avoiding packet collisions a cornerstone design choice. The concept of synchronous transmissions challenges this view. As collisions are not necessarily destructive, under specific circumstances, commodity low-power wireless radios are often able to receive useful information even in the presence of superimposed signals from different transmitters. We survey the growing number of protocols that exploit synchronous transmissions for higher robustness and efficiency as well as unprecedented functionality and versatility compared to conventional designs. The illustration of protocols based on synchronous transmissions is cast in a conceptional framework we establish, with the goal of highlighting differences and similarities among the proposed solutions. We conclude the paper with a discussion on open research questions in this field.Comment: Submitted to ACM Computing Survey

Distributed Real-time Systems - Deterministic Protocols for Wireless Networks and Model-Driven Development with SDL

In a networked system, the communication system is indispensable but often the weakest link w.r.t. performance and reliability. This, particularly, holds for wireless communication systems, where the error- and interference-prone medium and the character of network topologies implicate special challenges. However, there are many scenarios of wireless networks, in which a certain quality-of-service has to be provided despite these conditions. In this regard, distributed real-time systems, whose realization by wireless multi-hop networks becomes increasingly popular, are a particular challenge. For such systems, it is of crucial importance that communication protocols are deterministic and come with the required amount of efficiency and predictability, while additionally considering scarce hardware resources that are a major limiting factor of wireless sensor nodes. This, in turn, does not only place demands on the behavior of a protocol but also on its implementation, which has to comply with timing and resource constraints. The first part of this thesis presents a deterministic protocol for wireless multi-hop networks with time-critical behavior. The protocol is referred to as Arbitrating and Cooperative Transfer Protocol (ACTP), and is an instance of a binary countdown protocol. It enables the reliable transfer of bit sequences of adjustable length and deterministically resolves contest among nodes based on a flexible priority assignment, with constant delays, and within configurable arbitration radii. The protocol's key requirement is the collision-resistant encoding of bits, which is achieved by the incorporation of black bursts. Besides revisiting black bursts and proposing measures to optimize their detection, robustness, and implementation on wireless sensor nodes, the first part of this thesis presents the mode of operation and time behavior of ACTP. In addition, possible applications of ACTP are illustrated, presenting solutions to well-known problems of distributed systems like leader election and data dissemination. Furthermore, results of experimental evaluations with customary wireless transceivers are outlined to provide evidence of the protocol's implementability and benefits. In the second part of this thesis, the focus is shifted from concrete deterministic protocols to their model-driven development with the Specification and Description Language (SDL). Though SDL is well-established in the domain of telecommunication and distributed systems, the predictability of its implementations is often insufficient as previous projects have shown. To increase this predictability and to improve SDL's applicability to time-critical systems, real-time tasks, an approved concept in the design of real-time systems, are transferred to SDL and extended to cover node-spanning system tasks. In this regard, a priority-based execution and suspension model is introduced in SDL, which enables task-specific priority assignments in the SDL specification that are orthogonal to the static structure of SDL systems and control transition execution orders on design as well as on implementation level. Both the formal incorporation of real-time tasks into SDL and their implementation in a novel scheduling strategy are discussed in this context. By means of evaluations on wireless sensor nodes, evidence is provided that these extensions reduce worst-case execution times substantially, and improve the predictability of SDL implementations and the language's applicability to real-time systems

Design and implementation of a modular controller for robotic machines

This research focused on the design and implementation of an Intelligent Modular Controller (IMC) architecture designed to be reconfigurable over a robust network. The design incorporates novel communication, hardware, and software architectures. This was motivated by current industrial needs for distributed control systems due to growing demand for less complexity, more processing power, flexibility, and greater fault tolerance. To this end, three main contributions were made. Most distributed control architectures depend on multi-tier heterogeneous communication networks requiring linking devices and/or complex middleware. In this study, first, a communication architecture was proposed and implemented with a homogenous network employing the ubiquitous Ethernet for both real-time and non real-time communication. This was achieved by a producer-consumer coordination model for real-time data communication over a segmented network, and a client-server model for point-to-point transactions. The protocols deployed use a Time-Triggered (TT) approach to schedule real-time tasks on the network. Unlike other TT approaches, the scheduling mechanism does not need to be configured explicitly when controller nodes are added or removed. An implicit clock synchronization technique was also developed to complement the architecture. Second, a reconfigurable mechanism based on an auto-configuration protocol was developed. Modules on the network use this protocol to automatically detect themselves, establish communication, and negotiate for a desired configuration. Third, the research demonstrated hardware/software co-design as a contribution to the growing discipline of mechatronics. The IMC consists of a motion controller board designed and prototyped in-house, and a Java microcontroller. An IMC is mapped to each machine/robot axis, and an additional IMC can be configured to serve as a real-time coordinator. The entire architecture was implemented in Java, thus reinforcing uniformity, simplicity, modularity, and openness. Evaluation results showed the potential of the flexible controller to meet medium to high performance machining requirements

Synchronous and Concurrent Transmissions for Consensus in Low-Power Wireless

With the emergence of the Internet of Things, autonomous vehicles and the Industry 4.0, the need for dependable yet adaptive network protocols is arising. Many of these applications build their operations on distributed consensus. For example, UAVs agree on maneuvers to execute, and industrial systems agree on set-points for actuators.Moreover, such scenarios imply a dynamic network topology due to mobility and interference, for example. Many applications are mission- and safety-critical, too.Failures could cost lives or precipitate economic losses.In this thesis, we design, implement and evaluate network protocols as a step towards enabling a low-power, adaptive and dependable ubiquitous networking that enables consensus in the Internet of Things. We make four main contributions:- We introduce Orchestra that addresses the challenge of bringing TSCH (Time Slotted Channel Hopping) to dynamic networks as envisioned in the Internet of Things. In Orchestra, nodes autonomously compute their local schedules and update automatically as the topology evolves without signaling overhead. Besides, it does not require a central or distributed scheduler. Instead, it relies on the existing network stack information to maintain the schedules.- We present A2 : Agreement in the Air, a system that brings distributed consensus to low-power multihop networks. A2 introduces Synchrotron, a synchronous transmissions kernel that builds a robust mesh by exploiting the capture effect, frequency hopping with parallel channels, and link-layer security. A2 builds on top of this layer and enables the two- and three-phase commit protocols, and services such as group membership, hopping sequence distribution, and re-keying.- We present Wireless Paxos, a fault-tolerant, network-wide consensus primitive for low-power wireless networks. It is a new variant of Paxos, a widely used consensus protocol, and is specifically designed to tackle the challenges of low-power wireless networks. By utilizing concurrent transmissions, it provides a dependable low-latency consensus.- We present BlueFlood, a protocol that adapts concurrent transmissions to Bluetooth. The result is fast and efficient data dissemination in multihop Bluetooth networks. Moreover, BlueFlood floods can be reliably received by off-the-shelf Bluetooth devices such as smartphones, opening new applications of concurrent transmissions and seamless integration with existing technologies

Lightweight Event-driven Real-time Operating System for Resource Constrained Connectivity

Wirepas Connectivity (WPC) is a complex protocol stack for large scale mesh-based Internet of Things (IoT) networks. The communication units in a WPC network are called nodes and these are designed to be cheap, resource constrained and battery operated. In contrast, each node requires several levels of parallel and real-time processing, which is best provided by a Real-Time Operating System (RTOS). The resource constraint aspect places requirements for the RTOS design. The RTOS kernel should take less than 10 kB of program memory and under 1 kB of data memory. It must be energy efficient for battery operation and for this reason its scheduling must be tickless (as opposed to time-sharing). Furthermore, the WPC protocol stack requires deterministic real-time timings with microsecond accuracy from the RTOS. This thesis studies the feasibility of related RTOSs Contiki, TinyOS, µC/OS and FreeRTOS for WPC use. The study shows that none of the related RTOSs are feasible without major modification. Contiki and TinyOS would complicate software development. µC/OS is commercially licensed and would increase per node cost. FreeRTOS lacks sufficient real-time operation for WPC. Furthermore, these RTOSs are designed to be general purpose and thus they are wasteful with precious memory and energy resources. To better deal with these challenges, a more specific approach is required. As a solution, this thesis presents a completely new RTOS called WPC-OS, designed specifically for WPC. The RTOS design targets to timing determinism and energy efficiency in all its functions. The WPC-OS scheduler provides a novel and lightweight timetabled scheduling approach, which uses task durations to determine the next task. Event-driven operation is provided on top of this to achieve reactiveness to concurrent events. For evaluation and measuring WPC-OS design efficiency, it was implemented on an nRF52832 platform. The measurement results show that the WPC-OS kernel achieved a small memory footprint. With the typical WPC node configuration, it uses only 5 kB of program memory and 350 B of data memory. It can handle the WPC timing requirements with its real-time event service, which guarantees 1 us timing accuracy. It provides lightweight multitasking capability for applications, while being energy efficient. WPC-OS solves all design requirements WPC imposes on RTOS design, and is suitable for mass production. As future work, coroutine and hybrid scheduling options for WPC-OS should be investigated