ReTiF: A declarative real-time scheduling framework for POSIX systems

This paper proposes a novel framework providing a declarative interface to access real-time process scheduling services available in an operating system kernel. The main idea is to let applications declare their temporal requirements or characteristics without knowing exactly which underlying scheduling algorithms are offered by the system. The proposed framework can adequately handle such a set of heterogeneous requirements configuring the platform and partitioning the requests among the available multitude of cores, so to exploit the various scheduling disciplines that are available in the kernel, matching application requirements in the best possible way. The framework is realized with a modular architecture in which different plugins handle independently certain real-time scheduling features. The architecture is designed to make its behavior customization easier and enhance the support for other operating systems by introducing and configuring additional plugins

Integration of generic operating systems in partitioned architectures

Tese de mestrado, Engenharia Informática (Arquitectura, Sistemas e Redes de Computadores), Universidade de Lisboa, Faculdade de Ciências, 2009The Integrated Modular Avionics (IMA) specification defines a partitioned environment hosting multiple avionics functions of different criticalities on a shared computing platform. ARINC 653, one of the specifications related to the IMA concept, defines a standard interface between the software applications and the underlying operating system. Both these specifications come from the world of civil aviation, but they are getting interest from space industry partners, who have identified common requirements to those of aeronautic applications. Within the scope of this interest, the AIR architecture was defined, under a contract from the European Space Agency (ESA). AIR provides temporal and spatial segregation, and foresees the use of different operating systems in each partition. Temporal segregation is achieved through the fixed cyclic scheduling of computing resources to partitions. The present work extends the foreseen partition operating system (POS) heterogeneity to generic non-real-time operating systems. This was motivated by documented difficulties in porting applications to RTOSs, and by the notion that proper integration of a non-real-time POS will not compromise the timeliness of critical real-time functions. For this purpose, Linux is used as a case study. An embedded variant of Linux is built and evaluated regarding its adequacy as a POS in the AIR architecture. To guarantee safe integration, a solution based on the Linux paravirtualization interface, paravirt-ops, is proposed. In the course of these activities, the AIR architecture definition was also subject to improvements. The most significant one, motivated by the intended increased POS heterogeneity, was the introduction of a new component, the AIR Partition OS Adaptation Layer (PAL). The AIR PAL provides greater POS-independence to the major components of the AIR architecture, easing their independent certification efforts. Other improvements provide enhanced timeliness mechanisms, such as mode-based schedules and process deadline violation monitoring.A especificação Integrated Modular Avionics (IMA) define um ambiente compartimentado com funções de aviónica de diferentes criticalidades a coexistir numa plataforma computacional. A especificação relacionada ARINC 653 define uma interface padrão entre as aplicações e o sistema operativo subjacente. Ambas as especificações provêm do mundo da aviónica, mas estão a ganhar o interesse de parceiros da indústria espacial, que identificaram requisitos em comum entre as aplicações aeronáuticas e espaciais. No âmbito deste interesse, foi definida a arquitectura AIR, sob contrato da Agência Espacial Europeia (ESA). Esta arquitectura fornece segregação temporale espacial, e prevê o uso de diferentes sistemas operativos em cada partição. A segregação temporal é obtida através do escalonamento fixo e cíclico dos recursos às partições. Este trabalho estende a heterogeneidade prevista entre os sistemas operativos das partições (POS). Tal foi motivado pelas dificuldades documentadas em portar aplicações para sistemas operativos de tempo-real, e pela noção de que a integração apropriada de um POS não-tempo-real não comprometerá a pontualidade das funções críticas de tempo-real. Para este efeito, o Linux foi utilizado como caso de estudo. Uma variante embedida de Linux é construída e avaliada quanto à sua adequação como POS na arquitectura AIR. Para garantir uma integração segura, é proposta uma solução baseada na interface de paravirtualização do Linux, paravirt-ops. No decurso destas actividades, foram também feitas melhorias à definição da arquitectura AIR. O mais significante, motivado pelo pretendido aumento da heterogeneidade entre POSs, foi a introdução de um novo componente, AIR Partition OS Adaptation Layer (PAL). Este componente proporciona aos principais componentes da arquitectura AIR maior independência face ao POS, facilitando os esforços para a sua certificação independente. Outros melhoramentos fornecem mecanismos avançados de pontualidade, como mode-based schedules e monitorização de incumprimento de metas temporais de processos.ESA/ITI - European Space Agency Innovation Triangular Initiative (through ESTEC Contract 21217/07/NL/CB-Project AIR-II) and FCT - Fundação para a Ciência e Tecnologia (through the Multiannual Funding Programme

Flexible Scheduling in Middleware for Distributed rate-based real-time applications - Doctoral Dissertation, May 2002

Distributed rate-based real-time systems, such as process control and avionics mission computing systems, have traditionally been scheduled statically. Static scheduling provides assurance of schedulability prior to run-time overhead. However, static scheduling is brittle in the face of unanticipated overload, and treats invocation-to-invocation variations in resource requirements inflexibly. As a consequence, processing resources are often under-utilized in the average case, and the resulting systems are hard to adapt to meet new real-time processing requirements. Dynamic scheduling offers relief from the limitations of static scheduling. However, dynamic scheduling offers relief from the limitations of static scheduling. However, dynamic scheduling often has a high run-time cost because certain decisions are enforced on-line. Furthermore, under conditions of overload tasks can be scheduled dynamically that may never be dispatched, or that upon dispatch would miss their deadlines. We review the implications of these factors on rate-based distributed systems, and posits the necessity to combine static and dynamic approaches to exploit the strengths and compensate for the weakness of either approach in isolation. We present a general hybrid approach to real-time scheduling and dispatching in middleware, that can employ both static and dynamic components. This approach provides (1) feasibility assurance for the most critical tasks, (2) the ability to extend this assurance incrementally to operations in successively lower criticality equivalence classes, (3) the ability to trade off bounds on feasible utilization and dispatching over-head in cases where, for example, execution jitter is a factor or rates are not harmonically related, and (4) overall flexibility to make more optimal use of scarce computing resources and to enforce a wider range of application-specified execution requirements. This approach also meets additional constraints of an increasingly important class of rate-based systems, those with requirements for robust management of real-time performance in the face of rapidly and widely changing operating conditions. To support these requirements, we present a middleware framework that implements the hybrid scheduling and dispatching approach described above, and also provides support for (1) adaptive re-scheduling of operations at run-time and (2) reflective alternation among several scheduling strategies to improve real-time performance in the face of changing operating conditions. Adaptive re-scheduling must be performed whenever operating conditions exceed the ability of the scheduling and dispatching infrastructure to meet the critical real-time requirements of the system under the currently specified rates and execution times of operations. Adaptive re-scheduling relies on the ability to change the rates of execution of at least some operations, and may occur under the control of a higher-level middleware resource manager. Different rates of execution may be specified under different operating conditions, and the number of such possible combinations may be arbitrarily large. Furthermore, adaptive rescheduling may in turn require notification of rate-sensitive application components. It is therefore desirable to handle variations in operating conditions entirely within the scheduling and dispatching infrastructure when possible. A rate-based distributed real-time application, or a higher-level resource manager, could thus fall back on adaptive re-scheduling only when it cannot achieve acceptable real-time performance through self-adaptation. Reflective alternation among scheduling heuristics offers a way to tune real-time performance internally, and we offer foundational support for this approach. In particular, run-time observable information such as that provided by our metrics-feedback framework makes it possible to detect that a given current scheduling heuristic is underperforming the level of service another could provide. Furthermore we present empirical results for our framework in a realistic avionics mission computing environment. This forms the basis for guided adaption. This dissertation makes five contributions in support of flexible and adaptive scheduling and dispatching in middleware. First, we provide a middle scheduling framework that supports arbitrary and fine-grained composition of static/dynamic scheduling, to assure critical timeliness constraints while improving noncritical performance under a range of conditions. Second, we provide a flexible dispatching infrastructure framework composed of fine-grained primitives, and describe how appropriate configurations can be generated automatically based on the output of the scheduling framework. Third, we describe algorithms to reduce the overhead and duration of adaptive rescheduling, based on sorting for rate selection and priority assignment. Fourth, we provide timely and efficient performance information through an optimized metrics-feedback framework, to support higher-level reflection and adaptation decisions. Fifth, we present the results of empirical studies to quantify and evaluate the performance of alternative canonical scheduling heuristics, across a range of load and load jitter conditions. These studies were conducted within an avionics mission computing applications framework running on realistic middleware and embedded hardware. The results obtained from these studies (1) demonstrate the potential benefits of reflective alternation among distinct scheduling heuristics at run-time, and (2) suggest performance factors of interest for future work on adaptive control policies and mechanisms using this framework

Flexible Real-Time Linux a New Environment for Flexible Hard Real-Time Systems

[ES] La presente tesis propone un nuevo entorno general para la construcción de sistemas flexibles de tiempo real estricto, esto es, sistemas que necesitan de garantías de tiempo real estricto y de un comportamiento flexible. El entorno propuesto es capaz de integrar tareas con varios niveles de criticidad y diferentes paradigmas de planificación en el mismo sistema. Como resultado, el entorno permite proporcionar garantías de tiempo real estricto a las tareas críticas y además conseguir una planificación adaptativa e inteligente de las tareas menos críticas. El entorno se define en términos de un modelo de tareas, una arquitectura software y un conjunto de servicios. El modelo de tareas propone construir una aplicación flexible de tiempo real estricto como un conjunto de tareas, donde cada tarea se estructura en una secuencia de componentes obligatorios y opcionales. La arquitectura software propone separar la ejecución de las tareas en dos niveles de planificación interrelacionados, de manera que un nivel planifica los componentes obligatorios mediante una política de planificación de tiempo real estricto mientras que el otro nivel planifica los componentes opcionales mediante una política de planificación basada en la utilidad. El conjunto de servicios incluye, por una parte, un sistema de comunicación entre los componentes de las tareas (tanto obligatorios como opcionales) y, por otra, una serie de mecanismos para la detección y tratamiento de excepciones temporales producidas en ejecución. Por otra parte, la presente tesis muestra que el entorno teórico propuesto puede ser implementado realmente. En concreto, se presenta el diseño e implementación de un sistema de ejecución (es decir, un núcleo de sistema operativo) capaz de soportar las características de dicho entorno. Este sistema, denominado Flexible Real-Time Linux (FRTL), ha sido desarrollado a partir de un núcleo mínimo existente denominado Real-Time Linux (RT-Linux). Finalmente, esta tesis presenta una caracterización temporal completa del sistema FRTL y medidas reales de su sobrecarga. La caracterización temporal ha permitido el desarrollo de un test de garantía completo de todo el sistema (incluyendo la aplicación y el núcleo de FRTL), que puede ser utilizado para verificar las restricciones temporales de cualquier aplicación implementada sobre FRTL. Por su parte, las medidas de la sobrecarga de FRTL muestran que este núcleo ha sido diseñado e implementado de manera eficiente. En conjunto, se demuestra que el núcleo FRTL es a la vez predecible y eficiente, dos características que informan de su utilidad en la implementación real de aplicaciones flexibles de tiempo real estricto.[CA] Aquesta tesi proposa un nou entorn general per a la construcció de sistemes flexibles de temps real estricte, això és, sistemes que requereixen garanties de temps real estricte i un comportament flexible. L'entorn proposat és capaç d'integrar tasques amb diferentsnivells de criticitat i diferents paradigmes de planificació al mateix sistema. Com a resultat, l'entorn permet proporcionar garanties detemps real estricte a les tasques crítiques i a més aconseguir una planificació adaptativa i intel¿ligent de les tasques menys crítiques. L'entorn es defineix en termes d'un model de tasques, una arquitectura software i un conjunt de serveis. El model de tasques proposa la construcció d'una aplicació flexible de temps real estricte com a un conjunt de tasques on cadascuna és estructurada com una seqüència de components obligatòries i opcionals. L'arquitectura software proposa la separació de l'execució de les tasques en dos nivells de planificació interrelacionats, de manera que un nivel planifica les components obligatòries mitjançant una política de planificació de temps real estricte mentre que l'altre nivell planifica les components opcionals mitjançant una política de planificació basada en la utilitat. El conjunt de serveis inclou, per una part, un sistema de comunicació entre les components de les tasques (tant obligatòries com opcionals) i, per una altra, una sèrie de mecanismes per a la detecció i tractament d'excepcions temporals produïdes en execució. Per altra banda, la present tesi mostra que el proposat entorn teòric pot ésser implementat realment. En concret, es presenta el diseny i la implementació d'un sistema d'execució (es a dir, un nucli de sistema operatiu) capaç de suportar les característiques d'aquest entorn. Aquest sistema, anomenat Flexible Real-Time Linux (FRTL), ha sigut desenvolupat a partir d'un nucli mínim existent anomenat Real-Time Linux (RT-Linux). Finalment, aquesta tesi presenta una caracterització temporal completa del sistema FRTL i mesures reals de la seua sobrecàrrega. La caracterització temporal ha permés el desenvolupament d'un test de garantia complet de tot el sistema (incloent l'aplicació i el nucli FRTL), que pot ésser utilitzat per a verificar les restriccions temporals de qualsevol aplicació implementada sobre FRTL. Per la seua part, les mesures de la sobrecàrrega de FRTL mostren que aquest nucli ha sigut disenyat i implementat de manera eficient. En conjunt, es demostra que el nucli FRTL és al mateix temps predible i eficient, dos característiques que informen de la seua utilitat a la implementació real d'aplicacions flexibles de temps real estricte.Terrasa Barrena, AM. (2001). Flexible Real-Time Linux a New Environment for Flexible Hard Real-Time Systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/1806

Modular software architecture for flexible reservation mechanisms on heterogeneous resources

Management, allocation and scheduling of heterogeneous resources for complex distributed real-time applications is a chal- lenging problem. Timing constraints of applications may be fulfilled by a proper use of real-time scheduling policies, admission control and enforcement of timing constraints. However, it is not easy to design basic infrastructure services that allow for an easy access to the allocation of multiple heterogeneous resources in a distributed environment. In this paper, we present a middleware for providing distributed soft real-time applications with a uniform API for reserving heterogeneous resources with real-time scheduling capabilities in a distributed environment. The architecture relies on standard POSIX OS facilities, such as time management and standard TCP/IP networking services, and it is designed around CORBA, in order to facilitate modularity, flexibility and portability of the applications using it. However, real-time scheduling is supported by proper extensions at the kernel-level, plugged within the framework by means of dedicated resource managers. Our current implementation on Linux supports reservation of CPU, disk and network bandwidth. However, additional resource managers supporting alternative real-time schedulers for these resources, as well as additional types of resources, may be easily added. We present experimental results gathered on both synthetic applications and a real multimedia video streaming case study, showing advantages deriving from the use of the proposed middleware. Finally, overhead figures are reported, showing sustainability of the approach for a wide class of complex, distributed, soft real-time applications

A Hierarchical Scheduling Model for Dynamic Soft-Realtime System

We present a new hierarchical approximation and scheduling approach for applications and tasks with multiple modes on a single processor. Our model allows for a temporal and spatial distribution of the feasibility problem for a variable set of tasks with non-deterministic and fluctuating costs at runtime. In case of overloads an optimal degradation strategy selects one of several application modes or even temporarily deactivates applications. Hence, transient and permanent bottlenecks can be overcome with an optimal system quality, which is dynamically decided. This paper gives the first comprehensive and complete overview of all aspects of our research, including a novel CBS concept to confine entire applications, an evaluation of our system by using a video-on-demand application, an outline for adding further resource dimension, and aspects of our protoype implementation based on RTSJ

Segment Routing: a Comprehensive Survey of Research Activities, Standardization Efforts and Implementation Results

Fixed and mobile telecom operators, enterprise network operators and cloud providers strive to face the challenging demands coming from the evolution of IP networks (e.g. huge bandwidth requirements, integration of billions of devices and millions of services in the cloud). Proposed in the early 2010s, Segment Routing (SR) architecture helps face these challenging demands, and it is currently being adopted and deployed. SR architecture is based on the concept of source routing and has interesting scalability properties, as it dramatically reduces the amount of state information to be configured in the core nodes to support complex services. SR architecture was first implemented with the MPLS dataplane and then, quite recently, with the IPv6 dataplane (SRv6). IPv6 SR architecture (SRv6) has been extended from the simple steering of packets across nodes to a general network programming approach, making it very suitable for use cases such as Service Function Chaining and Network Function Virtualization. In this paper we present a tutorial and a comprehensive survey on SR technology, analyzing standardization efforts, patents, research activities and implementation results. We start with an introduction on the motivations for Segment Routing and an overview of its evolution and standardization. Then, we provide a tutorial on Segment Routing technology, with a focus on the novel SRv6 solution. We discuss the standardization efforts and the patents providing details on the most important documents and mentioning other ongoing activities. We then thoroughly analyze research activities according to a taxonomy. We have identified 8 main categories during our analysis of the current state of play: Monitoring, Traffic Engineering, Failure Recovery, Centrally Controlled Architectures, Path Encoding, Network Programming, Performance Evaluation and Miscellaneous...Comment: SUBMITTED TO IEEE COMMUNICATIONS SURVEYS & TUTORIAL

Automating Seccomp Filter Generation for Linux Applications

Software vulnerabilities in applications undermine the security of applications. By blocking unused functionality, the impact of potential exploits can be reduced. While seccomp provides a solution for filtering syscalls, it requires manual implementation of filter rules for each individual application. Recent work has investigated automated approaches for detecting and installing the necessary filter rules. However, as we show, these approaches make assumptions that are not necessary or require overly time-consuming analysis. In this paper, we propose Chestnut, an automated approach for generating strict syscall filters for Linux userspace applications with lower requirements and limitations. Chestnut comprises two phases, with the first phase consisting of two static components, i.e., a compiler and a binary analyzer, that extract the used syscalls during compilation or in an analysis of the binary. The compiler-based approach of Chestnut is up to factor 73 faster than previous approaches without affecting the accuracy adversely. On the binary analysis level, we demonstrate that the requirement of position-independent binaries of related work is not needed, enlarging the set of applications for which Chestnut is usable. In an optional second phase, Chestnut provides a dynamic refinement tool that allows restricting the set of allowed syscalls further. We demonstrate that Chestnut on average blocks 302 syscalls (86.5%) via the compiler and 288 (82.5%) using the binary-level analysis on a set of 18 widely used applications. We found that Chestnut blocks the dangerous exec syscall in 50% and 77.7% of the tested applications using the compiler- and binary-based approach, respectively. For the tested applications, Chestnut prevents exploitation of more than 62% of the 175 CVEs that target the kernel via syscalls. Finally, we perform a 6 month long-term study of a sandboxed Nginx server