Poster: In-Network Total Order Guarantees supporting State Machine Replication with P4 Programmable Switches

In this paper we propose P4-based Atomic Multicast (P4mCast), a new in-network atomic multicast protocol to support total order guarantees for State Machine Replication (SMR) in cloud-based fault-tolerant and distributed applications. P4mCast builds on in-network computing, applying leader-based consensus to groups of prominent P4 programmable switches in modern data center networks. P4mCast achieves significantly lower latency overhead in the microseconds scale while increasing the throughput one order of magnitude higher compared to state of the art software-based solutions

Poster: In-Network Total Order Guarantees supporting State Machine Replication with P4 Programmable Switches

In this paper we propose P4-based Atomic Multicast (P4mCast), a new in-network atomic multicast protocol to support total order guarantees for State Machine Replication (SMR) in cloud-based fault-tolerant and distributed applications. P4mCast builds on in-network computing, applying leader-based consensus to groups of prominent P4 programmable switches in modern data center networks. P4mCast achieves significantly lower latency overhead in the microseconds scale while increasing the throughput one order of magnitude higher compared to state of the art software-based solutions

Netlang : un langage de haut niveau pour les routeurs programmables dans le contexte des réseaux SDN

Développer des applications réseaux pour des routeurs programmables basés sur les Network Processors (NPs) implique l'utilisation de langages de bas-niveau et d'outils propriétaires fortement dépendants des architectures matérielles sous-jacentes. Le code source, généralement écrit en langage assembleur, n'est pas facile à écrire et cause des problèmes de maintenance. Les applications résultantes sont également difficiles à déboguer. Dans ce mémoire nous proposons NETLANG, un nouveau langage de programmation de haut-niveau dédié aux NPs. De plus d'être un langage simple et élégant, de réduire les coûts de développement et de la maintenance, et d'améliorer la réutilisation du code, NETLANG a pour objectif essentiel de décrire le comportement des paquets dans un NP. NETLANG est un langage qui permet de développer des applications de traitement de paquets. Il établit deux niveaux. Le premier niveau du langage offre une abstraction et une description du routeur à travers un pipeline de tables OpenFlow et des règles de forwarding ayant l'aptitude d'être modifiées dynamiquement et donc de permettre de changer le comportement du routeur à la volée. La sémantique du langage est inspirée du protocole OpenFlow qui a permis d'exprimer les principales tâches de traitement de paquets telles que le parsing, le lookup et la modification. Le langage est bâti en respectant le modèle des Software Defined Networks (SDNs) qui définit un nouveau plan de séparation entre le control plane et le data plane. Le deuxième niveau de NETLANG est traduit en matériel et permet l'adaptabilité du langage à plusieurs plateformes. Des adaptateurs spécifiques à des plateformes différentes sont intégrés au compilateur de NETLANG et permettent de rendre le langage portable. En effet, nous avons utilisé deux environnements pour l'implémentation de NETLANG ; le NP4 d'EZchip caractérisé par sa structure de TOPs (Task Optimized Processors) en pipeline et le NFP-3240 de Netronome connu pour son parallélisme et l'exploitation du multithreading. La validation de NETLANG s'est basée sur un ensemble d'applications réseau ayant des complexités et des domaines différents. A travers ce mémoire nous avons démontré qu'on est capable d'avoir aujourd'hui un langage pour les routeurs programmables. La sémantique d'OpenFlow, sur laquelle nous avons basé notre langage NETLANG, est suffisante et même pertinente en termes de description de comportement des paquets dans un NP.\ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : langages à domaine spécifique, réseaux programmables, processeurs de réseau

Window-based Parallel Operator Execution with In-Network Computing

Data parallel processing is a key concept to increase the scalability and elasticity in event streaming systems. Often data parallelism is accomplished in a splitter-merger architecture where the splitter divides incoming streams into partitions and forwards them to parallel operator instances. The splitter performance is a limiting factor to the system throughput and the parallelization degree. This work studies how to leverage novel methods of in-network computing to accelerate the splitter functionality by implementing it as an in-network function. While dedicated hardware for in-network computing has a high potential to enhance the splitter performance, in-network programming models like the P4 language are also highly limited in their expressiveness to support corresponding parallelization models. We propose P4 Splitter Switch (P4SS) which supports overlapping and non-overlapping count-based windows for multiple independent data streams and parallelizes them to a dynamically configurable number of operator instances. We validate in the context of a prototypical implementation our splitting strategy and its scalability in terms of switch resource consumption

In-Network Management of Parallel Data Streams over Programmable Data Planes

Current data centers host an ever increasing number of data analytics applications who are dealing with a growing number of data sources and a continuously increasing volume of data. Parallel stream processing is a powerful paradigm supporting the large-scale deployment of data-analytics applications. However, its performance is limited by its processing capacity of splitting the data streams into parallelizable sub-streams. The splitter that is traditionally executed on general purpose computational resources can benefit from in-network computing nodes on the communication path. Programmable data planes and corresponding programming models, e.g., Programming Protocol-independent Packet Processors (P4), offer the flexibility of enabling distinct parallelization semantics that can be individually adapted to the dynamic workload. In this paper, we propose Stateful and Scalable Splitter Switch (S4), a network-centric approach leveraging P4 to support parallel stream processing. S4 supports up to 286k concurrent data streams, with a parallelism degree of up to ∼ 500k operator instances and a latency overhead of only 2µs

Accelerating the Performance of Data Analytics using Network-centric Processing

Distributed execution of real-time data analytics such as event stream processing is the key to scalability, performance and reliable detection of situation changes. Although real-time analytics is highly I/O centric, existing methods supporting the efficient execution of data analytics functions mostly rely on traditional compute models that are available in data centers, e.g., CPU or GPU based processing models, but treat the network mainly as a blackbox. However, with recent advance in software-defined networking (SDN) and the standardization of packet processing pipeline, data analytics functions can be offloaded to programmable switches and benefit from hardware acceleration in an easier and more flexible way than a decade ago. In this paper we focus on the potential of in-network processing to enhance the performance of the overall real-time data analytics application. We aim to contribute to an (i) understanding on how in-network processing can accelerate real-time data analytics and (ii) assess what models of in-network computing can accelerate which event processing functions considering the limitations of network models compared to traditional compute models. We motivate the potential and illustrate the research problems in the context of load balancing which is an important concept in the data-parallel execution of event processing systems