Deep Reinforcement Learning for Smart Queue Management

With the goal of meeting the stringent throughput and delay requirements of classified network flows, we propose a Deep Q-learning Network (DQN) for optimal weight selection in an active queue management system based on Weighted Fair Queuing (WFQ). Our system schedules flows belonging to different priority classes (Gold, Silver, and Bronze) into separate queues, and learns how and when to dequeue from each queue. The neural network implements deep reinforcement learning tools such as target networks and replay buffers to help learn the best weights depending on the network state. We show, via simulations, that our algorithm converges to an efficient model capable of adapting to the flow demands, producing thus lower delays with respect to traditional WFQ

Mixed-Criticality on the AFDX Network: Challenges and Potential Solutions

In this paper, we first assess the most relevant existing solutions enabling mixed-criticality on the AFDX and select the most adequate one. Afterwards, the specification of an extended AFDX, based on the Burst-Limiting Shaper (BLS), is detailed to fulfill the main avionics requirements and challenges. Finally, the preliminary evaluation of such a proposal is conducted through simulations. Results show its ability to guarantee the highest criticality traffic constraints, while limiting its impact on the current AFDX traffic

On Time Synchronization Issues in Time-Sensitive Networks with Regulators and Nonideal Clocks

Flow reshaping is used in time-sensitive networks (as in the context of IEEE TSN and IETF Detnet) in order to reduce burstiness inside the network and to support the computation of guaranteed latency bounds. This is performed using per-flow regulators (such as the Token Bucket Filter) or interleaved regulators (as with IEEE TSN Asynchronous Traffic Shaping). Both types of regulators are beneficial as they cancel the increase of burstiness due to multiplexing inside the network. It was demonstrated, by using network calculus, that they do not increase the worst-case latency. However, the properties of regulators were established assuming that time is perfect in all network nodes. In reality, nodes use local, imperfect clocks. Time-sensitive networks exist in two flavours: (1) in non-synchronized networks, local clocks run independently at every node and their deviations are not controlled and (2) in synchronized networks, the deviations of local clocks are kept within very small bounds using for example a synchronization protocol (such as PTP) or a satellite based geo-positioning system (such as GPS). We revisit the properties of regulators in both cases. In non-synchronized networks, we show that ignoring the timing inaccuracies can lead to network instability due to unbounded delay in per-flow or interleaved regulators. We propose and analyze two methods (rate and burst cascade, and asynchronous dual arrival-curve method) for avoiding this problem. In synchronized networks, we show that there is no instability with per-flow regulators but, surprisingly, interleaved regulators can lead to instability. To establish these results, we develop a new framework that captures industrial requirements on clocks in both non-synchronized and synchronized networks, and we develop a toolbox that extends network calculus to account for clock imperfections.Comment: ACM SIGMETRICS 2020 Boston, Massachusetts, USA June 8-12, 202

Vers la convergence de réseaux dans l'avionique

AFDX est le standard Ethernet commuté utilisé pour la transmission des flux avioniques. Pour des raisons de certification, le réseau AFDX déployé à présent dans les avions civils est très peu chargé. Cette thèse vise à étudier la possibilité envisagée par les avionneurs d’utiliser la bande passante AFDX restante pour transporter des flux non-avioniques additionnels (vidéo, audio, service). Ces flux ne doivent pas affecter les délais de transmission des flux avioniques. Pour multiplexer des flux avioniques et non-avioniques des politiques d’ordonnancement sont nécessaires au niveau des systèmes d’extrémité (end systems) et des commutateurs. Dans cette thèse, nous considérons l’exemple de la transmission sur AFDX de flux vidéo provenant des caméras de surveillance de l’avion. Le multiplexage des flux avioniques et vidéo est réalisé par l’introduction d’une table d’ordonnancement au niveau des end systems émetteurs et d’une politique de type SPQ dans les ports de sortie du commutateur. Cette solution préserve les contraintes temps-réel des flux avioniques, mais peut introduire des variations sur les délais de bout-en-bout des flux vidéo. Une allocation appropriée des flux avioniques dans la table d’ordonnancement peut réduire le retard d’émission des flux vidéo et ainsi, limiter les variations de délai. Nous proposons deux stratégies d’allocation des flux avioniques dans la table d’ordonnancement : une heuristique simple et une allocation optimale. L’allocation optimale est dérivée en résolvant un problème d’optimisation par contraintes qui minimise le retard d’émission des flux vidéo. Dans le cas des end systems moins chargés, l’allocation par heuristique est proche de l’optimale

Deficit Round Robin with Network Calculus

Generalised Processor Sharing (GPS) is a well-known ideal service policy designed to share the capacity of a server among the input flows fairly: each backlogged flow receives a pre-defined fraction of the total server capacity, according to its weight. Several practical implementations of GPS have been proposed, among which Deficit Round Robin (DRR) is widely deployed since it can be implemented in a very efficient way. The worst-case performance of DRR has been studied by several papers, all of which assume that the shared server has a constant rate. This paper studies DRR using Network Calculus, under very general assumptions. Latency results that generalise all the previous works are derived, and a residual service is derived from DRR parameters. This residual service is shown to be as good as or even better than previous studies when restricting it to the same assumptions