Learning to Communicate Using Counterfactual Reasoning

This paper introduces a new approach for multi-agent communication learning called multi-agent counterfactual communication (MACC) learning. Many real-world problems are currently tackled using multi-agent techniques. However, in many of these tasks the agents do not observe the full state of the environment but only a limited observation. This absence of knowledge about the full state makes completing the objectives significantly more complex or even impossible. The key to this problem lies in sharing observation information between agents or learning how to communicate the essential data. In this paper we present a novel multi-agent communication learning approach called MACC. It addresses the partial observability problem of the agents. MACC lets the agent learn the action policy and the communication policy simultaneously. We focus on decentralized Markov Decision Processes (Dec-MDP), where the agents have joint observability. This means that the full state of the environment can be determined using the observations of all agents. MACC uses counterfactual reasoning to train both the action and the communication policy. This allows the agents to anticipate on how other agents will react to certain messages and on how the environment will react to certain actions, allowing them to learn more effective policies. MACC uses actor-critic with a centralized critic and decentralized actors. The critic is used to calculate an advantage for both the action and communication policy. We demonstrate our method by applying it on the Simple Reference Particle environment of OpenAI and a MNIST game. Our results are compared with a communication and non-communication baseline. These experiments demonstrate that MACC is able to train agents for each of these problems with effective communication policies.Comment: Submitted to NeurIPS2020. Contains 10 pages with 9 figures and 4 appendice

Task-Set Generator for Schedulability Analysis using the TACLeBench benchmark suite

ABSTRACT Current real-time embedded systems evolve towards complex systems using new state of the art technologies such as multi-core processors and virtualization techniques. Both technologies requires new real-time scheduling algorithms. For uniprocessor scheduling, utilization-based evaluation methodologies are common and well-established. For multicore systems and virtualization, evaluating and comparing scheduling techniques using the tasks' parameters is more realistic. Evaluating these different scheduling techniques requires relevant and standardised task sets. Scheduling algorithms can be evaluated on three evaluation levels: 1) by using the mathematical model of the scheduling algorithm, 2) by simulating the scheduling algorithm and 3) by implementing the algorithm on the target platform. Generating task sets is straightforward in case of the first two levels; only the parameters of the tasks are required. Evaluating and comparing scheduling algorithms on the target platform itself, however, requires real executable tasks matching the predefined standardised task sets. Generating those executable tasks is not standardized yet. Therefore, we developed a task-set generator that generates reproducible, standardised task sets that are suitable at all levels. Beside generating the tasks' parameters, it includes an executable generator methodology that generates executables by combining publicly available benchmarks with know execution times. This paper presents and evaluates this task-set generator. The executables approximate the wanted execution time on the hardware platform

Vehicular communication management framework : a flexible hybrid connectivity platform for CCAM services

In the upcoming decade and beyond, the Cooperative, Connected and Automated Mobility (CCAM) initiative will play a huge role in increasing road safety, traffic efficiency and comfort of driving in Europe. While several individual vehicular wireless communication technologies exist, there is still a lack of real flexible and modular platforms that can support the need for hybrid communication. In this paper, we propose a novel vehicular communication management framework (CAMINO), which incorporates flexible support for both short-range direct and long-range cellular technologies and offers built-in Cooperative Intelligent Transport Systems' (C-ITS) services for experimental validation in real-life settings. Moreover, integration with vehicle and infrastructure sensors/actuators and external services is enabled using a Distributed Uniform Streaming (DUST) framework. The framework is implemented and evaluated in the Smart Highway test site for two targeted use cases, proofing the functional operation in realistic environments. The flexibility and the modular architecture of the hybrid CAMINO framework offers valuable research potential in the field of vehicular communications and CCAM services and can enable cross-technology vehicular connectivity

Requirements and Specifications for the Orchestration of Network Intelligence in 6G

Next-generation mobile networks are expected to flaunt highly (if not fully) automated management. Network Intelligence (NI) will be the key enabler for such a vision, empowering myriad of orchestrators and controllers across network domains. In this paper, we elaborate on the DAEMON architectural model, which proposes introducing a NI Orchestration layer for the effective end-to-end coordination of NI instances deployed across the whole mobile network infrastructure. Specifically, we first outline requirements and specifications for NI design that stem from data management, control timescales, and network technology characteristics. Then, we build on such analysis to derive initial principles for the design of the NI Orchestration layer, focusing on (i) proposals for the interaction loop between NI instances and the NI Orchestrator, and (ii) a unified representation of NI algorithms based on an extended MAPE-K model. Our work contributes to the definition of the interfaces and operation of a NI Orchestration layer that foster a native integration of NI in mobile network architectures.This project has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement no.101017109 DAEMON

A New Hybrid Approach on WCET Analysis for Real-Time Systems Using Machine Learning

The notion of the Worst-Case Execution Time (WCET) allows system engineers to create safe real-time systems. This value is used to schedule all software tasks before their deadlines. Failing these deadlines will cause catastrophic events, e.g. vehicle crashes, failing to detect dangerous anomalies, etc. Different analysis methodologies exist to determine the WCET. However, these methods do not provide early insight in the WCET during development. Therefore, pessimistic assumptions are made by system designers resulting in more expensive, overqualified hardware. In this paper, an extension on the hybrid methodology is proposed which implements a predictor model using Machine Learning (ML). This new approach estimates the WCET on smaller entities of the code, so-called hybrid blocks, based on software and hardware features. As a result, the ML-based hybrid analysis provides insight of the WCET early-on in the development process and refines its estimate when more detailed features are available. In order to facilitate the extraction of code-related features, a new tool for the COBRA framework is proposed. This paper proves the potential of the ML-based hybrid approach by conducting multiple experiments based on the TACLeBench on a first prototype. A set of annotated code features were used to train and validate eight different regression models. The results already show promising estimates without tuning any hyperparameters, proving the potential of the methodology

Task-set generator for schedulability analysis using the TACLeBench benchmark suite

ABSTRACT Current real-time embedded systems evolve towards complex systems using new state of the art technologies such as multi-core processors and virtualization techniques. Both technologies requires new real-time scheduling algorithms. For uniprocessor scheduling, utilization-based evaluation methodologies are common and well-established. For multicore systems and virtualization, evaluating and comparing scheduling techniques using the tasks' parameters is more realistic. Evaluating these different scheduling techniques requires relevant and standardised task sets. Scheduling algorithms can be evaluated on three evaluation levels: 1) by using the mathematical model of the scheduling algorithm, 2) by simulating the scheduling algorithm and 3) by implementing the algorithm on the target platform. Generating task sets is straightforward in case of the first two levels; only the parameters of the tasks are required. Evaluating and comparing scheduling algorithms on the target platform itself, however, requires real executable tasks matching the predefined standardised task sets. Generating those executable tasks is not standardized yet. Therefore, we developed a task-set generator that generates reproducible, standardised task sets that are suitable at all levels. Beside generating the tasks' parameters, it includes an executable generator methodology that generates executables by combining publicly available benchmarks with know execution times. This paper presents and evaluates this task-set generator. The executables approximate the wanted execution time on the hardware platform