Benchmarking the Round-Trip Latency of Various Java-Based Middleware Platforms

International audienceNowadays, distributed Java-based applications could be built on top of a plethora of middleware technologies such as Object Request Brokers (ORB) like CORBA and Java RMI, Web Services, and component-oriented platforms like EJB or CCM. Choosing the right technology fitting with application requirements is driven by various criteria such as economic costs, available features, performance, etc. The main contribution of this paper is to present an experience report on the design and implementation of a simple benchmark to evaluate the round-trip latency of various Java-based middleware platforms. Empirical results and analysis are discussed on a large set of widely available implementations including various ORB (Java RMI, Java IDL, ORBacus, JacORB, OpenORB, and Ice), Web Services projects (Apache XML-RPC and Axis), and componentoriented platforms (JBoss, JOnAS, OpenCCM, Fractal, Pro-Active)

BORDER: A Benchmarking Framework for Distributed MQTT Brokers

[EN] Message queuing telemetry transport (MQTT), one of the most popular application layer protocols for the Internet of Things, works according to a publish/subscribe paradigm where clients connect to a centralized broker. Sometimes (e.g., in high scalability and low-latency applications), it is required to depart from such a centralized approach and move to a distributed one, where multiple MQTT brokers cooperate together. Many MQTT brokers (both open source or commercially available) allow to create such a distributed environment: however, it is challenging to select the right solution due to the many available choices. This article proposes, therefore benchmarking framework for distributed MQTT brokers (BORDER), a framework for creating and evaluating distributed architectures of MQTT brokers with realistic and customizable network topologies. Based on isolated Docker containers and emulated network components, the framework provides quantitative metrics about the overall system performance, such as End-to-End latency as well as network and physical resources consumed. We use BORDER to compare five of the most popular MQTT brokers that allow the creation of distributed architectures and we release it as an open-source project to allow for reproducible researches.This work was supported in part by the Project BASE5G under Project 1155850 funded by Regione Lombardia within the framework POR FESR 2014-2020.Longo, E.; Redondi, A.; Cesana, M.; Manzoni, P. (2022). BORDER: A Benchmarking Framework for Distributed MQTT Brokers. IEEE Internet of Things. 9(18):17728-17740. https://doi.org/10.1109/JIOT.2022.3155872177281774091

Asynchronous Integration of Real-Time Simulators for HIL-based Validation of Smart Grids

As the landscape of devices that interact with the electrical grid expands, also the complexity of the scenarios that arise from these interactions increases. Validation methods and tools are typically domain specific and are designed to approach mainly component level testing. For this kind of applications, software and hardware-in-the-loop based simulations as well as lab experiments are all tools that allow testing with different degrees of accuracy at various stages in the development life-cycle. However, things are vastly different when analysing the tools and the methodology available for performing system-level validation. Until now there are no available well-defined approaches for testing complex use cases involving components from different domains. Smart grid applications would typically include a relatively large number of physical devices, software components, as well as communication technology, all working hand in hand. This paper explores the possibilities that are opened in terms of testing by the integration of a real-time simulator into co-simulation environments. Three practical implementations of such systems together with performance metrics are discussed. Two control-related examples are selected in order to show the capabilities of the proposed approach.Comment: IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Societ

FaaSdom: A Benchmark Suite for Serverless Computing

Serverless computing has become a major trend among cloud providers. With serverless computing, developers fully delegate the task of managing the servers, dynamically allocating the required resources, as well as handling availability and fault-tolerance matters to the cloud provider. In doing so, developers can solely focus on the application logic of their software, which is then deployed and completely managed in the cloud. Despite its increasing popularity, not much is known regarding the actual system performance achievable on the currently available serverless platforms. Specifically, it is cumbersome to benchmark such systems in a language- or runtime-independent manner. Instead, one must resort to a full application deployment, to later take informed decisions on the most convenient solution along several dimensions, including performance and economic costs. FaaSdom is a modular architecture and proof-of-concept implementation of a benchmark suite for serverless computing platforms. It currently supports the current mainstream serverless cloud providers (i.e., AWS, Azure, Google, IBM), a large set of benchmark tests and a variety of implementation languages. The suite fully automatizes the deployment, execution and clean-up of such tests, providing insights (including historical) on the performance observed by serverless applications. FaaSdom also integrates a model to estimate budget costs for deployments across the supported providers. FaaSdom is open-source and available at https://github.com/bschitter/benchmark-suite-serverless-computing.Comment: ACM DEBS'2

Techniques and Patterns for Safe and Efficient Real-Time Middleware

Over 90 percent of all microprocessors are now used for real-time and embedded applications. The behavior of these applications is often constrained by the physical world. It is therefore important to devise higher-level languages and middleware that meet conventional functional requirements, as well as dependably and productively enforce real-time constraints. Real-Time Java is emerging as a safe, real-time environment. In this thesis we use it as our experimentation platform; however, our findings are easily adapted to other similar platforms. This thesis provides the following contributions to the study of safe and efficient real-time middleware. First, it identifies potential bottlenecks and problem with respect to guaranteeing real-time performance in middleware. Second, it presents a series of techniques and patterns that allow the design and implementation of safe, predictable, and highly efficient real-time middleware. Third, it provides a set of architectural and design patterns that application developers can use when designing real-time systems. Finally, it provides a methodology for evaluating the merits and benefits of real-time middleware. Empirical results are presented using that methodology for the techniques presented in this thesis. The methodology helps compare the performance and predictability of general, real-time middleware platforms