Report from GI-Dagstuhl Seminar 16394: Software Performance Engineering in the DevOps World

This report documents the program and the outcomes of GI-Dagstuhl Seminar 16394 "Software Performance Engineering in the DevOps World". The seminar addressed the problem of performance-aware DevOps. Both, DevOps and performance engineering have been growing trends over the past one to two years, in no small part due to the rise in importance of identifying performance anomalies in the operations (Ops) of cloud and big data systems and feeding these back to the development (Dev). However, so far, the research community has treated software engineering, performance engineering, and cloud computing mostly as individual research areas. We aimed to identify cross-community collaboration, and to set the path for long-lasting collaborations towards performance-aware DevOps. The main goal of the seminar was to bring together young researchers (PhD students in a later stage of their PhD, as well as PostDocs or Junior Professors) in the areas of (i) software engineering, (ii) performance engineering, and (iii) cloud computing and big data to present their current research projects, to exchange experience and expertise, to discuss research challenges, and to develop ideas for future collaborations

Dimmer: Self-Adaptive Network-Wide Flooding with Reinforcement Learning

The last decade saw an emergence of Synchronous Transmissions (ST) as an effective communication paradigm in low-power wireless networks. Numerous ST protocols provide high reliability and energy efficiency in normal wireless conditions, for a large variety of traffic requirements. Recently, with the EWSN dependability competitions, the community pushed ST to harsher and highly-interfered environments, improving upon classical ST protocols through the use of custom rules, hand-tailored parameters, and additional retransmissions. The results are sophisticated protocols, that require prior expert knowledge and extensive testing, often tuned for a specific deployment and envisioned scenario. In this paper, we explore how ST protocols can benefit from self-adaptivity; a self-adaptive ST protocol selects itself its best parameters to (1) tackle external environment dynamics and (2) adapt to its topology over time. We introduce Dimmer as a self-adaptive ST protocol. Dimmer builds on LWB and uses Reinforcement Learning to tune its parameters and match the current properties of the wireless medium. By learning how to behave from an unlabeled dataset, Dimmer adapts to different interference types and patterns, and is able to tackle previously unseen interference. With Dimmer, we explore how to efficiently design AI-based systems for constrained devices, and outline the benefits and downfalls of AI-based low-power networking. We evaluate our protocol on two deployments of resource-constrained nodes achieving 95.8% reliability against strong, unknown WiFi interference. Our results outperform baselines such as non-adaptive ST protocols (27%) and PID controllers, and show a performance close to hand-crafted and more sophisticated solutions, such as Crystal (99%)

Adaptive Mechanisms for Mobile Spatio-Temporal Applications

Mobile spatio-temporal applications play a key role in many mission critical fields, including Business Intelligence, Traffic Management and Disaster Management. They are characterized by high data volume, velocity and large and variable number of mobile users. The design and implementation of these applications should not only consider this variablility, but also support other quality requirements such as performance and cost. In this thesis we propose an architecture for mobile spatio-temporal applications, which enables multiple angles of adaptivity. We also introduce a two-level adaptation mechanism that ensures system performance while facilitating scalability and context-aware adaptivity. We validate the architecture and adaptation mechanisms by implementing a road quality assessment mobile application as a use case and by performing a series of experiments on cloud environment. We show that our proposed architecture can adapt at runtime and maintain service level objectives while offering cost-efficiency and robustness

Collaborative, Intelligent, and Adaptive Systems for the Low-Power Internet of Things

With the emergence of the Internet of Things (IoT), more and more devices are getting equipped with communication capabilities, often via wireless radios. Their deployments pave the way for new and mission-critical applications: cars will communicate with nearby vehicles to coordinate at intersections; industrial wireless closed-loop systems will improve operational safety in factories; while swarms of drones will coordinate to plan collision-free trajectories. To achieve these goals, IoT devices will need to communicate, coordinate, and collaborate over the wireless medium. However, these envisioned applications necessitate new characteristics that current solutions and protocols cannot fulfill: IoT devices require consistency guarantees from their communication and demand for adaptive behavior in complex and dynamic environments.In this thesis, we design, implement, and evaluate systems and mechanisms to enable safe coordination and adaptivity for the smallest IoT devices. To ensure consistent coordination, we bring fault-tolerant consensus to low-power wireless communication and introduce Wireless Paxos, a flavor of the Paxos algorithm specifically tailored to low-power IoT. We then present STARC, a wireless coordination mechanism for intersection management combining commit semantics with synchronous transmissions. To enable adaptivity in the wireless networking stack, we introduce Dimmer and eAFH. Dimmer combines Reinforcement Learning and Multi-Armed Bandits to adapt its communication parameters and counteract the adverse effects of wireless interference at runtime while optimizing energy consumption in normal conditions. eAFH provides dynamic channel management in Bluetooth Low Energy by excluding and dynamically re-including channels in scenarios with mobility. Finally, we demonstrate with BlueSeer that a device can classify its environment, i.e., recognize whether it is located in a home, office, street, or transport, solely from received Bluetooth Low Energy signals fed into an embedded machine learning model. BlueSeer therefore increases the intelligence of the smallest IoT devices, allowing them to adapt their behaviors to their current surroundings

Intelligence at the Extreme Edge: A Survey on Reformable TinyML

The rapid miniaturization of Machine Learning (ML) for low powered processing has opened gateways to provide cognition at the extreme edge (E.g., sensors and actuators). Dubbed Tiny Machine Learning (TinyML), this upsurging research field proposes to democratize the use of Machine Learning (ML) and Deep Learning (DL) on frugal Microcontroller Units (MCUs). MCUs are highly energy-efficient pervasive devices capable of operating with less than a few Milliwatts of power. Nevertheless, many solutions assume that TinyML can only run inference. Despite this, growing interest in TinyML has led to work that makes them reformable, i.e., work that permits TinyML to improve once deployed. In line with this, roadblocks in MCU based solutions in general, such as reduced physical access and long deployment periods of MCUs, deem reformable TinyML to play a significant part in more effective solutions. In this work, we present a survey on reformable TinyML solutions with the proposal of a novel taxonomy for ease of separation. Here, we also discuss the suitability of each hierarchical layer in the taxonomy for allowing reformability. In addition to these, we explore the workflow of TinyML and analyze the identified deployment schemes and the scarcely available benchmarking tools. Furthermore, we discuss how reformable TinyML can impact a few selected industrial areas and discuss the challenges and future directions

Energy Efficient Task Mapping and Resource Management on Multi-core Architectures

Reducing energy consumption of parallel applications executing on chip multi- processors (CMPs) is important for green computing. Hardware vendors have been developing a variety of system features to support energy efficient computing, for example, integrating asymmetric core types on a single chip referred to as static asymmetry and supporting dynamic voltage and frequency scaling (DVFS) referred to as dynamic asymmetry.A common parallelization scheme to exploit CMPs is task parallelism, which can express a wide range of computations in the form of task directed acyclic graphs (DAGs). Existing studies that target energy efficient task scheduling have demonstrated the benefits of leveraging DVFS, particularly per-core DVFS. Their scheduling decisions are mainly based on heuristics, such as task criticality, task dependencies and workload sizes. To enable energy efficient task scheduling, we identify multiple crucial factors that influence energy consumption - varying task characteristics, exploitation of intra-task parallelism (task moldability), and task granularity - which we collectively refer to as task heterogeneity. Task heterogeneity and architecture asymmetry features together complicate the task scheduling problem, since the most energy efficient configuration of resource allocation and frequency setting varies with each task. Our analysis shows that leveraging task heterogeneity in conjunction with static and dynamic asymmetry provides significant opportunities for energy reduction.This thesis contributes two scheduling techniques - ERASE and STEER - that target different scenarios. ERASE focuses on fine-grained tasking and in environments where DVFS is not under user control. It leverages the insights of task characteristics, task moldability, and instantaneous task parallelism detection for guiding scheduling decisions. ERASE comprises four modules: online performance modeling, power profiling, core activity tracing and a task scheduler. Online performance modeling and power profiling provide runtime with execution time and power predictions. Core activity tracing offers the instantaneous task parallelism and the task scheduler combines these information to enable the energy predictions and dynamically determine the best resource allocation for each task during runtime. STEER focuses on environments where DVFS is under user control and where the platform comprises multiple asymmetric cores grouped into clusters. STEER explores how much energy could be potentially saved by leveraging static asymmetry, dynamic asymmetry and task heterogeneity in conjunction. STEER comprises two predictive models for performance and power predictions, and a task scheduler that utilizes models for energy predictions and then identifies the best resource allocation and frequency settings for tasks. Moreover, it applies adaptive scheduling techniques based on task granularity to manage DVFS overheads, and coordinates the cluster frequency settings to reduce interference from concurrent running tasks on cluster-based architectures.The evaluation on an NVIDIA Jetson TX2 shows that ERASE achieves 10% energy savings on average compared to the state-of-the-art DVFS-based schedulers and can adapt to external DVFS changes, and STEER consumes 38% less energy on average than both the state-of-the-art and ERASE