Assessing Black-box Test Case Generation Techniques for Microservices

Testing of microservices architectures (MSA) – today a popular software architectural style - demands for automation in its several tasks, like tests generation, prioritization and execution. Automated black-box generation of test cases for MSA currently borrows techniques and tools from the testing of RESTful Web Services. This paper: i) proposes the uTest stateless pairwise combinatorial technique (and its automation tool) for test cases generation for functional and robustness microservices testing, and ii) experimentally compares - with three open-source MSA used as subjects - four state-of-the-art black-box tools conceived for Web Services, adopting evolutionary-, dependencies- and mutation-based generation techniques, and the pro- posed uTest combinatorial tool. The comparison shows little differences in coverage values; uTest pairwise testing achieves better average failure rate with a considerably lower number of tests. Web Services tools do not perform for MSA as well as a tester might expect, highlighting the need for MSA-specific techniques

MicroFuzz: An Efficient Fuzzing Framework for Microservices

This paper presents a novel fuzzing framework, called MicroFuzz, specifically designed for Microservices. Mocking-Assisted Seed Execution, Distributed Tracing, Seed Refresh and Pipeline Parallelism approaches are adopted to address the environmental complexities and dynamics of Microservices and improve the efficiency of fuzzing. MicroFuzz has been successfully implemented and deployed in Ant Group, a prominent FinTech company. Its performance has been evaluated in three distinct industrial scenarios: normalized fuzzing, iteration testing, and taint verification.Throughout five months of operation, MicroFuzz has diligently analyzed a substantial codebase, consisting of 261 Apps with over 74.6 million lines of code (LOC). The framework's effectiveness is evident in its detection of 5,718 potential quality or security risks, with 1,764 of them confirmed and fixed as actual security threats by software specialists. Moreover, MicroFuzz significantly increased program coverage by 12.24% and detected program behavior by 38.42% in the iteration testing.Comment: Accepted by ICSE-SEIP 202

Leveraging Distributed Tracing and Container Cloning for Replay Debugging of Microservices

Microservice architectures have gained prominence in recent years for building large-scale industrial distributed systems. However, microservice architectures make the usage of replay debugging, a powerful technique for finding root causes of faults, very challenging because of the polyglot (written in several languages) services, large accumulated state of services, and tight latency limits imposed by long hop-chains. This work attempts to provide a framework for enabling replay debugging in production microservice applications. We study 25 real-world faults in microservice systems collected from diverse sources, categorize these faults by fault symptoms, and create 15 application agnostic mutation operators for microservices. We then propose a language agnostic replay debugging framework for microservice applications that uses a distributed tracing system to record network requests and enables replay of those requests on cloned service containers running in a debug environment. A key component of this framework is an anomaly detector that uses span-level and container-level monitoring to detect fault symptoms found in our study and localizes faults to trace level so that faulty traces can be easily replayed to find the root cause. An open-source microservices application injected successively with the mutation operators is used for an evaluation that shows that our framework is upto an order of magnitude lighter-weight than language-specific recording tools such as Chrome DevTools or VisualVM and can help in finding root causes of 9 out of 15 mutations at a line or function level

Management and Security of IoT systems using Microservices

Devices that assist the user with some task or help them to make an informed decision are called smart devices. A network of such devices connected to internet are collectively called as Internet of Things (IoT). The applications of IoT are expanding exponentially and are becoming a part of our day to day lives. The rise of IoT led to new security and management issues. In this project, we propose a solution for some major problems faced by the IoT devices, including the problem of complexity due to heterogeneous platforms and the lack of IoT device monitoring for security and fault tolerance. We aim to solve the above issues in a microservice architecture. We build a data pipeline for IoT devices to send data through a messaging platform Kafka and monitor the devices using the collected data by making real time dashboards and a machine learning model to give better insights of the data. For proof of concept, we test the proposed solution on a heterogeneous cluster, including Raspberry Pi’s and IoT devices from different vendors. We validate our design by presenting some simple experimental results

The cloudification perspectives of search-based software testing

To promote and sustain the future of our society, the most critical challenge of contemporary software engineering and cloud computing experts are related to the efficient integration of emerging cloudification and DevOps practices in the development and testing processes of modern systems. In this context, we argue that SBST can play a critical role in improving testing practices and automating the verification and validation (V&V) of cloudification properties of Cloud Native Applications (CNA). Hence, in this paper, we focus on the untouched side of SBST in the cloud field, by discussing (1) the testing challenges in the cloud research field and (2) summarizing the recent contributions of SBST in supporting development practices of CNA. Finally, we discuss the emerging research topics characterizing the cloudification perspectives of SBST in the cloud field

Genetic Algorithm for Multi-Objective Optimization of Container Allocation in Cloud Architecture

The use of containers in cloud architectures has become widespread because of advantages such as limited overhead, easier and faster deployment and higher portability. Moreover, they are a suitable architectural solution for deployment of applications created using a microservices development pattern. Despite the large number of solutions and implementations, open issues have not been addressed in container automation and management. Container resource allocation influences system performance and resource consumption so it is a key factor for cloud providers. We propose a genetic algorithm approach, using the Non-dominated Sorting Genetic Algorithm-II (NSGA-II), to optimize container allocation and elasticity management due to the good results obtained with this algorithm in other resource management optimization problems in cloud architectures. The optimization has been focused on a tight use of the resources and a reduction of the network overhead and system failure rate. A model for cloud cluster, containers, microservices and four optimization objectives is presented. Experimental results have shown that our approach is a suitable solution to address the problem of container allocation and elasticity and it obtains better objectives values than the container management policies implemented in Kubernetes

Immutable Infrastructure Calls for Immutable Architecture

With the advent of cloud computing and the concept of immutable infrastructure, the scaling and deployment of applications has become significantly easier. This increases the possibility of “configuration drift” as an operations team manages this cluster of machines, both virtual and actual. In this paper we propose a revised view on configuration and architecture. We propose that software deployed on a public or private cloud should, to the furthest possible extent, be immutable and source controlled. This reduces configuration drift and ensures no configuration problems in production as a result of updates or changes. We will show an example of a software project deployed on Amazon Web Services with an immutable Jenkins setup which manages updating the whole cluster and is self-regenerating. We will also discuss how this lends itself naturally to interoperability between clouds, because of the infrastructure-agnostic nature of this approach

Quality Assurance in MLOps Setting: An Industrial Perspective

Today, machine learning (ML) is widely used in industry to provide the core functionality of production systems. However, it is practically always used in production systems as part of a larger end-to-end software system that is made up of several other components in addition to the ML model. Due to production demand and time constraints, automated software engineering practices are highly applicable. The increased use of automated ML software engineering practices in industries such as manufacturing and utilities requires an automated Quality Assurance (QA) approach as an integral part of ML software. Here, QA helps reduce risk by offering an objective perspective on the software task. Although conventional software engineering has automated tools for QA data analysis for data-driven ML, the use of QA practices for ML in operation (MLOps) is lacking. This paper examines the QA challenges that arise in industrial MLOps and conceptualizes modular strategies to deal with data integrity and Data Quality (DQ). The paper is accompanied by real industrial use-cases from industrial partners. The paper also presents several challenges that may serve as a basis for future studies.Comment: Accepted in ISE2022 of the 29th Asia-Pacific Software Engineering Conference (APSEC 2022