On Observability and Monitoring of Distributed Systems: An Industry Interview Study

Business success of companies heavily depends on the availability and performance of their client applications. Due to modern development paradigms such as DevOps and microservice architectural styles, applications are decoupled into services with complex interactions and dependencies. Although these paradigms enable individual development cycles with reduced delivery times, they cause several challenges to manage the services in distributed systems. One major challenge is to observe and monitor such distributed systems. This paper provides a qualitative study to understand the challenges and good practices in the field of observability and monitoring of distributed systems. In 28 semi-structured interviews with software professionals we discovered increasing complexity and dynamics in that field. Especially observability becomes an essential prerequisite to ensure stable services and further development of client applications. However, the participants mentioned a discrepancy in the awareness regarding the importance of the topic, both from the management as well as from the developer perspective. Besides technical challenges, we identified a strong need for an organizational concept including strategy, roles and responsibilities. Our results support practitioners in developing and implementing systematic observability and monitoring for distributed systems

Distributed Environment for Efficient Virtual Machine Image Management in Federated Cloud Architectures

The use of Virtual Machines (VM) in Cloud computing provides various benefits in the overall software engineering lifecycle. These include efficient elasticity mechanisms resulting in higher resource utilization and lower operational costs. VM as software artifacts are created using provider-specific templates, called VM images (VMI), and are stored in proprietary or public repositories for further use. However, some technology specific choices can limit the interoperability among various Cloud providers and bundle the VMIs with nonessential or redundant software packages, leading to increased storage size, prolonged VMI delivery, stagnant VMI instantiation and ultimately vendor lock-in. To address these challenges, we present a set of novel functionalities and design approaches for efficient operation of distributed VMI repositories, specifically tailored for enabling: (i) simplified creation of lightweight and size optimized VMIs tuned for specific application requirements; (ii) multi-objective VMI repository optimization; and (iii) efficient reasoning mechanism to help optimizing complex VMI operations. The evaluation results confirm that the presented approaches can enable VMI size reduction by up to 55%, while trimming the image creation time by 66%. Furthermore, the repository optimization algorithms, can reduce the VMI delivery time by up to 51% and cut down the storage expenses by 3%. Moreover, by implementing replication strategies, the optimization algorithms can increase the system reliability by 74%

Service based virtual RAN architecture for next generation cellular systems

Service based architecture (SBA) is a paradigm shift from Service-Oriented Architecture (SOA) to microservices, combining their principles. Network virtualization enables the application of SBA in cellular systems. To better guide the software design of this virtualized cellular system with SBA, this paper presents a software perspective and a positional approach to using fundamental development principles for adapting SBA in virtualized Radio Access Networks (vRANs). First, we present the motivation for using an SBA in cellular radio systems. Then, we explore the critical requirements, key principles, and components for the software to provide radio services in SBA. We also explore the potential of applying SBA-based Radio Access Network (RAN) by comparing the functional split requirements of 5G RAN with existing open-source software and accelerated hardware implementations of service bus, and discuss the limitations of SBA. Finally, we present some discussions, future directions, and a roadmap of applying such a high-level design perspective of SBA to next-generation RAN infrastructure.This work was supported in part by the European Union (EU) H2020 5GROWTH Project under Grant 856709, in part by the Generalitat de Catalunya under Grant 2017 SGR 1195, and in part by the National Program on Equipment and Scientific and Technical Infrastructure under the European Regional Development Fund (FEDER) under Grant EQC2018-005257-P

Alarm reduction and root cause inference based on association mining in communication network

With the growing demand for data computation and communication, the size and complexity of communication networks have grown significantly. However, due to hardware and software problems, in a large-scale communication network (e.g., telecommunication network), the daily alarm events are massive, e.g., millions of alarms occur in a serious failure, which contains crucial information such as the time, content, and device of exceptions. With the expansion of the communication network, the number of components and their interactions become more complex, leading to numerous alarm events and complex alarm propagation. Moreover, these alarm events are redundant and consume much effort to resolve. To reduce alarms and pinpoint root causes from them, we propose a data-driven and unsupervised alarm analysis framework, which can effectively compress massive alarm events and improve the efficiency of root cause localization. In our framework, an offline learning procedure obtains results of association reduction based on a period of historical alarms. Then, an online analysis procedure matches and compresses real-time alarms and generates root cause groups. The evaluation is based on real communication network alarms from telecom operators, and the results show that our method can associate and reduce communication network alarms with an accuracy of more than 91%, reducing more than 62% of redundant alarms. In addition, we validate it on fault data coming from a microservices system, and it achieves an accuracy of 95% in root cause location. Compared with existing methods, the proposed method is more suitable for operation and maintenance analysis in communication networks

Smart Mobility for All: A Global Federated Market for Mobility-as-a-Service Operators

International audienceMulti-modal travelling is a common phenomenon. However, planning multi-modal journeys is still an unstructured and time-consuming experience for customers: they lose time assembling a comprehensive plan out of disparate data, spread over a multitude of information systems — each corresponding to a different company responsible for one of the legs in the journey. Also transport operators are affected by the sparsity of the transportation market, as they might lose potential customers who could not find or know about their services. In this paper, we propose Mobility as a Service (MaaS) as a solution to such problems. Key element of MaaS is that MaaS operators can aggregate solutions of multiple providers to deliver dynamic, transparent multi-modal travels to their users, who experience transportation as managed directly by a single operator. However, given the volume and sparsity of the transportation market, we argue that MaaS operators cannot rely on one-to-one, custom contracts of usage with single mobility operators. Instead, we envision the creation of platforms that automatise the marketing of services for mobility among many mobility providers. In this work, we detail the required features of a general software platform for such a MaaS market. In particular, we provide a precise definition of MaaS through the MaaS Stack — a tiered view of the components needed by entities to join the MaaS market. Then, through the lens of the MaaS Stack, we elicit the features of an enabling software platform. Finally, to validate our approach, we present a compliant prototype, called SMAll, and discuss its main design choices, among which: i) how SMAll supports the creation of a federation-based MaaS market and ii) how microservices — an emerging architectural style that fosters cohesiveness and minimality of components — enhance flexibility and let the platform and the services of its members efficiently scale according to dynamic demands

Building a Neighborhood Resource Map for IoT and Cyber-Physical systems in Resource-Constrained Environments

Creating and maintaining a shared resource map between observation nodes that have a behavior where they are mostly sleeping, and have a wake up schedule that are determined at each node locally is challenging. This thesis looks at these challenges, and possible solutions have been proposed to overcome them. Previous research on the topic of constrained IoT networks have looked at the network, energy, and human constraints separately. But no one has looked at what is needed when all the limitations have to be accounted for simultaneously. Three methods for exchanging resource descriptions are created in this paper. To evaluate the different exchange methods with different node behaviors, a custom simulator is made. The simulator will simulate communication and resource description exchanges between nodes. The results show that different node behavior has a drastic affect on when each of the exchange methods work best. The main contribution of this paper is to guide designers of IoT and sensor networks, when they are choosing how the nodes will behave in resource constrained environments. And the main conclusion are that when there are complete overlap of node behavior, the best method to spread resource descriptions is; to have everyone just sharing description for its own resource. When the nodes are not guaranteed to overlap, some other techniques for exchanging information must be used, like SLM or SLMV that are presented in this paper. Also when there is no overlap, the node behavior becomes even more important. Structuring the wakeup schedules even a bit can help improve overall time to create the resource map

ICSEA 2021: the sixteenth international conference on software engineering advances

The Sixteenth International Conference on Software Engineering Advances (ICSEA 2021), held on October 3 - 7, 2021 in Barcelona, Spain, continued a series of events covering a broad spectrum of software-related topics. The conference covered fundamentals on designing, implementing, testing, validating and maintaining various kinds of software. The tracks treated the topics from theory to practice, in terms of methodologies, design, implementation, testing, use cases, tools, and lessons learnt. The conference topics covered classical and advanced methodologies, open source, agile software, as well as software deployment and software economics and education. The conference had the following tracks: Advances in fundamentals for software development Advanced mechanisms for software development Advanced design tools for developing software Software engineering for service computing (SOA and Cloud) Advanced facilities for accessing software Software performance Software security, privacy, safeness Advances in software testing Specialized software advanced applications Web Accessibility Open source software Agile and Lean approaches in software engineering Software deployment and maintenance Software engineering techniques, metrics, and formalisms Software economics, adoption, and education Business technology Improving productivity in research on software engineering Trends and achievements Similar to the previous edition, this event continued to be very competitive in its selection process and very well perceived by the international software engineering community. As such, it is attracting excellent contributions and active participation from all over the world. We were very pleased to receive a large amount of top quality contributions. We take here the opportunity to warmly thank all the members of the ICSEA 2021 technical program committee as well as the numerous reviewers. The creation of such a broad and high quality conference program would not have been possible without their involvement. We also kindly thank all the authors that dedicated much of their time and efforts to contribute to the ICSEA 2021. We truly believe that thanks to all these efforts, the final conference program consists of top quality contributions. This event could also not have been a reality without the support of many individuals, organizations and sponsors. We also gratefully thank the members of the ICSEA 2021 organizing committee for their help in handling the logistics and for their work that is making this professional meeting a success. We hope the ICSEA 2021 was a successful international forum for the exchange of ideas and results between academia and industry and to promote further progress in software engineering research