On Observability and Monitoring of Distributed Systems: An Industry Interview Study

Business success of companies heavily depends on the availability and performance of their client applications. Due to modern development paradigms such as DevOps and microservice architectural styles, applications are decoupled into services with complex interactions and dependencies. Although these paradigms enable individual development cycles with reduced delivery times, they cause several challenges to manage the services in distributed systems. One major challenge is to observe and monitor such distributed systems. This paper provides a qualitative study to understand the challenges and good practices in the field of observability and monitoring of distributed systems. In 28 semi-structured interviews with software professionals we discovered increasing complexity and dynamics in that field. Especially observability becomes an essential prerequisite to ensure stable services and further development of client applications. However, the participants mentioned a discrepancy in the awareness regarding the importance of the topic, both from the management as well as from the developer perspective. Besides technical challenges, we identified a strong need for an organizational concept including strategy, roles and responsibilities. Our results support practitioners in developing and implementing systematic observability and monitoring for distributed systems

Data Management in Microservices: State of the Practice, Challenges, and Research Directions

We are recently witnessing an increased adoption of microservice architectures by the industry for achieving scalability by functional decomposition, fault-tolerance by deployment of small and independent services, and polyglot persistence by the adoption of different database technologies specific to the needs of each service. Despite the accelerating industrial adoption and the extensive research on microservices, there is a lack of thorough investigation on the state of the practice and the major challenges faced by practitioners with regard to data management. To bridge this gap, this paper presents a detailed investigation of data management in microservices. Our exploratory study is based on the following methodology: we conducted a systematic literature review of articles reporting the adoption of microservices in industry, where more than 300 articles were filtered down to 11 representative studies; we analyzed a set of 9 popular open-source microservice-based applications, selected out of more than 20 open-source projects; furthermore, to strengthen our evidence, we conducted an online survey that we then used to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 practitioners and researchers. Through this process, we were able to categorize the state of practice and reveal several principled challenges that cannot be solved by software engineering practices, but rather need system-level support to alleviate the burden of practitioners. Based on the observations we also identified a series of research directions to achieve this goal. Fundamentally, novel database systems and data management tools that support isolation for microservices, which include fault isolation, performance isolation, data ownership, and independent schema evolution across microservices must be built to address the needs of this growing architectural style

Industrial DevOps

The visions and ideas of Industry 4.0 require a profound interconnection of machines, plants, and IT systems in industrial production environments. This significantly increases the importance of software, which is coincidentally one of the main obstacles to the introduction of Industry 4.0. Lack of experience and knowledge, high investment and maintenance costs, as well as uncertainty about future developments cause many small and medium-sized enterprises hesitating to adopt Industry 4.0 solutions. We propose Industrial DevOps as an approach to introduce methods and culture of DevOps into industrial production environments. The fundamental concept of this approach is a continuous process of operation, observation, and development of the entire production environment. This way, all stakeholders, systems, and data can thus be integrated via incremental steps and adjustments can be made quickly. Furthermore, we present the Titan software platform accompanied by a role model for integrating production environments with Industrial DevOps. In two initial industrial application scenarios, we address the challenges of energy management and predictive maintenance with the methods, organizational structures, and tools of Industrial DevOps.Comment: 10 page

Mikropalveluiden haasteet

Ohjelmistojen määrän kasvaessa ohjelmistoarkkitehtuurin merkitys kasvaa. Perinteisesti ohjelmistot ovat olleet monoliittisia, jolloin kaikki ohjelman suoritettava koodi on yhdessä suoritettavassa ohjelmassa. Ohjelmistojen koon ja määrän kasvaessa monoliittisesta ohjelmistoarkkitehtuurista on siirrytty kohti jaettua ohjelmistoarkkitehtuuria, jossa ohjelman suoritettava koodi on jaettu useampaan eri osaan. Mikropalveluarkkitehtuuri on yksi jaetun arkkitehtuurin tyyleistä joka on saavuttanut suosiota viime vuosina. Se perustuu pieniin, itsenäisiin palveluihin. Tässä tutkielmassa tutustutaan ohjelmistoarkkitehtuuriin ja erityisesti mikropalveluarkkitehtuuriin. Tutkielmassa tehtiin systemaattinen kirjallisuuskatsaus mikropalveluarkkitehtuurin haasteisiin. Kirjallisuuskatsauksella etsittiin tietoa mikropalveluiden kehittäjien kokemista haasteista sekä ratkaisuista niihin haasteisiin. Tuloksia vertailtiin aiempiin aiheesta tehtyihin tutkimuksiin. Kirjallisuuskatsauksen aineistoon valittiin yhteensä 23 lähdettä, jotka olivat vuosilta 2017-2022. Aineistojen tutkimusmetodeina oli pääasiassa haastattelut tai kyselyt. Havaittiin, että haasteet ovat edelleen pääasiassa teknisiä ja ne olivat osittain samoja kuin aiemmissa tutkimuksissa. Haasteiden keskinäisessä tärkeysjärjestyksessä oli muutoksia: esimerkiksi mikropalveluiden suunnitteluun liittyviä haasteita ei koettu aiemmassa tutkimuksessa yhtä tärkeiksi kuin nyt. Teknisien haasteiden lisäksi esille nousi kuitenkin selkeästi erilaiset organisaatioon ja ihmisiin liittyvät haasteet, joita aiempi tutkimus ei ollut löytänyt yhtä hyvin

Framework For The Successful Set-up Of A Common Data Model In The Context Of An Industry 4.0-ready Plant Design Process

The production plant design process consists of a multitude of individual engineering disciplines, which rely on a variety of digital models. The individual tasks build up on each other, while each discipline consumes information from the previous processes. However, sharing relevant data across multiple companies is challenging and susceptible to miscommunication and delays. Furthermore, integrating diverse software systems, tools, and technologies create compatibility issues and hinder seamless integration. As a result, a heterogeneous, non-automated data and information landscape is created, characterized by a high level of manual data transfer. This represents a major problem on the way towards Industry 4.0. The goal of this paper is to provide a framework for the successful set-up of a common data model in the context of an Industry 4.0-ready plant design process across and along the value chain. For this purpose, a literature review of current problems in the cross-company and cross-departmental collaboration in the plant design process is provided and requirements for the framework are derived. Existing solutions and research projects are compiled and evaluated against the requirements, from which the framework's structure is concluded. The framework itself is intended to be holistic and must therefore not only include technical aspects (e.g. data interfaces, semantics), but also enable the entire organization and value chain to implement the common data model as part of the digital transformation process (e.g. employee skills, business strategy, legal conditions). Based on this, the framework is further elaborated by deducing calls for action for a successful set-up of a common data model within the research project DIAMOND (Digital plant modeling with neutral data formats). The focus should be on employees and their competencies, as these are prerequisites for shaping digital transformation. Future research must prioritize these actions to enhance technology readiness and organizational Industry 4.0 preparation

Smart and Pervasive Healthcare

Smart and pervasive healthcare aims at facilitating better healthcare access, provision, and delivery by overcoming spatial and temporal barriers. It represents a shift toward understanding what patients and clinicians really need when placed within a specific context, where traditional face-to-face encounters may not be possible or sufficient. As such, technological innovation is a necessary facilitating conduit. This book is a collection of chapters written by prominent researchers and academics worldwide that provide insights into the design and adoption of new platforms in smart and pervasive healthcare. With the COVID-19 pandemic necessitating changes to the traditional model of healthcare access and its delivery around the world, this book is a timely contribution

Agile Processes in Software Engineering and Extreme Programming – Workshops

This open access book constitutes papers from the 5 research workshops, the poster presentations, as well as two panel discussions which were presented at XP 2021, the 22nd International Conference on Agile Software Development, which was held online during June 14-18, 2021. XP is the premier agile software development conference combining research and practice. It is a unique forum where agile researchers, practitioners, thought leaders, coaches, and trainers get together to present and discuss their most recent innovations, research results, experiences, concerns, challenges, and trends. XP conferences provide an informal environment to learn and trigger discussions and welcome both people new to agile and seasoned agile practitioners. The 18 papers included in this volume were carefully reviewed and selected from overall 37 submissions. They stem from the following workshops: 3rd International Workshop on Agile Transformation 9th International Workshop on Large-Scale Agile Development 1st International Workshop on Agile Sustainability 4th International Workshop on Software-Intensive Business 2nd International Workshop on Agility with Microservices Programmin

Strategies for IT Product Managers to Manage Microservice Systems in Enterprises

Many information technology (IT) product managers have experienced significant challenges in adopting microservice architecture (MSA) systems successfully in their organizations. Inefficiencies resulting from MSA system adoption are of concern to IT product managers as these inefficiencies increase the cost of maintenance and increase the time to deliver software updates to the business. Grounded in the technology-organization-environment theory, the purpose of this qualitative multiple case study was to explore the strategies used by IT product managers for the domain-driven design and development and operations practices to reduce the inefficiencies during the MSA system adoption. The participants were 18 IT product managers associated with the adoption and operation of microservice architecture systems at a global consumer goods manufacturer company and a global financial company, both with headquarters in Europe. Data were collected from semi structured interviews and a review of 7 documents. A thematic analysis was used to analyze the data. Four major themes emerged to include organizational alignment in adopting MSA, ways of working, experienced-based approach to design MSA systems, and MSA environment landscape. A key recommendation for IT product managers is to adopt an IT organization structure aligned with the business context of the MSA system allowing for a full lifecycle approach. The implications for positive social change include the potential for IT product managers to improve the work environment for the MSA-related teams, which may lead to robust software systems and easier to use applications by removing barriers and increasing accessibility for users, thus supporting individuals in their daily life