On Observability and Monitoring of Distributed Systems: An Industry Interview Study

Business success of companies heavily depends on the availability and performance of their client applications. Due to modern development paradigms such as DevOps and microservice architectural styles, applications are decoupled into services with complex interactions and dependencies. Although these paradigms enable individual development cycles with reduced delivery times, they cause several challenges to manage the services in distributed systems. One major challenge is to observe and monitor such distributed systems. This paper provides a qualitative study to understand the challenges and good practices in the field of observability and monitoring of distributed systems. In 28 semi-structured interviews with software professionals we discovered increasing complexity and dynamics in that field. Especially observability becomes an essential prerequisite to ensure stable services and further development of client applications. However, the participants mentioned a discrepancy in the awareness regarding the importance of the topic, both from the management as well as from the developer perspective. Besides technical challenges, we identified a strong need for an organizational concept including strategy, roles and responsibilities. Our results support practitioners in developing and implementing systematic observability and monitoring for distributed systems

INSTRUMENTATION-BASED MONITORING TECHNIQUES SURVEY ON HOST, PLATFORM, AND SERVICE LEVEL IN MICROSERVICE ARCHITECTURE

Microservice is an application architecture that separates one big application into smaller ones. The architecture simplifies development, deployment, and management process. However, the architecture is quite complex thus the monitoring process becomes much more challenging. Classifications for the instrumentations that are used in the monitoring process is needed to achieve better practicality for the administrators. We surveyed the monitoring technique classification method in microservice architecture. The method is divided into three levels. They are host level, platform level, and service level. In this paper, we present the latest instruments that are being used in the monitoring process in each level. Correlation between the goals, needs, and stakeholder is also presented

Ambidextrous IT Governance: The Art of Balancing Exploration and Exploitation in IT Governance

Through a case study at a global technology company, we investigate how organizations can adapt their IT governance approach to the information system at hand. This is done by considering the degree of information system integration and whether the system is related to supporting operational efficiency (exploitation) or innovation (exploration). Based on the findings of the case study, we introduce the concept of ambidextrous IT governance to describe how IT governance can be adapted to fit the dual needs of both exploration and exploitation through the use of IS

ICSEA 2021: the sixteenth international conference on software engineering advances

The Sixteenth International Conference on Software Engineering Advances (ICSEA 2021), held on October 3 - 7, 2021 in Barcelona, Spain, continued a series of events covering a broad spectrum of software-related topics. The conference covered fundamentals on designing, implementing, testing, validating and maintaining various kinds of software. The tracks treated the topics from theory to practice, in terms of methodologies, design, implementation, testing, use cases, tools, and lessons learnt. The conference topics covered classical and advanced methodologies, open source, agile software, as well as software deployment and software economics and education. The conference had the following tracks: Advances in fundamentals for software development Advanced mechanisms for software development Advanced design tools for developing software Software engineering for service computing (SOA and Cloud) Advanced facilities for accessing software Software performance Software security, privacy, safeness Advances in software testing Specialized software advanced applications Web Accessibility Open source software Agile and Lean approaches in software engineering Software deployment and maintenance Software engineering techniques, metrics, and formalisms Software economics, adoption, and education Business technology Improving productivity in research on software engineering Trends and achievements Similar to the previous edition, this event continued to be very competitive in its selection process and very well perceived by the international software engineering community. As such, it is attracting excellent contributions and active participation from all over the world. We were very pleased to receive a large amount of top quality contributions. We take here the opportunity to warmly thank all the members of the ICSEA 2021 technical program committee as well as the numerous reviewers. The creation of such a broad and high quality conference program would not have been possible without their involvement. We also kindly thank all the authors that dedicated much of their time and efforts to contribute to the ICSEA 2021. We truly believe that thanks to all these efforts, the final conference program consists of top quality contributions. This event could also not have been a reality without the support of many individuals, organizations and sponsors. We also gratefully thank the members of the ICSEA 2021 organizing committee for their help in handling the logistics and for their work that is making this professional meeting a success. We hope the ICSEA 2021 was a successful international forum for the exchange of ideas and results between academia and industry and to promote further progress in software engineering research

Analysis of requirements and technologies to migrate software development to the PaaS model

Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies ManagementSoftware development has been evolving during the last years and, more and more, the software architecture to support this development has become more complex to meet the new requirements and new technologies. With the new cloud computing architecture and models, IT departments and ISV are developing new applications and moving the traditional software architecture to the cloud. In this context, Platform as a Service (PaaS) model can provide software development services and components within a new architecture for building a new generation of software with all benefits of cloud, like scalability and elasticity. However, currently, most companies have significant challenges to adapt and change its software development process to use the PaaS architecture and the cloud services. In this dissertation, it will first be identified and analyzed the changes and challenges for develop software with the PaaS architecture. Afterwards, will be analyzed and identified the requirements in a traditional software development and architecture (on premise) to development new software or adapt the existents software with the PaaS.Dissertation submitted as partial requirement for obtaining the Master’s degree in Information Managemen

Ohjelmistoprojektin riskien ja prosessien hallinta startup-yrityksessä: Tapaustutkimus

Software projects are notorious for their failure rates and software maintenance is a complex task that often becomes more time-consuming as the software ages. In modern software development, maintenance is often done in an iterative fashion with the help of continuous integration and deployment tools to help with quality assurance. This thesis is a postmortem case study of the design and development involved in a user interface rewrite project conducted for a healthtech SaaS-product. The focus of the study is on investigating how efficient the methods of working were, what pain points were identified and how well the risks were managed for the project. It aims to provide insight on how early-stage companies with limited resources can see through a sizable effort such as this efficiently. Focus is also given to whether a transition towards a microservice-architecture is a viable choice within this context. The key findings from the conducted case study are that even when following agile practices, a systematic approach to software engineering is essential for success. Projects should have a clear scope and clear responsibilities in order for their success to be measurable. Team composition and individual skills are the crucial elements in a development team, and tools and practices only strengthen the results of individuals. However, open communication, motivated individuals and visibility into progress are also essential.Ohjelmistoprojektien epäonnistuminen on tutkitusti yleistä ja ohjelmistojen ylläpito on kompleksinen tehtävä, jonka vaatimat resurssit usein kasvavat ohjelmiston vanhetessa. Modernissa ohjelmistokehityksessä ylläpito usein tehdään iteratiivisesti, hyödyntäen jatkuvaa integraatiota laadunvarmistuksen apuna. Tämä diplomityö on tapaustutkimus terveysteknologiaan keskittyneen SaaS-sovelluksen uudistukseen liittyneestä kehitys- ja suunnittelutyöstä. Tutkielma keskittyy tutkimaan projektin riskinhallintaa, kehitysmetodien ja -prosessien tehokkuutta sekä löytämään näistä kipupisteitä. Työn tavoitteena on löytää resursseiltaan rajallisille alkuvaiheen ohjelmistoyrityksille soveltuvia työtapoja sekä selvittää, kuinka tämänkaltainen laaja kehitystyö voidaan suorittaa onnistuneesti. Tutkimus myös pyrkii selvittämään onko mikropalveluarkkitehtuuriin siirtyminen kannattavaa tässä kontekstissa. Työn tuloksena havaittiin, että systemaattinen lähestyminen ohjelmistokehitykseen on olennaista onnistumisen kannalta myös ketteriä menetelmiä hyödynnettäessä. Projekteilla tulisi olla selkeä laajuus ja selkeät tavoitteet, jotta projektin onnistumista voidaan mitata objektiivisesti. Kehitystiimin dynamiikka ja yksilöiden taidot ovat tärkein osa kehitystiimiä, ja työkaluilla ja menetelmillä on vain toissijainen vaikutus yksilöiden suorituskykyyn. Toisaalta myös avoin kommunikaatio, motivoituneet yksilöt ja kehityksen läpinäkyvyys ovat olennaisessa asemassa

A Dynamic Fault Tolerance Model for Microservices Architecture

Microservices architecture is popular for its distributive system styles due to the independent character of each of the services in the architecture. Microservices are built to be single and each service has its running process and interconnecting with a lightweight mechanism that called application programming interface (API). The interaction through microservices needs to communicate internally. Microservices are a service that is likely to become unreachable to its consumers because, in any distributed setup, communication will fail on occasions due to the number of messages passing between services. Failures can occur when the networks are unreliable, and thus the connections can be latent which may lead to failure or slow response. This might be a problem for synchronous remote calls actively waiting for a response. If they do not use a proper timeout mechanism, they may end up waiting for an extended amount of time. Applications usually set a timeout for all remote calls to avoid hanging of the whole application due to network failure or component failure. However, this timeout needs to be set carefully to make the system or microservice application to work as required. This would prevent further problems because if a remote call is waiting too long for a reply, it can slow down the system in its entirety, and if a connection timeout is extremely fast, it may ignore a response that is sent after timeout. This thesis proposes a dynamic fault tolerance (DFTM) Model to improve the stability and resilience of the microservices architecture. The Model is designed using a two-states Circuit Breaker called Switch Circuit Breaker with Markov-Chain. In addition, a modified Circuit Breaker (three states – open, closed, and half-open) to Switch Circuit Breaker (two states – open and closed) is presented here. The Circuit Breaker uses timeout to detect fault but timeouts usage hinges on assumptions about the real-time behavior of the system and awaiting process can be deduced from the occurrence of a timeout that a failure has occurred. Therefore, DFTM model adopted Markov Chain based model to detect fault without a timeout. Then, it sends the fault directly to Switch Circuit Breaker that uses a 2-states to cover the faults. An important finding is that the DFTM model presents a solution to the problem of transient failures or faults in the interservice communication of microservices architecture. Also, it improves the performance and reliability of microservices architecture