Factors that Contribute to the Success of a Software Organisation's DevOps Environment: A Systematic Review

This research assesses the aspects of software organizations' DevOps environments and identifies the factors contributing to these environments' success. DevOps is a recent concept, and many organizations are moving from old-style software development methods to agile approaches such as DevOps. However, there is no comprehensive information on what factors impact the success of the DevOps environment once organizations adopt it. This research focused on addressing this gap through a systematic literature review. The systematic review consisted of 33 articles from five selected search systems and databases from 2015 to 2021. Based on the included articles, 15 factors were identified and grouped into four categories: Collaborative Culture, Organizational Aspects, Tooling and Technology, and Continuous Practices. In addition, this research proposes a DevOps environment success factors model to potentially contribute to DevOps research and practice. Recommendations are made for additional research on the effectiveness of the proposed model and its success factors.Comment: 15 pages, 3 figures, 1 tabl

Requirements Engineering that Balances Agility of Teams and System-level Information Needs at Scale

Context: Motivated by their success in software development, large-scale systems development companies are increasingly adopting agile methods and their practices. Such companies need to accommodate different development cycles of hardware and software and are usually subject to regulation and safety concerns. Also, for such companies, requirements engineering is an essential activity that involves upfront and detailed analysis which can be at odds with agile development methods. Objective: The overall aim of this thesis is to investigate the challenges and solution candidates of performing effective requirements engineering in an agile environment, based on empirical evidence. Illustrated with studies on safety and system-level information needs, we explore RE challenges and solutions in large-scale agile development, both in general and from the teams’ perspectives. Method: To meet our aim, we performed a secondary study and a series of empirical studies based on case studies. We collected qualitative data using interviews, focus groups and workshops to derive challenges and potential solutions from industry. Findings: Our findings show that there are numerous challenges of conducting requirements engineering in agile development especially where systems development is concerned. The challenges discovered sprout from an integration problem of working with agile methods while relying on established plan-driven processes for the overall system. We highlight the communication challenge of crossing the boundary of agile methods and system-level (or plan-driven) development, which also proves the coexistence of both methods. Conclusions: Our results highlight the painful areas of requirements engineering in agile development and propose solutions that can be explored further. This thesis contributes to future research, by establishing a holistic map of challenges and candidate solutions that can be further developed to make RE more efficient within agile environments

Requirements Engineering Challenges of Supporting Agile Teams in System Development

Context: Agile methods have attracted many companies due to their reported benefits of short time-to-market and improved quality outputs. In the systems development context, additional constraints apply e.g. as a result of scale or parallel development of hardware and software. Traditionally, stage-gate processes with a focus on up-front requirements analysis are common in large- scale systems engineering. These processes clash with the companies’ desire to become more agile.Objective: The aim of this thesis is to discover challenges that new requirements engineering approaches should address to enable agile system devel- opment at scale (RE4Agile). With a focus on value and building system understanding, we explore these challenges from the perspective of the agile development teams.Method: To meet our aim, we conducted a series of empirical studies based on case studies, and a secondary review to explore the problem domain while deriving challenges and potential solutions from industry and literature respectively.Findings: Our findings show that there are numerous challenges of conducting requirements engineering in agile development especially where systems development is concerned. These challenges relate to user value and overall system understanding. However, there are some cross-cutting concerns, e.g safety- critical development, that have generated much interest both from practitioners and academicians at large.Conclusions: The challenges discovered sprout from an integration problem of working with agile methods while using the already existing processes as well. However, solution candidates exist and our future research aims to validate some of the solution candidates in the view of deriving new RE approaches. This thesis contributes to such future research, by establishing a holistic map of challenges that allows to assess whether a given solution is beneficial in the larger context or whether it over-optimizes only one area

A mapping study on documentation in Continuous Software Development

Context: With an increase in Agile, Lean, and DevOps software methodologies over the last years (collectively referred to as Continuous Software Development (CSD)), we have observed that documentation is often poor. Objective: This work aims at collecting studies on documentation challenges, documentation practices, and tools that can support documentation in CSD. Method: A systematic mapping study was conducted to identify and analyze research on documentation in CSD, covering publications between 2001 and 2019. Results: A total of 63 studies were selected. We found 40 studies related to documentation practices and challenges, and 23 studies related to tools used in CSD. The challenges include: informal documentation is hard to understand, documentation is considered as waste, productivity is measured by working software only, documentation is out-of-sync with the software and there is a short-term focus. The practices include: non-written and informal communication, the usage of development artifacts for documentation, and the use of architecture frameworks. We also made an inventory of numerous tools that can be used for documentation purposes in CSD. Overall, we recommend the usage of executable documentation, modern tools and technologies to retrieve information and transform it into documentation, and the practice of minimal documentation upfront combined with detailed design for knowledge transfer afterwards. Conclusion: It is of paramount importance to increase the quantity and quality of documentation in CSD. While this remains challenging, practitioners will benefit from applying the identified practices and tools in order to mitigate the stated challenges

Architectural Support for Software Performance in Continuous Software Engineering: A Systematic Mapping Study

The continuous software engineering paradigm is gaining popularity in modern development practices, where the interleaving of design and runtime activities is induced by the continuous evolution of software systems. In this context, performance assessment is not easy, but recent studies have shown that architectural models evolving with the software can support this goal. In this paper, we present a mapping study aimed at classifying existing scientific contributions that deal with the architectural support for performance-targeted continuous software engineering. We have applied the systematic mapping methodology to an initial set of 215 potentially relevant papers and selected 66 primary studies that we have analyzed to characterize and classify the current state of research. This classification helps to focus on the main aspects that are being considered in this domain and, mostly, on the emerging findings and implications for future researc

Open source tools for Linux distribution development and maintenance in corporate environment

In this thesis, we will look into utilizing open source software build tools in an enterprise environment. We will aim at providing a complete set of tools starting from developer support and leading to software delivery. We will discuss the different tools that we will use to offer more reliable, easy to use and efficient process of composing software products. Many open source projects will be utilized and we will examine the required steps to be able to successfully operate them in a closed environment. We will also look into providing a completely new base image for in-house cloud platform building. The process of internally composing the operating system from open source components will be discussed in depth

Developer’s Perspective on Containerized Development Environments : A Case Study and Review of Gray Literature

Context: The advent of Docker containers in 2013 provided developers with a way of bundling code and its dependencies into containers that run identically on any Docker Engine, effectively mitigating platform and dependency related issues. In recent years an interesting trend has emerged of developers attempting to leverage the benefits provided by the Docker container platform in their development environments. Objective: In this thesis we chart the motivations behind the move towards Containerized Development Environments (CDE) and seek to categorize claims made about benefits and challenges experienced by developers after their adoption. The goal of this thesis is to establish the current state of the trend and lay the groundwork for future research. Methods: The study is structured into three parts. In the first part we conduct a systematic review of gray literature, using 27 sources acquired from three different websites. The sources were extracted for relevant quotes that were used for creating a set of higher level concepts for expressed motivations, benefits, and challenges. The second part of the study is a qualitative single-case study where we conduct semi-structured theme interviews with all members of a small-sized software development team that had recently taken a containerized development environment into use. The case team was purposefully selected for its practical relevance as well as convenient access to its members for data collection. In the last part of the study we compare the transcribed interview data against the set of concepts formed in the literature review. Results: Cross-environment consistency and a simplified initial setup driven by a desire to increase developer happiness and productivity were commonly expressed motivations that were also experienced in practice. Decreased performance, required knowledge of Docker, and difficulties in the technical implementation of CDE’s were mentioned as primary challenges. Many developers experienced additional benefits of using the Docker platform for infrastructure provisioning and shared configuration management. The case team additionally used the CDE as a platform for implementing end to end testing, and viewed the correct type of team and management as necessary preconditions for its successful adoption. Conclusions: CDE’s offer many valuable benefits that come at a cost and teams have to weigh the trade-off between consistency and performance, and whether the investment of development resources to its implementation is warranted. The use of the Docker container platform as an infrastructure package manager could be considered a game-changer, enabling development teams to provision new services like databases, load-balancers and message brokers with just a few lines of code. The case study reports one account of an improved onboarding experience and points towards an area for future research. CDE’s would appear to be a good fit for microservice oriented teams that seek to foster a DevOps culture, as indicated by the experience of the case team. The implementation of CDE’s is a non-trivial challenge that requires expertise from the teams and developers using them. Additionally, the case team’s novel use of containers for testing appears to be an interesting research topic in its own right. ACM Computing Classification System (CCS): Software and its engineering →Software creation and management →Software development technique

Adoption of microservices in industrial information systems: a systematic literature review

The internet, digitalization and globalization have transformed customer expectations and the way business is done. Product life cycles have shortened, products need to be customizable, and the production needs to be scalable. These changes reflect also to the industrial operations. Quick technological advancements have increased the role of software in industrial facilities. The software in use has to enable untraditional flexibility, interoperability and scalability. Microservices based architecture has been seen as the state of the art way for developing flexible, interoperable and scalable software. Microservices have been applied to cloud native applications for consumers with enormous success. The goal of this thesis is to analyze how to adopt microservices to indstrial information systems. General information and characteristics of microservices are provided as background information and a systematic literature review is conducted to answer the research problem. Material for the systematic literature review was found from multiple digital libraries and 17 scientific papers matched the set inclusion cirteria. The material was then analyzed with an extensively documentated method. The thesis brought together the available publications on the topic. Guidelines for adopting microservices to industrial information systems were derived based on the analysis. Real time applications need special attention when using microservices architecture, the developers need to use proper tools for the tasks, and the developers and users need to be properly introduced to service-oriented systems. Based on this thesis microservices seems like a suitable approach for developing flexible industrial information systems, which satisfy the new business requirements

Orchestrator selection process for cloud-native machine learning experimentation

Dissertação de mestrado integrado em Informatics EngineeringMachine learning (ML) model development is a very experimental, repetitive, and error prone task, because ML is itself very obscure - there is no way to know what model works best for our goals beforehand, so practitioners have an incentive to experiment with as many models, approaches and techniques as they can. Additionally, going from raw data to a well adjusted model is a delicate process that often requires complex, multi-step pipelines. Combine the two factors and it becomes apparent how easy it is to get lost within a sea of artifacts and results without a well defined process, hindering the development process with poor reusability, lots of technical debt, and integration-hell. This makes adherence to best practices - MLOps - paramount. However, with the recent boom experienced in this field came a plethora of different tools and services, all trying to satisfy different subsets of needs of the model life cycle, meaning that, more often than not, ML practitioners do not know what the best set of tools for their use case might be. The experimental nature of ML means we should indeed try different tools, but there is a high risk that it might not fit the necessary requirements, generating needless costs. One particularly relevant type of tool is the orchestrator - a central piece of the experimentation process which controls the communication and execution of the components of a model pipeline. This work follows the creation process for an enterprise ML cloud environment, with particular focus on the selection of an adequate orchestrator for cloud-native setups. Additionally, it presents MetaTool, a web application designed to speed up future tool selection processes by leveraging knowledge gathered during previous instances. Finally, it reaches two key conclusions: first, broader organizational factors that might seem out of scope can influence or even alter the final choice, and second, although using a tool like MetaTool might speed up the decision-making process, it requires significant organizational commitment.O desenvolvimento de modelos de machine learning (ML) é uma atividade muito experimental, repetitiva e propícia a erros, porque ML é muito obscura - não há forma de saber de antemão qual o modelo mais adequado para os nossos objetivos, pelo que os praticantes têm um incentivo para experimentar com o maior número possível de modelos, abordagens e técnicas que conseguirem. Adicionalmente, passar de dados para um modelo bem ajustado é um processo delicado que frequentemente requer pipelines complexas e com vários passos. Combinando os dois fatores fica aparente o quão fácil é ficar perdido num mar de artefactos e resultados sem um processo bem definido, dificultando o processo de desenvolvimento com fraca capacidade de reutilização, muita technical debt, e integration hell. Isto torna a adesão às melhores práticas - MLOps - imperativa. Contudo, com o recente avanço verificado neste domínio veio uma abundância de diferentes ferramentas e serviços, todos tentando satisfazer diferentes subconjuntos de necessidades do ciclo de vida dos modelos, pelo que os praticantes de ML acabam frequentemente na dúvida de qual poderá ser o melhor conjunto de ferramentas para os seus casos de uso. A natureza experimental de ML faz com que se devam experimentar diferentes ferramentas, mas há um grande risco de escolher algo não satisfaça os requisitos necessários, levando a custos desnecessários. Uma categoria de ferramentas particularmente relevantes são os orquestradores - uma peça central no processo de experimentação que controla a comunicação e execução dos componentes da pipeline do modelo. Este trabalho acompanha a criação dum ambiente cloud industrial para ML, com particular foco na escolha do orquestrador adequado para ambientes na nuvem. Adicionalmente, apresenta MetaTool, uma aplicação web pensada para acelerar futuros processos de tomada de decisão empregando conhecimento adquirido durante processos anteriores. Finalmente, alcança duas conclusões chave: primeiro, fatores organizacionais aparentemente irrelevantes podem influenciar ou até alterar a escolha final, e segundo, apesar de ferramentas como MetaTool poderem acelerar o processo de tomada de decisão, requerem um empenho da organização