DevSecOps metrics: Learning from academics and professionals

DevSecOps is an emerging paradigm that breaks the Security team silo into the DevOps team, adding security practices to the Software Development Lifecycle (SDL) from inception. Security practices, in SDL, are important to avoid data breaches, guarantee compliance with the law and for organizations, it is an obligation to protect customer data. This study aims to identify metrics teams can use to measure the effectiveness of DevSecOps implementation inside organizations. To that end, this study was conducted using a Design Science Research (DSR) as its research methodology, with the intent of producing an artefact containing the most relevant DevSecOps metrics. A total of nine DevSecOps metrics purposed by professionals and academics were identified and listed on the artefact produced by this study. Interviews were conducted with DevSecOps professionals as a method of evaluating if the identified metrics were useful. Through the interviews, it was possible to identify the metrics that are being used by professionals and which are the most useful. Interviewees purposed three additional metrics. This study identifies a total of twelve metrics that can be used to measure effectiveness in DevSecOps.Ao longo dos anos, várias são as abordagens que tem sido adotadas como processo de desenvolvimento de Software, tais como o modelo em Cascata e o desenvolvimento Ágil, mais recentemente o termo DevOps foi introduzido, refere-se a uma abordagem que junta elementos da equipa de desenvolvimento e operações na mesma equipa, de modo a que exista uma coloboração mais próxima e partilha de conhecimento entre estes elementos, com o intuito de se atingir entregas do Software em desenvolvimento com tempos menores, com mais frequência e qualidade. DevSecOps é uma abordagem ao processo de desenvolvimento de Software emergente que junta elementos da equipa de segurança à equipa de DevOps, trazendo práticas de segurança para o ciclo de desenvolvimento de Software. As práticas de segurança são cada vez mais importantes no ciclo de desenvolvimento de software pois visam a evitar violações de dados e verificar o cumprimento da lei. Mais, ganharam extrema importância para as organizações visto que as mesmas têm por obrigação a proteção de dados dos seus clientes. Este estudo pretende identificar métricas, que podem ser utilizadas pelas equipas de modo a medir a eficiência da implementação de DevSecOps nas suas organizações. Para identificar essas métricas, este estudo foi realizado usando como metodologia de investigação uma Ciência de Design, esta metodologia caracteriza-se por ser uma pesquisa orientada a resultados, tendo sido escolhida, com o objetivo de produzir um artefacto, contendo, as métricas para DevSecOps mais relevantes. Foi possível identificar 9 métricas para DevSecOps, sugeridas por profissionais e académicos da área estando estas listadas no artefacto produzido por este estudo. Mais, foram conduzidas entrevistas com os profissionais de DevSecOps com o intuito de avaliar a utilidade das métricas. Com a ajuda das entrevistas, foi possível identificar as métricas utilizadas pelos profissionais e determinar as mais úteis e relevantes. Os entrevistados sugeriram 3 métricas adicionais perfazendo assim 12 métricas incluídas neste documento

DevOps in an ISO 13485 Regulated Environment: A Multivocal Literature Review

Background: Medical device development projects must follow proper directives and regulations to be able to market and sell the end-product in their respective territories. The regulations describe requirements that seem to be opposite to efficient software development and short time-to-market. As agile approaches, like DevOps, are becoming more and more popular in software industry, a discrepancy between these modern methods and traditional regulated development has been reported. Although examples of successful adoption in this context exist, the research is sparse. Aims: The objective of this study is twofold: to review the current state of DevOps adoption in regulated medical device environment; and to propose a checklist based on that review for introducing DevOps in that context. Method: A multivocal literature review is performed and evidence is synthesized from sources published between 2015 to March of 2020 to capture the opinions of experts and community in this field. Results: Our findings reveal that adoption of DevOps in a regulated medical device environment such as ISO 13485 has its challenges, but potential benefits may outweigh those in areas such as regulatory, compliance, security, organizational and technical. Conclusion: DevOps for regulated medical device environments is a highly appealing approach as compared to traditional methods and could be particularly suited for regulated medical development. However, an organization must properly anchor a transition to DevOps in top-level management and be supportive in the initial phase utilizing professional coaching and space for iterative learning; as such an initiative is a complex organizational and technical task.Comment: ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '20), October 8--9, 2020, Bari, Ital

NoOps – A Multivocal literature review

Traditionally, an organization had to have in-house servers and hardware to build a web application. This evolved into Cloud computing where the possibility for cost reduction and scalable data storage became a reality. With the introduction of cloud computing came a concept known as NoOps, or No Operations. This paper aims to take a closer look into what NoOps is and the benefits and challenges of NoOps. The authors identified three RQs that could help to give more insight into NoOps. Further we discussed the findings and RQs and lay out the way forward for future studies into NoOps. We also looked at artificial intelligence (AI) and how AI seems to be heavily linked with a true NoOps environment. With the lack of scientific studies into NoOps, a Multivocal literature review was selected as the method used to investigate the concept and its implications. We try to show voices both for and against NoOps. Further, we try to look at a misconception of what NoOps really is, what true NoOps could be. Finally we look at what requirements there are for companies wanting to go NoOps, and discuss the possibility that many companies unknowingly are moving towards a NoOps environment.publishedVersio

Benefitting from the Grey Literature in Software Engineering Research

Researchers generally place the most trust in peer-reviewed, published information, such as journals and conference papers. By contrast, software engineering (SE) practitioners typically do not have the time, access or expertise to review and benefit from such publications. As a result, practitioners are more likely to turn to other sources of information that they trust, e.g., trade magazines, online blog-posts, survey results or technical reports, collectively referred to as Grey Literature (GL). Furthermore, practitioners also share their ideas and experiences as GL, which can serve as a valuable data source for research. While GL itself is not a new topic in SE, using, benefitting and synthesizing knowledge from the GL in SE is a contemporary topic in empirical SE research and we are seeing that researchers are increasingly benefitting from the knowledge available within GL. The goal of this chapter is to provide an overview to GL in SE, together with insights on how SE researchers can effectively use and benefit from the knowledge and evidence available in the vast amount of GL

The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice

Companies struggle to continuously develop and deploy AI models to complex production systems due to AI characteristics while assuring quality. To ease the development process, continuous pipelines for AI have become an active research area where consolidated and in-depth analysis regarding the terminology, triggers, tasks, and challenges is required. This paper includes a Multivocal Literature Review where we consolidated 151 relevant formal and informal sources. In addition, nine-semi structured interviews with participants from academia and industry verified and extended the obtained information. Based on these sources, this paper provides and compares terminologies for DevOps and CI/CD for AI, MLOps, (end-to-end) lifecycle management, and CD4ML. Furthermore, the paper provides an aggregated list of potential triggers for reiterating the pipeline, such as alert systems or schedules. In addition, this work uses a taxonomy creation strategy to present a consolidated pipeline comprising tasks regarding the continuous development of AI. This pipeline consists of four stages: Data Handling, Model Learning, Software Development and System Operations. Moreover, we map challenges regarding pipeline implementation, adaption, and usage for the continuous development of AI to these four stages.Comment: accepted in the Journal Systems and Softwar

Revisit security in the era of DevOps : An evidence-based inquiry into DevSecOps industry

By adopting agile and lean practices, DevOps aims to achieve rapid value delivery by speeding up development and deployment cycles, which however lead to more security concerns that cannot be fully addressed by an isolated security role only in the final stage of development. DevSecOps promotes security as a shared responsibility integrated into the DevOps process that seamlessly intertwines development, operations, and security from the start throughout to the end of cycles. While some companies have already begun to embrace this new strategy, both industry and academia are still seeking a common understanding of the DevSecOps movement. The goal of this study is to report the state-of-the-practice of DevSecOps, including the impact of DevOps on security, practitioners' understanding of DevSecOps, and the practices associated with DevSecOps as well as the challenges of implementing DevSecOps. The authors used a mixed-methods approach for this research. The authors carried out a grey literature review on DevSecOps, and surveyed the practitioners of DevSecOps in industry of China. The status quo of DevSecOps in industry is summarized. Three major software security risks are identified with DevOps, where the establishment of DevOps pipeline provides opportunities for security-related activities. The authors classify the interpretations of DevSecOps into three core aspects of DevSecOps capabilities, cultural enablers, and technological enablers. To materialise the interpretations into daily software production activities, the recommended DevSecOps practices from three perspectives—people, process, and technology. Although a preliminary consensus is that DevSecOps is regarded as an extension of DevOps, there is a debate on whether DevSecOps is a superfluous term. While DevSecOps is attracting an increasing attention by industry, it is still in its infancy and more effort needs to be invested to promote it in both research and industry communities

Unleash the Power of Citizen Development: Leveraging Organizational Capabilities for Successful Low-Code Development Platform Adoption

Given the increasing demand for application development and process automation, Low-Code Development Platforms (LCDPs) have become highly relevant in recent years. However, the lack of familiarity with the implementation and application of LCDP in organizations poses a challenge. This publication therefore aims to shed light on the essential organizational capabilities that companies must master to overcome this obstacle. Using action design research, this study develops a model-based framework of 21 organizational capabilities for successful LCDP adoption. It underscores the importance of conceptual development as a prerequisite for effective management and long-term application of the technology. Furthermore, it emphasizes the importance of considering both technical and social aspects of the LCDP information system. The findings contribute to academia by providing a model-based capability framework, which serves as a structure for driving future research. Moreover, practitioners benefit from a practice-oriented and evaluated summary of initialization tasks and capabilities required for successful adoption

From Ad-Hoc Data Analytics to DataOps

The collection of high-quality data provides a key competitive advantage to companies in their decision-making process. It helps to understand customer behavior and enables the usage and deployment of new technologies based on machine learning. However, the process from collecting the data, to clean and process it to be used by data scientists and applications is often manual, non-optimized and error-prone. This increases the time that the data takes to deliver value for the business. To reduce this time companies are looking into automation and validation of the data processes. Data processes are the operational side of data analytic workflow.DataOps, a recently coined term by data scientists, data analysts and data engineers refer to a general process aimed to shorten the end-to-end data analytic life-cycle time by introducing automation in the data collection, validation, and verification process. Despite its increasing popularity among practitioners, research on this topic has been limited and does not provide a clear definition for the term or how a data analytic process evolves from ad-hoc data collection to fully automated data analytics as envisioned by DataOps.This research provides three main contributions. First, utilizing multi-vocal literature we provide a definition and a scope for the general process referred to as DataOps. Second, based on a case study with a large mobile telecommunication organization, we analyze how multiple data analytic teams evolve their infrastructure and processes towards DataOps. Also, we provide a stairway showing the different stages of the evolution process. With this evolution model, companies can identify the stage which they belong to and also, can try to move to the next stage by overcoming the challenges they encounter in the current stage