Report from GI-Dagstuhl Seminar 16394: Software Performance Engineering in the DevOps World

This report documents the program and the outcomes of GI-Dagstuhl Seminar 16394 "Software Performance Engineering in the DevOps World". The seminar addressed the problem of performance-aware DevOps. Both, DevOps and performance engineering have been growing trends over the past one to two years, in no small part due to the rise in importance of identifying performance anomalies in the operations (Ops) of cloud and big data systems and feeding these back to the development (Dev). However, so far, the research community has treated software engineering, performance engineering, and cloud computing mostly as individual research areas. We aimed to identify cross-community collaboration, and to set the path for long-lasting collaborations towards performance-aware DevOps. The main goal of the seminar was to bring together young researchers (PhD students in a later stage of their PhD, as well as PostDocs or Junior Professors) in the areas of (i) software engineering, (ii) performance engineering, and (iii) cloud computing and big data to present their current research projects, to exchange experience and expertise, to discuss research challenges, and to develop ideas for future collaborations

Critical success factors for DevOps adoption in information systems development

Adopting DevOps is challenging since it makes a significant paradigm shift in the Information Systems Development process. DevOps is a trending approach attached to the Agile Software Development Methodology, which facilitates adaptation to the customers\u27 rapidly-changing requirements. It keeps one front step by introducing software operators who support the transmission between software and implementation into the software development team by confirming faster development, quality assurance, and easy maintenance of Information Systems. However, software development companies reported challenges in adopting DevOps. It is critical to control those challenges while getting hold of the benefits by studying Critical Success Factors (CSF) for adopting DevOps. This study aimed to analyze the use of DevOps approach in IS developments by exploring CSFs of DevOps. A systematic literature review was applied to identify CSFs. These factors were confirmed by interviewing DevOps practitioners while identifying more frequent CSFs in the software development industry. Finally, the research presents a conceptual model for CSFs of DevOps, which is a guide to reap the DevOps benefits while reducing the hurdles for enhancing the success of Information Systems. The conceptual model presents CSFs of DevOps by grouping them into four areas: collaborative culture, DevOps practices, proficient DevOps team, and Metrics & Measurement

Critical success factors for DevOps adoption in information systems development

Adopting DevOps is challenging since it makes a significant paradigm shift in the Information Systems Development process. DevOps is a trending approach attached to the Agile Software Development Methodology, which facilitates adaptation to the customers' rapidly-changing requirements. It keeps one front step by introducing software operators who support the transmission between software and implementation into the software development team by confirming faster development, quality assurance, and easy maintenance of Information Systems (IS). However, software development companies reported challenges in adopting DevOps. It is critical to control those challenges while getting hold of the benefits by studying Critical Success Factors (CSF) for adopting DevOps. This study aimed to analyze the use of DevOps approach in IS developments by exploring CSFs of DevOps. A systematic literature review was applied to identify CSFs. These factors were confirmed by interviewing DevOps practitioners while identifying more frequent CSFs in the software development industry. Finally, the research presents a conceptual model for CSFs of DevOps, which is a guide to reap the DevOps benefits while reducing the hurdles for enhancing the success of IS. The conceptual model presents CSFs of DevOps by grouping them into four areas: collaborative culture, DevOps practices, proficient DevOps team, and metrics & measurement

Automation of machine learning models benchmarking

Dissertação de mestrado em Engenharia InformáticaNa área de ciência de dados, o machine learning está-se a revelar uma ferramenta essencial para resolver problemas complexos. As empresas estão a investir em equipas de ciência de dados e Machine Learning para desenvolver modelos que apresentem valor para os clientes. No entanto, estes modelos são uma pequena percentagem de uma pipeline de projetos de Machine Learning (ML) e, para entregar um produto de ML completo, é necessário um número maior de componentes. DevOps é uma mentalidade de engenharia e um conjunto de práticas que visa unificar o processo de desenvolvimento e o processo de operações em um software, MLOps é um conceito similar a DevOps mas aplicado ao desenvolvimento e entrega de soluções de ML. O nível de automatização das etapas em uma pipeline de ML define a maturidade do processo de ML, que reflete a velocidade de treino de novos modelos com novos dados ou de treino de novos modelos com diferentes implementações. Um sistema de ML é um sistema de software, desenvolvimento e atualizações contínuas são necessárias para garantir um sistema que escale conforme as necessidades. O principal objetivo desta tese é apoiar a criação de um sistema integrado de ML com uma arquitetura que proporcione a capacidade de ser continuamente operada em um ambiente de produção. Um conceito para avaliação de desempenho de algoritmos deve ser elaborado e implementado. O principal obetivo e melhorar e ace'erar o cicio de desenvolvimento de modelos de ML na empresa. Para atingir este objetivo surge a necessidade de definir uma arquitetura com especificações e a implementação de processos automatizadas num pipeline de ML existente, este processo têm como objetivo alcançar uma ferramenta de benchmark de modelos, com capacidade de analisar o desempenho do modelo, um motor de inferência e um banco de dados para armazenar todas as métricas computadas. Um sistema baseado em IA em desenvolvimento fornece o caso de estudo para desenvolver e validar a arquitetura. Os avanços atuais na área da condução semiautomática introduz a necessidade de sistemas de monitoramento que podem localizar e detectar eventos especificas no veículo. Os conjuntos de sensores são instalados dentro da cabine para alimentar sistemas inteligentes que visam analisar e sinalizar certos comportamentos que podem impactar a segurança e o conforto dos passageiros..In the field of data science, ML is proving to be a core feature to solve complex real-world problems. Businesses are investing in data science and ML teams to develop AI based models that can deliver business value to their users. However, these models are only a small fraction of an ML project pipeline, and to deliver an end to end ML product, a greater number of components are needed. DevOps is an engineering mindset and a set of practices that aims to unify the development process and the operation process on software. MlOps is a similar concept to DevOps but applicable to the development and delivery of ML based solutions. The automation of the steps in a ML pipeline defines the maturity of the ML process, reflecting the velocity of training new models given new data or training new models given new implementations. An ML system is a software system that can support development, provide continuous integration and continuous delivery apply to help guarantee that one can reliably build and operate ML systems at scale. The main objective of this thesis are to support the creation of an integrated ML system with an archi tecture that provides the ability to be continuously operated in a production-like environment. Furthermore, a concept to evaluate the performance of algorithms shall be devised and implemented. The end goal is to improve and accelerate the ML development lifecycle. To achieve this goal surges the need to define an architecture alongside specifications and the implementation of several automated steps into an existing ML pipeline. To improve and accelerate model development an model engine benchmark tool is devised capable of several features, including the ability to have dashboards for model performance evaluation, an automatic inference engine, performance metrics for the model and a database to store all the computed metrics and metadata. An AI-based system under development provides the case study to develop and validate this architec ture. The current advances of semi-automated driving introduce the need for monitoring systems to scan and detect specific events in the vehicle. Sensor clusters are installed inside the vehicle cabin to feed data to intelligent systems that aim to analyze and red flag certain behaviours that can potentially impact passengers safety and comfort while using the vehicle

DevOps : Continuous integration and continuous deployment applied

DevOps is a trending concept in the SW industry introduced and popularized during the last decade, this thesis goes deep into the concept, the culture and fields related to it which are present in almost each project and company nowadays. The research model followed in the thesis basically consists in getting in contact with the DevOps culture by designing, developing and implementing a Python tool with some Continuous Integration and Continuous Deployment features. Basically this tool consists on a back-end REST API capable of creating and automating builds on any connected slave node to the tool, similar to traditional DevOps tools logic such as Jenkins or TeamCity, but in a lightweight, portable and OS independent solution, also a frontend is developed in order to make the tool easier and simpler to use. In order to demonstrate the power and the capabilities of the tool we will containerize a build environment with Docker and automate its build process with the tool as well as deploying the binaries resulting from the build process.DevOps es un concepto de tendencia en la industria del SW introducido y popularizado durante la última década, esta tesis profundiza en el concepto, la cultura y los campos relacionados que están presentes en casi cada proyecto y empresa en la actualidad. El modelo de investigación seguido en la tesis consiste básicamente en ponerse en contacto con la cultura DevOps mediante el diseño, desarrollo e implementación de una herramienta Python con algunas características de integración continua y despliegue continuo. Básicamente, esta herramienta consiste en una API REST de back-end capaz de crear y automatizar compilaciones en cualquier nodo esclavo conectado a la herramienta, similar a la lógica de herramientas DevOps tradicionales como Jenkins o TeamCity, pero en una solución ligera, portátil e independiente del sistema operativo, también un front-end se ha desarrollado para hacer que la herramienta sea más fácil y simple de usar. Para demostrar el poder y las capacidades de la herramienta, contenerizaremos un entorno de compilación con Docker y automatizaremos su proceso de compilación con la herramienta, así como desplegaremos los binarios resultantes del proceso de compilación.DevOps es un concepte tendència a la indústria del SW, introduït i popularitzat durant la darrera decada, aquesta tesis aprofundeix en el concepte, la cultura i aspectes relacionats, que son present a la majoria de projectes i empreses avui en dia. El model d'investigació a seguir durant la tesis es basa en entrar en contecte directe amb la cultura de DevOps dissenyant, desenvolupant i implementant una eina en Python amb característiques tant de Integració Continua com de Desplegament Continuu. En resum, la eina consisteix en un backend REST API que permet la creació i automatització de construccions de projectes en qualsevol node esclau conectat a la eina, de manera semblant a eines tradicional de DevOps com son Jenkins o Teamcity pero d'una manera lleugera, portable i independent del sistema operatiu. Tambe s'ha desenvolupat un frontend per facilitar l'us de l'eina. Finalment, per a demostrar el funcionament i la capacitat de l'eina crearem un entorn de construcció amb docker del qual automatitzarem el procés de construcció amb l'eina a més de desplegar els binaris resultants del procés

Beyond MLOps: The Lifecycle of Machine Learning-based Solutions

Organizations increasingly use machine learning (ML) to transform their operations. The technical complexity and unique challenges of ML lead to the emergence of ML operations (MLOps) practices. However, the research on MLOps is in its infancy and is fragmented across disciplines. We extend and integrate these conversations by developing a framework that accounts for the technical, organizational, behavioral, and temporal aspects of the overarching ML-based solution lifecycle. We identify the key components of ML-based solution lifecycle and their configuration through an in-depth study of Finland’s Artificial Intelligence Accelerator (FAIA) and follow-up semi-structured interviews with experts from multiple international organizations outside FAIA. This study contributes to the recent IS literature concerned with the sociotechnical aspects of ML. We bring new insights into the discussion on organizational learning, conjoined agency, and automation and augmentation. These insights extend and complement MLOps practices, thereby helping organizations better realize the potential of ML technology

Relational Leadership, DevOps, and The Post-PC Era: Toward a Practical Theory for 21st Century Technology Leaders

This theoretically oriented scholarly personal narrative (SPN) explored how the constructionist view of relational leadership might be applied in a post-PC technological era marked by fast-paced innovation and an always on technology organization and infrastructure. Through reflecting on my personal and professional experience, I hope to offer the reflective scholar-practitioner new ways of thinking, present relational practices and suggest ways of being a leader participating in the fast-paced technology driven world. This new way of being combined both relational leadership and new DevOps practices that reduce organizational friction, break down departmental silos, and increase employee engagement in technology operations. Through this inquiry, I uncovered several practices and ways of being that are grounded in philosophical, theoretical, and social domains. In challenging the taken-for-granted reality of managing technology, I am attempting to produce practices for higher performance, humane, sustainable, and inspiring corporate information technology (IT) departments. The electronic version of this Dissertation is at AURA, http://aura.antioch.edu/etds/ and OhioLink ETD Center, www.ohiolink.edu/et