The benefits of virtualization across the software development pipeline

Abstract. The emergence of cloud computing and the evolution into service-based solutions across the software industry have influenced many changes in software development paradigms and methods. As a result, various forms of virtualization and container-based solutions have become more and more commonplace throughout the field, with technologies and frameworks such as Docker and Kubernetes becoming industry standard solutions to virtualization. This thesis is a literature review into existing research on virtualization and containers, and their use in various categories of the software industry. The aim of the thesis is to look at the reasons for the proliferation of virtual machines and containers, along with their benefits for the software development process, the continuous integration and delivery pipeline, and the different cloud platforms and providers. The benefits of virtualization are clearest in the cloud infrastructure, as cloud services are inherently built to utilize virtual machines. Containers and container orchestration systems allow container management and dynamic resource allocation, improving efficiency and reducing costs. In software development and testing, the modular and self-contained nature of containers allows for faster iteration and more problem-averse development. And finally, in the continuous integration and delivery pipelines, containers and container management tools allows automation, and lower overhead and complexity, enabling lower-threshold software deployment. Along with enabling cloud infrastructure as it exists today, the evolution of virtualization and containers in the software industry provide benefits across the board

Development of an ERP with CI/CD application, Authentication and System Auditing

Developing and maintaining software like ERPs can be challenging because of the complexity and the amount of data that these systems require maintaining. Many of the software programs can grow with weak structure, which lead to great effort to maintain, and with more probability to error. This project proposes that a development cycle that incorporates DevOps can have major bene ts, by not only removing some hassle the programmers and systems admins have with testing and deploying the system, but can also give a early feedback if the changes made into the application brings problems to the systems. The design of a CI/CD pipeline and audit logs, and the implementation in an ERP development helped get more feedback and cause of root problems, which lead to more confidence in the developers to make changes, and to escalate more quickly since the deployment is automatized.Desenvolver "software" como os ERPs podem ser difícil de manter devido à complexidade e a quantidade de dados envolvida nestes sistemas. Isto leva a que muitos destes "softwares" cresçam com uma estrutura de código fraca, o que leva a um esforço adicional para manter, e com maior probabilidade para erros. Este projeto propõe que a incorporação do conceito de DevOps no ciclo de desenvolvimento traz muitas vantagens, não só a remover algum trabalho dos programadores e dos administradores de sistemas ao ser mais fácil testar o sistema e fazer deploy do mesmo, mas também fornece uma forma de feedback mais rápida para eventuais erros. O "design" de uma pipeline CI/CD e logs para auditoria do sistema, e a respetiva implementação destes conceitos no desenvolvimento consegue dar mais feedback a problemas, o que leva a uma maior confiança dos programadores para fazer alterações, e conseguir escalar a solução mais rapidamente visto que a implantação é automatizada

Continuous integration and application deployment with the Kubernetes technology

Poslední dobou by téměř každý chtěl své aplikace nasadit do Kubernetes. Jenže pro plné využití Kubernetes je třeba přijmout s otevřenou náručí postupy průběžné integrace (CI) a nasazení (CD). Je třeba CI/CD pipeline. Ale k dispozici je až zdrcující množství open-source nástrojů, kde každý pokrývá různé části celého procesu. Následující text vysvětlí základy technologií, kterých bude pro pipeline třeba. A následně shrne některé z populárních open-source nástrojů využívaných pro CI/CD. Z open-source nástrojů navrhneme pipeline. Závěrečné porovnání možných řešení (včetně proprietárních) poskytne čtenáři konkrétní tipy a rady ohledně vytváření vlastní pipeline.It seems nearly everyone would like to deploy to Kubernetes nowadays. To efficiently leverage the power of Kubernetes one must first fully embrace continuous integration (CI) and deployment (CD) practices. A CI/CD pipeline is needed. But there is an overwhelming amount of open-source tools that cover various parts of the whole process.The following text explains the basics of the underlying technologies needed for a pipeline deploying to Kubernetes. And subsequently summarizes some of the popular open-source tools used for CI/CD. Then it designs a working pipeline from the researched tools. Finally, it summarizes some of the possible pipelines (including proprietary) and provides the reader with specific bits of advice on how to implement a pipeline

Orquestração de um pipeline de ferramentas para apoio ao ensino

The variety and nature of tools that are nowadays used on both academic and professional contexts have been increasing over the past few years. With the non stopping evolving cycle that technology suffers on a daily basis, this is a consequence that will be even more noticeable in a not so distant future. As a result, several problems have emerged. How to handle all of the crucial tools in a simple and reliable way? How to structure that process so it can scale, in order to apply it whenever the tools are being used by several people? Services specialization, such as microservices, taking advantage of the core tools features, allows the automation of almost all the needed configurations required. This thesis focus on the design and implementation of a solution that enhances the ease and efficiency of creating and configuring working environments either of students or workers, being, however, more focused on the academic environment. To achieve this, a prototype tool has been developed, which consists in several services that when combined together are capable of creating and configuring a software development integration pipeline. The prototype is responsive to a certain configuration input and handles all the tasks needed in between the configuration steps, keeping the resulting pipeline always up to date.A variedade e natureza das ferramentas que são hoje em dia utilizadas tanto em contextos académicos como profissionais têm vindo a aumentar nos últimos anos. Com o constante ciclo evolutivo que a tecnologia sofre diariamente, este aumento é um fator que será ainda mais percetível num futuro próximo. Por conseguinte, vários problemas têm surgido. Como lidar com todas as ferramentas cruciais de uma forma simples e fiável? Como estruturar esse processo para que possa ser escalado, a fim de o aplicar a situações em que as ferramentas estejam a ser utilizadas por várias pessoas? A especialização em serviços, surgindo aqui o conceito de microserviços, aproveitando algumas das funcionalidades oferecidas por parte das ferramentas, permite a automação de quase todas as configurações necessárias. Esta tese tem como foco a conceção e implementação de uma solução que torne mais simples e eficiente o processo de criação e configuração de ambientes de trabalho, quer de estudantes quer de trabalhadores dando, no entanto, mais foco à vertente académica. Para o conseguir, foi desenvolvido um protótipo, que consiste em vários serviços que, quando combinados, são capazes de criar e configurar uma pipeline de integração de software. O protótipo tem como input uma determinada estrutura de dados e trata de todas as tarefas necessárias entre as etapas de configuração, mantendo assim a pipeline sempre atualizada. Esta tese relata todo o processo envolvido na elaboração da solução final em cima descrita, estando inerentes as fases de estudo acerca dos conceitos fulcrais ao problema, análise de valor e de negócio, proposta de design, implementação e avaliação da solução desejada

Achieving Continuous Delivery of Immutable Containerized Microservices with Mesos/Marathon

In the recent years, DevOps methodologies have been introduced to extend the traditional agile principles which have brought up on us a paradigm shift in migrating applications towards a cloud-native architecture. Today, microservices, containers, and Continuous Integration/Continuous Delivery have become critical to any organization’s transformation journey towards developing lean artifacts and dealing with the growing demand of pushing new features, iterating rapidly to keep the customers happy. Traditionally, applications have been packaged and delivered in virtual machines. But, with the adoption of microservices architectures, containerized applications are becoming the standard way to deploy services to production. Thanks to container orchestration tools like Marathon, containers can now be deployed and monitored at scale with ease. Microservices and Containers along with Container Orchestration tools disrupt and redefine DevOps, especially the delivery pipeline. This Master’s thesis project focuses on deploying highly scalable microservices packed as immutable containers onto a Mesos cluster using a container orchestrating framework called Marathon. This is achieved by implementing a CI/CD pipeline and bringing in to play some of the greatest and latest practices and tools like Docker, Terraform, Jenkins, Consul, Vault, Prometheus, etc. The thesis is aimed to showcase why we need to design systems around microservices architecture, packaging cloud-native applications into containers, service discovery and many other latest trends within the DevOps realm that contribute to the continuous delivery pipeline. At BetterDoctor Inc., it is observed that this project improved the avg. release cycle, increased team members’ productivity and collaboration, reduced infrastructure costs and deployment failure rates. With the CD pipeline in place along with container orchestration tools it has been observed that the organisation could achieve Hyperscale computing as and when business demands