Automation of the Continuous Integration (CI) - Continuous Delivery/Deployment (CD) Software Development

Continuous Integration (CI) is a practice in software development where developers periodically merge code changes in a central shared repository, after which automatic versions and tests are executed. CI entails an automation component (the target of this project) and a cultural one, as developers have to learn to integrate code periodically. The main goal of CI is to reduce the time to feedback over the software integration process, allowing to locate and fix bugs more easily and quickly, thus enhancing it quality while reducing the time to validate and publish new soIn traditional software development, where teams of developers worked on the same project in isolation, often led to problems integrating the resulting code. Due to this isolation, the project was not deliverable until the integration of all its parts, which was tedious and generated errors. The Continuous Integration (CI ) emerged as a practice to solve the problems of traditional methodology, with the aim of improving the quality of the code. This thesis sets out what is it and how Continuous Integration is achieved, the principles that makes it as effective as possible and the processes that follow as a consequence, to thus introduce the context of its objective: the creation of a system that automates the start-up and set-up of an environment to be able to apply the methodology of continuous integration

Provisioning Google Kubernetes Engine (GKE) Cluster dengan Menggunakan Terraform dan Jenkins pada Dua Environment

Pada sistem dengan cangkupan besar membutuhkan ratusan bahkan ribuan server.Setiap server yang ada juga harus saling terhubung satu sama lain agar dapat berkomunikasi secara du arah dan setiap server memiliki kapasitasnya masing masing.Pada tahap development selalu membutuhkan minimal dua environment, yaitu development environment dan production environment. Provisioning GKE Cluster menggunakan Terraform dan Jenkins dapat memudahkan tim DevOps dalam menyiapkan Server sesuai kebutuhan secara otomatis. Hanya memerlukan waktu 15 menit 18,412 detik untuk menyiapkan satu cluster dengan tiga node pada environment Development dan 13 menit 26,407 detik untuk menyiapkan satu cluster dengan satu node pada environment Production. Dengan adanya sistem in membantu tim DevOps untuk menyiapkan server dengan lebih cepat dan minim kendala karena semakin kompleksnya kebutuhan suatu perusahaan

Choosing the Right IaC Tool for Building Reusable Cloud Infrastructure

This study examines the critical role of Infrastructure as a Code (IaC) tools in automating the provisioning, configuration, and management of cloud infrastructure, which is essential for the rapid growth and success of organizations in the cloud computing domain. By enabling cloud developers to define infrastructure in code format, IaC tools facilitate easy versioning, change tracking, and deployment automation. This capability ensures that infrastructure can be consistently and reliably deployed across various environments, significantly reducing the risk of errors and downtime. However, the implementation of IaC tools without adequate knowledge can introduce risks. The research emphasizes the importance of selecting the right IaC tool, considering factors like scalability, security, and compliance enhancement. The study aims to compare the features, ease of use, scalability, security features, and integration capabilities of the most utilized IaC tools, providing insights into building reusable cloud infrastructure. Through a comprehensive exploration of cloud services evolution, prevalent cloud resources, and a detailed analysis of selected IaC tools, this thesis offers a comparative assessment of these tools across different cloud service providers. The findings, derived from empirical tests, offer a deep dive into the technical nuances, qualitative attributes, and integration potential of these tools, paving the way for future research in the field of cloud computing and infrastructure development

Infrastructure-as-Code: Automating the Deployment on AWS using Terraform

In my master’s project, I used Terraform to create a scalable infrastructure on Amazon Web Services (AWS) for my personal website. Terraform is an open-source infrastructure-as-code (IAC) tool that allows you to create, manage and provision infrastructure resources, such as virtual machines, storage accounts, networks, and more, across multiple cloud providers and on-premises data centers using a declarative configuration language. A scalable infrastructure is important because it enables a system or application to handle increasing amounts of traffic or workload without experiencing performance issues or downtime. It ensures that the system remains responsive, available, and reliable as an organization grows or its user base expands. Amazon Web Services (AWS) is a comprehensive cloud computing platform offered by Amazon. It provides a broad range of cloud-based services, including compute, storage, networking, database, analytics, machine learning, security and more. These services are designed to help organizations of all size to build, deploy and scale applications and infrastructure in the cloud with ease and flexibility. This project highlights the benefits of Infrastructure as Code and the power of Terraform with AWS for automating infrastructure deployment

Exploring Security Practices in Infrastructure as Code: An Empirical Study

Cloud computing has become popular thanks to the widespread use of Infrastructure as Code (IaC) tools, allowing the community to conveniently manage and configure cloud infrastructure using scripts. However, the scripting process itself does not automatically prevent practitioners from introducing misconfigurations, vulnerabilities, or privacy risks. As a result, ensuring security relies on practitioners understanding and the adoption of explicit policies, guidelines, or best practices. In order to understand how practitioners deal with this problem, in this work, we perform an empirical study analyzing the adoption of IaC scripted security best practices. First, we select and categorize widely recognized Terraform security practices promulgated in the industry for popular cloud providers such as AWS, Azure, and Google Cloud. Next, we assess the adoption of these practices by each cloud provider, analyzing a sample of 812 open-source projects hosted on GitHub. For that, we scan each project configuration files, looking for policy implementation through static analysis (checkov). Additionally, we investigate GitHub measures that might be correlated with adopting these best practices. The category Access policy emerges as the most widely adopted in all providers, while Encryption in rest are the most neglected policies. Regarding GitHub measures correlated with best practice adoption, we observe a positive, strong correlation between a repository number of stars and adopting practices in its cloud infrastructure. Based on our findings, we provide guidelines for cloud practitioners to limit infrastructure vulnerability and discuss further aspects associated with policies that have yet to be extensively embraced within the industry.Comment: 50 pages, 13 figures, 10 table

Viability analysis of infrastructure managers for the automated deployment of virtual HPC clusters on Cloud environments

Due to the increase of hybrid environments and the rise of Cloud resources in the IT sector, we have also seen an increase in the complexity of the administration we face today. Today, more than ever, automation tools are needed to facilitate not only the creation but also the management of these new environments. The lack of standardization, especially for HPC systems, forces us to present a proposal to evaluate the different tools that claim to facilitate the automation of this type of systems in Cloud environments and a subsequent analysis of them. In this Master's Thesis we will discover the importance of Cloud environments in HPC systems, the challenges that this presents and an analysis methodology that will serve as a reference for administrators and IT professionals who need to manage such complex and dynamic environments as HPC clusters in Cloud environments.Debido al aumento de entornos híbridos y al auge de los recursos Cloud en el sector IT, se ha podido observar también un incremento en la complejidad de administración de los mismos. Hoy más que nunca son necesarias las herramientas de automatización que faciliten no sólo la creación sino también la gestión de estos nuevos entornos. La falta de estandarización especialmente para los sistemas HPC fuerza a presentar una propuesta de evaluación de las distintas herramientas que presumen de facilitar la automatización de este tipo de sistemas en entornos Cloud y un posterior análisis sobre las mismas. En este Trabajo Final de Master descubriremos la importancia de los entornos Cloud en los sistemas HPC, los retos que ello presenta y una metodología de análisis que sirva de referencia para administradores y profesionales del sector TI que se vean en la necesidad de gestionar entornos tan complejos y dinámicos como los clústeres HPC en entornos Cloud

A DevOps approach to infrastructure on demand

As DevOps grows in importance in companies, there is an increasing interest in automating the process of building and deploying infrastructure, having as an objective reduce the complexity for non DevOps engineers and making it so that infrastructure is less error prone, which is not the case when doing it manually. This work aims to explore how to build a solution that allows to manage infrastructure on demand while supporting specific services that are relevant for git profiles analysis, such as Sonarqube and Jenkins. Firstly, this work starts by introducing its context, the problem that the solution is trying to solve and the methodology used to develop the solution. On the State-of-the-Art various topics are presented in order to give all the information needed to understand the implementation of the solution, including concepts such as DevOps and Automation, while going over specific technologies such as GraphQL, Docker, Terraform and Ansible. A value analysis was also done to explore what are the main concerns for stakeholders when managing their infrastructure and to define the value of the solution being developed. Lastly, the solution was implemented making use of various technologies and with scalability in mind that would allow it to grow in the amount of services supported with minimum changes. The work is interesting for someone that is interested in DevOps, Infrastructure-as-Code and automation in general.Com o crescimento da importância de DevOps em empresas existe um interesse acrescido em automatizar o processo de construir e de dar deploy de infra-estrutura, tendo como objectivo reduzir a complexidade para engenheiros menos proficientes em DevOps, e construir infraestrutura que é menos propensa a erros, o que não acontece quando feito manualmente. Este trabalho visa implementar uma solução capaz de gerir infra-estrutura a pedido e ao mesmo tempo suportar serviços específicos relevantes para a análise de perfis git, como por exemplo Sonarqube e Jenkins. Em primeiro lugar, este trabalho começa por introduzir o seu contexto, o problema que a solução está a tentar resolver e a metodologia utilizada para desenvolver a solução. No estado da arte são apresentados vários tópicos com a finalidade de fornecer toda a informação necessária para compreender a implementação da solução, incluindo conceitos como DevOps e automação, são também exploradas tecnologias específicas como GraphQL, Docker, Terraform e Ansible. Foi também feita uma análise de valor para explorar quais são as principais preocupações das partes interessadas na gestão das infra-estruturas das suas empresas e para definir o valor da solução que está a ser desenvolvida. Finalmente, a solução foi implementada, recorrendo a várias tecnologias e tendo em mente a escalabilidade da solução que permitiria crescer na quantidade de serviços suportados requerendo alterações mínimas. O trabalho é interessante para alguém que esteja interessado em DevOps, Infraestrutura como código e automatização em geral