Herramientas de infraestructura como código: Ansible, Terrafom, Chef, Puppet

Infrastructure tools such as code (IaC) allow the tasks performed by IT departments to be automated quickly and dynamically through the use of scripting languages, which allow managing, creating, manipulating and distributing multiple large-scale computing resources within of Cloud Computing infrastructure. Infrastructure tools such as Ansible code, Terrafom, Chef, Puppet, generate a virtual representation of all the physical and scalable infrastructure of a Cloud Computing and Data Center platform, making it easy to program and dynamic.Las herramientas de Infraestructura como código (IaC) permiten automatizar las tareas realizadas por los departamentos de IT de forma rápida y dinámica mediante el uso de lenguajes de programación de scripts, que permiten administrar, crear, manipular y distribuir múltiples recursos informáticos a gran escala dentro de una infraestructura de Cloud Computing. Las herramientas de Infraestructura como código Ansible, Terrafom,  Chef, Puppet  generan una representación virtual de toda la infraestructura física y  escalable de una plataforma Cloud Computing y de  Centro de Datos, facilitando  que esta sea programable y dinámica.&nbsp

Detection of unintended configuration changes in continuous deployment pipelines

Once an Amazon Web Services employee took numerous servers offline that should have stayed online. The resulting US-East outage originated from a single integer that was incorrectly inserted in a configuration file. The configuration file defined the number of servers that should be up and running. The change was legal, but the number of servers that where taken offline was to high. We took the idea of such simple errors and applied them to continuous deployment pipelines. Continuous deployment pipelines are the next evolutionary step in continuous build pipelines. The Amazon Cloud is a highly automated environment. Pipelines are similar to the Amazon cloud a highly automated environment. Small errors can have potential catastrophic outcomes. We interviewed two experts in two differently sized companies which are using continuous integration and continuous delivery pipelines. Based on the information that was provided by both experts about the state of current continuous pipelines, we derived influence factors that are problematic in continuous deployment pipelines. The discovered influence factors already exist in currently used continuous integration/delivery pipelines were they do pose less significant threats than in continuous deployment pipelines. The enhanced automated deployment process of continuous deployment pipelines are making these factors problematic. We developed classification and improvement methods for each of the discovered influence factors. These methods can be used to strengthen a pipeline against unintended configuration changes.Vor einiger Zeit schaltete ein Mitarbeiter bei Amazon Web Services einige Server aus, die jedoch hätten eingeschaltet bleiben sollen. Der resultierende Systemausfall der Zone US-East hatte ihren Ursprung in einem einzigen falsch eingegebenen Zahlenwert in einer Konfigurationsdatei. Die Konfigurationsdatei definierte die Anzahl der Server die angeschaltet waren. Die änderung war beabsichtigt, aber die Anzahl der abgeschalteten Server war zu hoch. Wir haben diesen Vorfall von Fehlern in automatisierten Umgebungen genommen und haben das Konzept auf Continuous Deployment Pipelines angewandt. Continuous Deployment Pipelines sind der nächste Schritt in der Evolution von Continuous Pipelines. Die Amazon Cloud ist eine hochautomatisiertes Umgebung. Pipelines sind ähnlich wie die Amazon Cloud ein ebenfalls hoch automatisiertes Umfeld. Selbst kleine Fehler einen katastrophalen Ausgang nehmen können. Wir haben zwei Experten in unterschiedlich großen Firmen interviewt. Beide Experten arbeiten mit den evolutionären Vorgängern von Continuous Deployment Pipelines, nämlich Continuous Integration und Continuous Delivery Pipelines. Basierend auf den Informationen, die uns die beiden beide Experten über den aktuellen Stand von Continuous Pipelines gaben, habe wir Einflussfaktoren abgeleitet die problematisch im Umfeld von Continuous Deployment Pipelines sind. Probleme, die in momentan existierenden Continuous Delivery Pipelines heute noch eine untergeordnete Rolle spiele, werden jedoch bei hochautomatisierten Continuous Deployment Pipelines problematischer. Die entdeckten Einflussfaktoren sind Probleme die bereits in momentan existierenden Continuous Integration/Delivery Pipelines existieren, in welchen sie jedoch nur eine untergeordnete Rolle spielen. Die erhöhte Automatisierung in Continuos Deployment Pipelines machen diese Einflussfaktoren problematischer. Wir haben sowohl Klassifikationsmöglichkeiten als auch Verbesserungsmöglichkeiten für jeden der entdeckten Einflussfaktoren entwickelt. Diese Methoden können zur Stabilisierung von Pipelines genutzt werden, um sie gegen unbeabsichtigte Konfigurationsänderungen zu schützen

Supply Chain (micro)TMS development

Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies ManagementThe rise of technology across many verticals has necessitated the company’s move to digitalization. Despite “XPTO” company a well know player on the retail and success on e‐commerce internal market, they aimed at the strategy of continuous innovation to drive business growth and strengthen their position as a premium brand. They decided to move forward into digitalism inside cloud based solutions to get all the advantages of microservices architecture: optimize logistics and supply chain management, speed up the workflow and maximize service efficiency. An agile organization is not achieved purely by shifting the focus from traditional functional/ technological oriented organizations. The new way to organize teams must reflect all the principles and right segregations of roles, which will be the most immediate and visible disruption and cutover from the traditional way of managing the IT. In this project we aim to use agile framework with development based in house cloud microservice solution for a (micro)TMS solution/system that address the immediate needs imposed by the market in order to use it has competitive advantage

Jatkuvan integraation ja jatkuvan toimittamisen toimintamallin hyödyntäminen tietoverkon hallinnassa

Jatkuvan integrointi ja jatkuva toimittaminen on toimintamalli, jota on hyödynnetty sovelluskehityksessä sekä sovellusympäristöjen ylläpidossa. Se on osaltaan mahdollistanut sellaisten tietojärjestelmien kehityksen, joissa tänä päivänä pyritään automatisoimaan toistuvien tehtävien suorittamista, ja jotka ovat käytettävissä paikasta riippumatta. Tietoverkot ovat osa näitä tietojärjestelmiä, ja niiden ylläpidolta vaaditaan vastaavanlaista ketteryyttä, varmuutta ja täsmällisyyttä. Yhtenä mahdollisuutena kehittää tietoverkkojen hallintaa nähdään jatkuvan integroinnin ja jatkuvan toimittamisen toimintamallin hyödyntäminen. Tässä tutkimuksessa selvitetään millainen tietoverkon hallinnan toimintamalli voisi olla. Verkonhallinnan näkökulmasta mm. erilaiset verkkoympäristöjen valvontatyökalut ovat osa hallintakokonaisuutta, ja myös niissä on omat mahdollisuutensa tehdä ohjelmallisia konfiguraatioita automaattisesti. Tutkimuksessa toimintamallin hyödyntäminen on kuitenkin rajattu verkkolaitteiden konfiguraatioiden tekemiseen. Tutkimus suoritettiin käyttämällä suunnittelutieteellistä tutkimusmenetelmää. Tutkimusongelma asetettiin alfa-yrityksessä muodostuneen tarpeen mukaisesti. Suunnittelusykli aloitettiin kokoamalla jatkuvan integroinnin ja jatkuvan toimittamisen tutkimustiedosta vaatimukset, jotka toimintamallin sovitus verkonhallintaan tulisi teorian pohjalta täyttää. Sykli eteni toimintamallin rakentamisella, ja arviointivaiheessa havainnoimalla miten artefakti pystyi vastaamaan asetettuihin vaatimuksiin. Lopuksi artefakti esiteltiin tutkimusongelman asettaneelle organisaatiolle simuloimalla. Organisaation palautteen lisäksi havainnoitiin, miten artefakti vastasi alkuperäiseen asetettuun ongelmaan. Jatkuvan integroinnin ja jatkuvan toimittamisen toimintamallia hyödyntämällä on mahdollista toteuttaa käyttöönottoputki, jolla verkkolaitteiden konfiguraatioita voidaan kehittää, testata, provisioida ja todentaa toimiviksi. Verrattuna manuaalisesti tehtävään provisointiin, konfiguraatioiden ajaminen verkkolaitteisiin tapahtuu toimintamallin avulla täsmällisesti sekä useisiin laitteisiin yhtäaikaisesti skaalautuen. Provisioinnin jälkeinen automaattinen, konfiguroitujen palveluiden toiminnallisuuden testaus on oleellinen mallin tuoma etu. Tärkeää on konfiguraatioita määritellessä suunnitella myös niitä vastaava testaus-skripti. Kun muutos tehdään, kaikki testit ajetaan automaattisesti. Tämä on huomattava parannus manuaalisesti tehtäville ja silmämääräisesti tarkistettaville testeille

Improving the maintainability and developer experience of Terraform code

Infrastructure as code is a popular way to describe and manage cloud infrastructure, allowing developers to manage infrastructure along with all other code. Infrastructure as code is however often neglected, resulting in increasing technical debt and hindrance to maintenance and further development possibilities. This thesis improves the infrastructure as code implementation of a case project through a constructive research approach. The case project is a production-use multienvironment infrastructure, implemented with Terraform and running on Google Cloud. The shortcomings of the case project were divided into seven action points, each of which was planned and implemented separately. Based on the results of the implementation, a set of best practices for working with infrastructure as code, Terraform and Google Cloud was compiled. The implementation of the action points was successful, improving the case project significantly. Most important improvement to the case project was the implementation of the stack of modules approach to declaring multiple environments. All in all the implementation shortened the infrastructure as code with some 2400 lines and reduced the need for tedious manual work. In addition, the thesis reaffirmed existing practices and conventions, but did not discover any new significant contributions. The same rules of law apply to infrastructure as code as to all other code. The thesis also verified that making fundamental changes to a production-use infrastructure as code implementation does not pose any blocking issues

B!SON: A Tool for Open Access Journal Recommendation

Finding a suitable open access journal to publish scientific work is a complex task: Researchers have to navigate a constantly growing number of journals, institutional agreements with publishers, funders’ conditions and the risk of Predatory Publishers. To help with these challenges, we introduce a web-based journal recommendation system called B!SON. It is developed based on a systematic requirements analysis, built on open data, gives publisher-independent recommendations and works across domains. It suggests open access journals based on title, abstract and references provided by the user. The recommendation quality has been evaluated using a large test set of 10,000 articles. Development by two German scientific libraries ensures the longevity of the project

MoCIP: Un enfoque dirigido por modelos para el aprovisionamiento de infraestructura en la nube

[ES] DevOps (Development & Operations) es un nuevo movimiento que fomenta la colaboración entre los desarrolladores y el personal de operaciones a través de un conjunto de principios, prácticas y herramientas para optimizar el tiempo de entrega del software. En particular, la práctica del despliegue continuo de software es una gran fuente de problemas y genera mucha atención cuando los artefactos de software se entregan tarde o cuando un defecto crítico llega a producción. Al mismo tiempo, la práctica del despliegue continuo es la frontera entre los desarrolladores y el personal de operaciones en el ciclo de entrega del software. En consecuencia, se recomienda iniciar la implantación de DevOps con la práctica del despliegue de software. Para enfrentar este desafío, los profesionales e investigadores están utilizando la Infraestructura como Código (Infrastructure as Code, IaC), que es un enfoque para la automatización de la infraestructura basado en prácticas de desarrollo de software. El objetivo de IaC es definir en un script todas las instrucciones para crear, actualizar y ejecutar recursos de infraestructura. En este escenario, la automatización del aprovisionamiento de la infraestructura acelera la práctica del despliegue continuo en el ciclo de entrega del software. La computación en la nube se ha convertido en el principal modelo de pago por uso utilizado por profesionales e investigadores para conseguir servicios en la nube en un corto período de tiempo. En este escenario, las compañías están pasando de generar potencia informática in-house hacia la obtención de recursos informáticos provistos en la nube a través de internet como servicios web. Al mismo tiempo, IaC y la computación en la nube están promoviendo algunos cambios en la industria. Por ejemplo, los equipos de operaciones pasan todo su tiempo trabajando en software, en lugar de configurar servidores y conectar cables de red. Existe una variedad de herramientas IaC que utilizan scripts para definir y ejecutar los recursos de infraestructura en diferentes proveedores de servicios IaaS (Infrastructure as a Service). Sin embargo, la diversidad de lenguajes de scripting de las herramientas IaC junto con la heterogeneidad del tipo de infraestructura que ofrece cada proveedor de servicios IaaS, han ocasionado que utilizar scripts IaC para el aprovisionamiento de infraestructura sea una actividad lenta y propensa a errores. El objetivo del proyecto de doctorado es proponer una solución a la diversidad de los lenguajes de scripting ¿de las herramientas IaC¿ y a la heterogeneidad del tipo de infraestructura que ofrece cada proveedor de servicios IaaS respecto al aprovisionamiento de infraestructura en la nube. Para afrontar estos desafíos se propone MoCIP (A Model-Driven Approach to Cloud Infrastructure Provisioning). MoCIP es un enfoque dirigido por modelos para el aprovisionamiento de la infraestructura en la nube que soporta IaC mediante la Ingeniería de Software Dirigida por Modelos (Model-Driven Engineering, MDE). MoCIP utiliza los dos principios fundamentales de MDE: abstracción y automatización. En primer lugar, se desarrolló el lenguaje específico de dominio ArgonML para abstraer las capacidades de la computación en la nube, tales como cómputo, elasticidad, almacenamiento y redes. El lenguaje ArgonML permite modelar los recursos de infraestructura de la nube. En segundo lugar, se desarrolló la herramienta ARGON para automatizar el aprovisionamiento de infraestructura en la nube. ARGON realiza transformaciones de modelo a modelo para generar modelos que representan la infraestructura de diferentes proveedores de servicios IaaS. Además, ARGON realiza transformaciones de modelo a texto para generar scripts IaC con la información subyacente de cada proveedor de servicios IaaS y de cada herramienta de aprovisionamiento de infraestructura.[CA] DevOps (Development & Operations) és un nou moviment que fomenta la col·laboració entre els desenvolupadors i el personal d'operacions a través d'un conjunt de principis, pràctiques i eines per a millorar el temps de lliurament del programari. En particular, la pràctica del desplegament continu de programari és una gran font de problemes i genera molta atenció quan els artefactes de programari s'entreguen tard o quan un defecte crític arriba a producció. Al mateix temps, la pràctica del desplegament continu és la frontera entre els desenvolupadors i el personal d'operacions en el cicle de lliurament del programari. En conseqüència, es recomana iniciar la implantació de DevOps amb la pràctica del desplegament de programari. Per a enfrontar aquest desafiament, els professionals i investigadors estan utilitzant la Infraestructura com a Codi (Infrastructure as Code, IaC), que és un enfocament per a l'automatització de la infraestructura basat en pràctiques de desenvolupament de programari. L'objectiu de IaC és definir en un script totes les instruccions per a crear, actualitzar i executar recursos d'infraestructura. En aquest escenari, l'automatització de l'aprovisionament de la infraestructura accelera la pràctica del desplegament continu en el cicle de lliurament del programari. La computació en el núvol s'ha convertit en el principal model de pagament per ús utilitzat per professionals i investigadors per a aconseguir serveis en el núvol en un curt període de temps. En aquest escenari, les companyies estan passant de generar potència informàtica in-house cap a l'obtenció de recursos informàtics proveïts en el núvol a través d'internet com a serveis web. Al mateix temps, IaC i la computació en el núvol estan promovent alguns canvis en la indústria, per exemple, els equips d'operacions passen tot el seu temps treballant en programari, en lloc de configurar servidors i connectar cables de xarxa. Existeix una varietat d'eines IaC que utilitzen scripts per a definir i executar els recursos d'infraestructura en diferents proveïdors de serveis IaaS (Infrastructure as a Service). No obstant això, la diversitat de llenguatges de scripting de les eines IaC juntament amb l'heterogeneïtat del tipus d'infraestructura que ofereix cada proveïdor de serveis IaaS, han ocasionat que utilitzar scripts IaC per a l'aprovisionament d'infraestructura siga una activitat lenta i propensa a errors. L'objectiu del projecte de doctorat és proposar una solució a la diversitat dels llenguatges de scripting ¿de les eines IaC¿ i a l'heterogeneïtat del tipus d'infraestructura que ofereix cada proveïdor de serveis IaaS respecte a l'aprovisionament d'infraestructura en el núvol. Per a afrontar aquests desafiaments es proposa MoCIP (A Model-Driven Approach to Cloud Infrastructure Provisioning). MoCIP és un enfocament dirigit per models per a l'aprovisionament de la infraestructura en el núvol que suporta IaC mitjançant l'Enginyeria de Programari Dirigida per Models (Model-Driven Engineering, MDE). MoCIP utilitza els dos principis fonamentals de MDE: abstracció i automatització. En primer lloc, es va desenvolupar el llenguatge específic de domini ArgonML per a abstraure les capacitats de la computació en el núvol tals com còmput, elasticitat, emmagatzematge i xarxes. El llenguatge ArgonML permet modelar els recursos d'infraestructura del núvol. En segon lloc, es va desenvolupar l'eina ARGON per a automatitzar l'aprovisionament d'infraestructura en el núvol. ARGON realitza transformacions de model a model per a generar models que representen la infraestructura de diferents proveïdors de serveis IaaS. A més, ARGON realitza transformacions de model a text per a generar scripts IaC amb la informació subjacent de cada proveïdor de serveis IaaS i de cada eina d'aprovisionament d'infraestructura.[EN] DevOps (Development & Operations) is a new movement that encourages collaboration between developers and operation staff through a set of principles, practices, and tools in order to optimize the software delivery time. In particular, continuous deployment is a software practice, which is a great source of problems and generates a lot of attention when software artifacts are delivered late, or a critical defect comes to production. At the same time, continuous deployment is the border between developers and operation staff in the software delivery cycle. Consequently, it is recommended to start the DevOps implantation with continuous deployment practice. To address this challenge, practitioners and researchers are using Infrastructure as Code (IaC), which is an approach to infrastructure automation based on software development practices. The aim of IaC is to define in a script all the instructions to create, update and execute infrastructure resources. In this scenario, infrastructure provisioning automation accelerates the continuous deployment in the software delivery cycle. Cloud computing has become the principal pay-as-you-go model used by practitioners and researchers to obtain cloud services in a short period of time. In this scenario, companies are moving from generating in-house computing power into obtaining computing resources provided by cloud computing through the Internet as web services. At the same time, IaC and cloud computing are promoting some changes in the industry. For instance, operation teams spend all their time working on software, instead of racking servers and plugging in network cables. There is a diversity of IaC tools that use scripts to define and execute infrastructure resources in different IaaS (Infrastructure as a Service) service providers. However, the diversity of scripting languages of IaC tools along with the heterogeneity of the type of infrastructure offered by each IaaS service provider have caused that the use of IaC scripts for infrastructure provisioning to be a time-consuming and error-prone activity. The aim of the Ph.D. project is to propose a solution to the diversity of scripting languages of IaC tools along with the heterogeneity of the type of infrastructure offered by each IaaS service provider regarding the cloud infrastructure provisioning. To face these challenges, a Model-Driven Approach to Cloud Infrastructure Provisioning (MoCIP) is proposed. MoCIP is a model-driven approach to cloud infrastructure provisioning that supports IaC through Model-Driven Engineering (MDE). MoCIP uses the fundamental principles of MDE: abstraction and automation. First, the domain-specific language called ArgonML was developed to abstract the capabilities of cloud computing such as computing, elasticity, storage, and networking. The ArgonML language allows modeling the cloud infrastructure. Second, the ARGON tool automates the cloud infrastructure provisioning. ARGON performs model-to-model transformations to generate models that represent the infrastructure of different IaaS service providers. In addition, ARGON performs model-to-text transformations to generate IaC scripts with the underlying information of the IaaS service provider along with the infrastructure provisioning tool.Sandobalín Guamán, JC. (2020). MoCIP: Un enfoque dirigido por modelos para el aprovisionamiento de infraestructura en la nube [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/139077TESI