Segurança de contentores em ambiente de desenvolvimento contínuo

The rising of the DevOps movement and the transition from a product economy to a service economy drove significant changes in the software development life cycle paradigm, among which the dropping of the waterfall in favor of agile methods. Since DevOps is itself an agile method, it allows us to monitor current releases, receiving constant feedback from clients, and improving the next software releases. Despite its extraordinary development, DevOps still presents limitations concerning security, which needs to be included in the Continuous Integration or Continuous Deployment pipelines (CI/CD) used in software development. The massive adoption of cloud services and open-source software, the widely spread containers and related orchestration, as well as microservice architectures, broke all conventional models of software development. Due to these new technologies, packaging and shipping new software is done in short periods nowadays and becomes almost instantly available to users worldwide. The usual approach to attach security at the end of the software development life cycle (SDLC) is now becoming obsolete, thus pushing the adoption of DevSecOps or SecDevOps, by injecting security into SDLC processes earlier and preventing security defects or issues from entering into production. This dissertation aims to reduce the impact of microservices’ vulnerabilities by examining the respective images and containers through a flexible and adaptable set of analysis tools running in dedicated CI/CD pipelines. This approach intends to provide a clean and secure collection of microservices for later release in cloud production environments. To achieve this purpose, we have developed a solution that allows programming and orchestrating a battery of tests. There is a form where we can select several security analysis tools, and the solution performs this set of tests in a controlled way according to the defined dependencies. To demonstrate the solution’s effectiveness, we program a battery of tests for different scenarios, defining the security analysis pipeline to incorporate various tools. Finally, we will show security tools working locally, which subsequently integrated into our solution return the same results.A ascensão da estratégia DevOps e a transição de uma economia de produto para uma economia de serviços conduziu a mudanças significativas no paradigma do ciclo de vida do desenvolvimento de software, entre as quais o abandono do modelo em cascata em favor de métodos ágeis. Uma vez que o DevOps é parte integrante de um método ágil, permite-nos monitorizar as versões actuais, recebendo feedback constante dos clientes, e melhorando as próximas versões de software. Apesar do seu extraordinário desenvolvimento, o DevOps ainda apresenta limitações relativas à segurança, que necessita de ser incluída nas pipelines de integração contínua ou implantação contínua (CI/CD) utilizadas no desenvolvimento de software. A adopção em massa de serviços na nuvem e software aberto, a ampla difusão de contentores e respectiva orquestração bem como das arquitecturas de micro-serviços, quebraram assim todos os modelos convencionais de desenvolvimento de software. Devido a estas novas tecnologias, a preparação e expedição de novo software é hoje em dia feita em curtos períodos temporais e ficando disponível quase instantaneamente a utilizadores em todo o mundo. Face a estes fatores, a abordagem habitual que adiciona segurança ao final do ciclo de vida do desenvolvimento de software está a tornar-se obsoleta, sendo crucial adotar metodologias DevSecOps ou SecDevOps, injetando a segurança mais cedo nos processos de desenvolvimento de software e impedindo que defeitos ou problemas de segurança fluam para os ambientes de produção. O objectivo desta dissertação é reduzir o impacto de vulnerabilidades em micro-serviços através do exame das respectivas imagens e contentores por um conjunto flexível e adaptável de ferramentas de análise que funcionam em pipelines CI/CD dedicadas. Esta abordagem pretende fornecer uma coleção limpa e segura de micro-serviços para posteriormente serem lançados em ambientes de produção na nuvem. Para atingir este objectivo, desenvolvemos uma solução que permite programar e orquestrar uma bateria de testes. Existe um formulário onde podemos seleccionar várias ferramentas de análise de segurança, e a solução executa este conjunto de testes de uma forma controlada de acordo com as dependências definidas. Para demonstrar a eficácia da solução, programamos um conjunto de testes para diferentes cenários, definindo as pipelines de análise de segurança para incorporar várias ferramentas. Finalmente, mostraremos ferramentas de segurança a funcionar localmente, que posteriormente integradas na nossa solução devolvem os mesmos resultados.Mestrado em Engenharia Informátic

Live-Migration in Cloud Computing Environment

O tráfego global de IP aumentou cinco vezes nos últimos cinco anos, e prevê-se que crescerá três vezes nos próximos cinco. Já para o período de 2013 a 2018, anteviu-se que o total do tráfego de IP iria aumentar a sua taxa composta de crescimento anual (CAGR) em, aproximadamente, 3.9 vezes. Assim, os Prestadores de Serviços estão a sofrer com este acréscimo exponencial, que é proveniente do número abismal de dispositivos e utilizadores que estão ligados à Internet, bem como das suas exigências por vários recursos e serviços de rede (como por exemplo, distribuição de conteúdo multimédia, segurança, mobilidade, etc.). Mais especificamente, estes estão com dificuldades em: introduzir novos serviços geradores de receitas; e otimizar e adaptar as suas infraestruturas mais caras, centros de processamento de dados, e redes empresariais e de longa distância (COMpuTIN, 2015). Estas redes continuam a ter sérios problemas (no que toca a agilidade, gestão, mobilidade e no tempo despendido para se adaptarem), que não foram corrigidos até ao momento. Portanto, foram propostos novos modelos de Virtualização de Funções da Rede (NFV) e tecnologias de Redes de Software Definidos (SDN) para solucionar gastos operacionais e de capital não otimizado, e limitações das redes (Lopez, 2014, Hakiri and Berthou, 2015). Para se ultrapassar tais adversidades, o Instituto Europeu de Normas de Telecomunicações (ETSI) e outras organizações propuseram novas arquiteturas de rede. De acordo com o ETSI, a NFV é uma técnica emergente e poderosa, com grande aplicabilidade, e com o objetivo de transformar a maneira como os operadores desenham as redes. Isto é alcançado pela evolução da tecnologia padrão de virtualização TI, de forma a consolidar vários tipos de equipamentos de redes como: servidores de grande volume, routers, switches e armazenamento (Xilouris et al., 2014). Nesta dissertação, foram usadas as soluções mais atuais de SDN e NFV, de forma a produzir um caso de uso que possa solucionar o crescimento do tráfego de rede e a excedência da sua capacidade máxima. Para o desenvolvimento e avalização da solução, foi instalada a plataforma de computação na nuvem OpenStack, de modo a implementar, gerir e testar um caso de uso de Live Migration.Global IP traffic has increased fivefold over the past five years, and will continue increasing threefold over the next five years. The overall IP traffic will grow at a compound annual growth rate (CAGR) nearly 3.9-fold from 2013 to 2018. Service Providers are experiencing the exponential growth of IP traffic that comes from the incredible increased number of devices and users who are connected to the internet along with their demands for various resources and network services like multimedia content distribution, security, mobility and else. Therefore, Service Providers are finding difficult to introduce new revenue generating services, optimize and adapt their expensive infrastructures, data centers, wide-area networks and enterprise networks (COMpuTIN, 2015). The networks continue to have serious known problems, such as, agility, manageability, mobility and time-to-application that have not been successfully addressed so far. Thus, novel Network Function Virtualization (NFV) models and Software-defined Networking (SDN) technologies have been proposed to solve the non-optimal capital and operational expenditures and network’s limitations (Lopez, 2014, Hakiri and Berthou, 2015). In order to solve these issues, the European Telecommunications Standards Institute (ETSI) and other standard organizations are proposing new network architecture approaches. According to ETSI, The Network Functions Virtualization is a powerful emerging technique with widespread applicability, aiming to transform the way that network operators design networks by evolving standard IT virtualization technology to consolidate many network equipment types: high volume servers, routers, switches and storage (Xilouris et al., 2014). In this thesis, the current Software-Defined Networking (SDN) and Network Function Virtualization (NFV) solutions were used in order to make a use case that can address the increasing of network traffic and exceeding its maximum capacity. To develop and evaluate the solution, OpenStack cloud computing platform was installed in order to deploy, manage and test a Live-Migration use-case

Orchestration of distributed ingestion and processing of IoT data for fog platforms

In recent years there has been an extraordinary growth of the Internet of Things (IoT) and its protocols. The increasing diffusion of electronic devices with identification, computing and communication capabilities is laying ground for the emergence of a highly distributed service and networking environment. The above mentioned situation implies that there is an increasing demand for advanced IoT data management and processing platforms. Such platforms require support for multiple protocols at the edge for extended connectivity with the objects, but also need to exhibit uniform internal data organization and advanced data processing capabilities to fulfill the demands of the application and services that consume IoT data. One of the initial approaches to address this demand is the integration between IoT and the Cloud computing paradigm. There are many benefits of integrating IoT with Cloud computing. The IoT generates massive amounts of data, and Cloud computing provides a pathway for that data to travel to its destination. But today’s Cloud computing models do not quite fit for the volume, variety, and velocity of data that the IoT generates. Among the new technologies emerging around the Internet of Things to provide a new whole scenario, the Fog Computing paradigm has become the most relevant. Fog computing was introduced a few years ago in response to challenges posed by many IoT applications, including requirements such as very low latency, real-time operation, large geo-distribution, and mobility. Also this low latency, geo-distributed and mobility environments are covered by the network architecture MEC (Mobile Edge Computing) that provides an IT service environment and Cloud-computing capabilities at the edge of the mobile network, within the Radio Access Network (RAN) and in close proximity to mobile subscribers. Fog computing addresses use cases with requirements far beyond Cloud-only solution capabilities. The interplay between Cloud and Fog computing is crucial for the evolution of the so-called IoT, but the reach and specification of such interplay is an open problem. This thesis aims to find the right techniques and design decisions to build a scalable distributed system for the IoT under the Fog Computing paradigm to ingest and process data. The final goal is to explore the trade-offs and challenges in the design of a solution from Edge to Cloud to address opportunities that current and future technologies will bring in an integrated way. This thesis describes an architectural approach that addresses some of the technical challenges behind the convergence between IoT, Cloud and Fog with special focus on bridging the gap between Cloud and Fog. To that end, new models and techniques are introduced in order to explore solutions for IoT environments. This thesis contributes to the architectural proposals for IoT ingestion and data processing by 1) proposing the characterization of a platform for hosting IoT workloads in the Cloud providing multi-tenant data stream processing capabilities, the interfaces over an advanced data-centric technology, including the building of a state-of-the-art infrastructure to evaluate the performance and to validate the proposed solution. 2) studying an architectural approach following the Fog paradigm that addresses some of the technical challenges found in the first contribution. The idea is to study an extension of the model that addresses some of the central challenges behind the converge of Fog and IoT. 3) Design a distributed and scalable platform to perform IoT operations in a moving data environment. The idea after study data processing in Cloud, and after study the convenience of the Fog paradigm to solve the IoT close to the Edge challenges, is to define the protocols, the interfaces and the data management to solve the ingestion and processing of data in a distributed and orchestrated manner for the Fog Computing paradigm for IoT in a moving data environment.En els últims anys hi ha hagut un gran creixement del Internet of Things (IoT) i els seus protocols. La creixent difusió de dispositius electrònics amb capacitats d'identificació, computació i comunicació esta establint les bases de l’aparició de serveis altament distribuïts i del seu entorn de xarxa. L’esmentada situació implica que hi ha una creixent demanda de plataformes de processament i gestió avançada de dades per IoT. Aquestes plataformes requereixen suport per a múltiples protocols al Edge per connectivitat amb el objectes, però també necessiten d’una organització de dades interna i capacitats avançades de processament de dades per satisfer les demandes de les aplicacions i els serveis que consumeixen dades IoT. Una de les aproximacions inicials per abordar aquesta demanda és la integració entre IoT i el paradigma del Cloud computing. Hi ha molts avantatges d'integrar IoT amb el Cloud. IoT genera quantitats massives de dades i el Cloud proporciona una via perquè aquestes dades viatgin a la seva destinació. Però els models actuals del Cloud no s'ajusten del tot al volum, varietat i velocitat de les dades que genera l'IoT. Entre les noves tecnologies que sorgeixen al voltant del IoT per proporcionar un escenari nou, el paradigma del Fog Computing s'ha convertit en la més rellevant. Fog Computing es va introduir fa uns anys com a resposta als desafiaments que plantegen moltes aplicacions IoT, incloent requisits com baixa latència, operacions en temps real, distribució geogràfica extensa i mobilitat. També aquest entorn està cobert per l'arquitectura de xarxa MEC (Mobile Edge Computing) que proporciona serveis de TI i capacitats Cloud al edge per la xarxa mòbil dins la Radio Access Network (RAN) i a prop dels subscriptors mòbils. El Fog aborda casos d?us amb requisits que van més enllà de les capacitats de solucions només Cloud. La interacció entre Cloud i Fog és crucial per a l'evolució de l'anomenat IoT, però l'abast i especificació d'aquesta interacció és un problema obert. Aquesta tesi té com objectiu trobar les decisions de disseny i les tècniques adequades per construir un sistema distribuït escalable per IoT sota el paradigma del Fog Computing per a ingerir i processar dades. L'objectiu final és explorar els avantatges/desavantatges i els desafiaments en el disseny d'una solució des del Edge al Cloud per abordar les oportunitats que les tecnologies actuals i futures portaran d'una manera integrada. Aquesta tesi descriu un enfocament arquitectònic que aborda alguns dels reptes tècnics que hi ha darrere de la convergència entre IoT, Cloud i Fog amb especial atenció a reduir la bretxa entre el Cloud i el Fog. Amb aquesta finalitat, s'introdueixen nous models i tècniques per explorar solucions per entorns IoT. Aquesta tesi contribueix a les propostes arquitectòniques per a la ingesta i el processament de dades IoT mitjançant 1) proposant la caracterització d'una plataforma per a l'allotjament de workloads IoT en el Cloud que proporcioni capacitats de processament de flux de dades multi-tenant, les interfícies a través d'una tecnologia centrada en dades incloent la construcció d'una infraestructura avançada per avaluar el rendiment i validar la solució proposada. 2) estudiar un enfocament arquitectònic seguint el paradigma Fog que aborda alguns dels reptes tècnics que es troben en la primera contribució. La idea és estudiar una extensió del model que abordi alguns dels reptes centrals que hi ha darrere de la convergència de Fog i IoT. 3) Dissenyar una plataforma distribuïda i escalable per a realitzar operacions IoT en un entorn de dades en moviment. La idea després d'estudiar el processament de dades a Cloud, i després d'estudiar la conveniència del paradigma Fog per resoldre el IoT prop dels desafiaments Edge, és definir els protocols, les interfícies i la gestió de dades per resoldre la ingestió i processament de dades en un distribuït i orquestrat per al paradigma Fog Computing per a l'IoT en un entorn de dades en moviment

Autonomous Incident Response

Trabalho de Projeto de Mestrado, Segurança Informática, 2022, Universidade de Lisboa, Faculdade de CiênciasInformation security is a must-have for any organization willing to stay relevant and grow, it plays an important role as a business enabler, be it from a regulatory perspective or a reputation perspective. Having people, process, and technology to solve the ever growing number of security incidents as fast as possible and with the least amount of impact is a challenge for small and big companies. To address this challenge, companies started investing in Security Orchestration, Automation, and Response (SOAR) [39, 68, 70]. Security orchestration is the planning, integration, cooperation, and coordination of the activities of security tools and experts to produce and automate required actions in response to any security incident across multiple technology paradigms [40]. In other words, the use of SOAR is a way to translate the manual procedures followed by the security analysts into automated actions, making the process faster and scalable while saving on human resources budget. This project proposes a low-cost cloud native SOAR platform that is based on serverless computing, presenting the underlying details of its design. The performance of the proposed solution was evaluated through 364 real-world incidents related to 11 use cases in a large multinational enterprise. The results show that the solution is able to decrease the duration of the tasks by an average of 98.81% while having an operating expense of less than $65/month. Prior to the SOAR, it took the analyst 75.84 hours to perform manual tasks related to the 11 use cases. Additionally, an estimated 450 hours of the analyst’s time would be used to run the Update threat intelligence database use case. After the SOAR, the same tasks were automatically ran in 31.2 minutes and the Update threat intelligence database use case ran 9.000 times in 5.3 hours