A cloud, microservice-based digital twin for the Oil industry a flow assurance case study

Digital Twins are one of the top recent technology trends, considered a front-runner in enabling the next generation of intelligent, digitalized industries. A Digital Twin is basi cally a real enterprise asset mirrored into the virtual world, created entirely in software, and utilized to innovate business, generate new revenue, and create value-producing op portunities. Naturally, building such demanding systems is not an easy task. Many organi zational and technological concerns should be addressed, like real-time processing, data management, cybersecurity, and low latency communication. Moreover, Digital Twins should be extensible and support data integration with its physical counterpart, allowing enterprises to streamline their operations more safely. Considering there are no standard methodologies or technologies to design a Digital Twin, software developers struggle to develop a robust, reliable Digital Twin architecture that meets customers’ needs. Therefore, in consideration of these technical challenges, we decided to explore solutions in the oil and gas industry. Oil plants are known for having notoriously complex pro cesses and an oppressive ecosystem when it comes to the adoption of new technologies, especially due to its intricate and ever-changing landscape. Despite these difficulties, this industry would certainly benefit from a well-designed Digital Twin. Hence, this work proposes a solution for an Oil Well Digital Twin architecture. We decided to use the flow assurance process of an Oil Well as a base to build a Digital Twin, aiming to study and fulfill those requirements and technical limitations. We implemented and tested several APIs, each one representing the virtual version of a real Oil Well component, enforcing the importance of microservices and cloud-native patterns in its design. When function ing together, these microservices comprise an entire Oil Well Digital twin solution, cov ering basic storage, communication, and monitoring fundamentals. This proof of concept demonstrates how a complex asset — like an Oil Well — can be built and organized in a completely digital manner. Lastly, this design is extensible and opens up further develop ment opportunities.Gêmeos Digitais são uma das principais tendências tecnológicas recentes, considerados pioneiros na habilitação da próxima geração de indústrias inteligentes e digitalizadas. Um Gêmeo Digital é basicamente um ativo corporativo espelhado no mundo virtual, criado inteiramente em software e destinado a inovar os modelos operacionais e de negócios, ao mesmo tempo em que fornece novas oportunidades de geração de receita e valor. Naturalmente, construir sistemas tão exigentes não é uma tarefa fácil. Muitas preocupações organizacionais e tecnológicas devem ser abordadas, como processamento em tempo real, gerenciamento de dados, segurança cibernética e comunicação de baixa latência. Além disso, os Gêmeos Digitais também devem ser extensíveis e oferecer suporte à integração de dados com sua contraparte física, permitindo que as empresas agilizem suas opera ções com mais segurança. Considerando que não existem metodologias ou tecnologias padrão para projetar um Gêmeo Digital, os desenvolvedores de software tem dificuldade em desenvolver uma arquitetura de Gêmeo Digital robusta e confiável que atenda às ne cessidades dos clientes. Além desses desafios técnicos, a indústria de óleo e gás é conhecida por ter processos notoriamente complexos e um ecossistema opressor na adoção de novas tecnologias, prin cipalmente devido ao seu cenário intrincado e em constante mudança. Apesar dessas dificuldades, essa indústria certamente se beneficiaria de um Gêmeo Digital bem proje tado. Assim, este trabalho propõe uma solução de uma arquitetura para um Gêmeo Digital de um Poço de Petróleo. Decidimos usar o processo de garantia de vazão de um poço de petróleo como base para construir um Gêmeo Digital, visando estudar e atender a esses requisitos e limitações técnicas. Implementamos e testamos várias APIs, cada uma repre sentando a versão virtual de um componente real de um poço de petróleo, reforçando a importância de microsserviços e padrões nativos de nuvem em seu design. Ao funciona rem juntos, esses microsserviços compreendem uma solução completa de gêmeos digitais de poços de petróleo, abrangendo os fundamentos básicos de armazenamento, comunica ção e monitoramento. Esta prova de conceito demonstra como um ativo complexo – como um poço de petróleo – pode ser construído e organizado de forma totalmente digital. Por fim, esse design é extensível e abre mais oportunidades de desenvolvimento

Service-Oriented Ad Hoc Grid Computing

Subject of this thesis are the design and implementation of an ad hoc Grid infrastructure. The vision of an ad hoc Grid further evolves conventional service-oriented Grid systems into a more robust, more flexible and more usable environment that is still standards compliant and interoperable with other Grid systems. A lot of work in current Grid middleware systems is focused on providing transparent access to high performance computing (HPC) resources (e.g. clusters) in virtual organizations spanning multiple institutions. The ad hoc Grid vision presented in this thesis exceeds this view in combining classical Grid components with more flexible components and usage models, allowing to form an environment combining dedicated HPC-resources with a large number of personal computers forming a "Desktop Grid". Three examples from medical research, media research and mechanical engineering are presented as application scenarios for a service-oriented ad hoc Grid infrastructure. These sample applications are also used to derive requirements for the runtime environment as well as development tools for such an ad hoc Grid environment. These requirements form the basis for the design and implementation of the Marburg ad hoc Grid Environment (MAGE) and the Grid Development Tools for Eclipse (GDT). MAGE is an implementation of a WSRF-compliant Grid middleware, that satisfies the criteria for an ad hoc Grid middleware presented in the introduction to this thesis. GDT extends the popular Eclipse integrated development environment by components that support application development both for traditional service-oriented Grid middleware systems as well as ad hoc Grid infrastructures such as MAGE. These development tools represent the first fully model driven approach to Grid service development integrated with infrastructure management components in service-oriented Grid computing. This thesis is concluded by a quantitative discussion of the performance overhead imposed by the presented extensions to a service-oriented Grid middleware as well as a discussion of the qualitative improvements gained by the overall solution. The conclusion of this thesis also gives an outlook on future developments and areas for further research. One of these qualitative improvements is "hot deployment" the ability to install and remove Grid services in a running node without interrupt to other active services on the same node. Hot deployment has been introduced as a novelty in service-oriented Grid systems as a result of the research conducted for this thesis. It extends service-oriented Grid computing with a new paradigm, making installation of individual application components a functional aspect of the application. This thesis further explores the idea of using peer-to-peer (P2P networking for Grid computing by combining a general purpose P2P framework with a standard compliant Grid middleware. In previous work the application of P2P systems has been limited to replica location and use of P2P index structures for discovery purposes. The work presented in this thesis also uses P2P networking to realize seamless communication accross network barriers. Even though the web service standards have been designed for the internet, the two-way communication requirement introduced by the WSRF-standards and particularly the notification pattern is not well supported by the web service standards. This defficiency can be answered by mechanisms that are part of such general purpose P2P communication frameworks. Existing security infrastructures for Grid systems focus on protection of data during transmission and access control to individual resources or the overall Grid environment. This thesis focuses on security issues within a single node of a dynamically changing service-oriented Grid environment. To counter the security threads arising from the new capabilities of an ad hoc Grid, a number of novel isolation solutions are presented. These solutions address security issues and isolation on a fine-grained level providing a range of applicable basic mechanisms for isolation, ranging from lightweight system call interposition to complete para-virtualization of the operating systems

API diversity for microservices in the domain of connected vehicles

Web services in the domain of connected vehicles are subject to various requirements including high availability and large workloads. Microservices are an architectural style which can fulfill those requirements by fostering the independence and decoupling of software components as reusable services. To achieve this independence, microservices have to implement all aspects of providing the services themselves, including different API technologies for heterogeneous consumers and supporting features like authentication. In this work, we examine the use of a service proxy that externalizes these concerns into a sidecar that provides multiple APIs and common service functionality in a platform-independent manner. We look at how different kinds of API styles and technologies solve selected classes of problems and how we can translate between API technologies. We design and implement a framework for building gateways that enables the creation and composition of reusable components, in the fashion of Lego bricks, to maximize flexibility, while reducing the effort for building gateway components. We design and implement selected components of common and reusable API functionality enabling us to build a reference setup with a service proxy as a sidecar using our framework. Finally, we evaluate the proposed solution to identify benefits and drawbacks of the approach of using our framework as a service proxy. We conclude that the examined approach provides benefits for the development of many polyglot microservices, but splitting one service into two components adds additional complexity that has to be managed.Web Services für vernetzte Fahrzeuge unterliegen unterschiedlichen Anforderungen, unter anderem einer hohen Verfügbarkeit und einem großen Datendurchsatz. Microservices sind ein Architekturstil, der diesen Anforderungen gerecht werden kann, indem er die Unabhängigkeit und Entkopplung von Softwarekomponenten als wiederverwendbare Services fördert. Zum Erreichen der Unabhängigkeit implementieren Microservices alle Aspekte der Servicebereitstellung eigenständig. Dazu gehört verschiedene API Technologien für heterogene Clients bereitzustellen und unterstützende Funktionalität wie Authentifizierung zu implementieren. In dieser Arbeit wird die Verwendung einer Proxy Komponente vor einem Service untersucht, durch welche die Bereitstellung verschiedener API Technologien und allgemeiner unterstützender Funktionalität aus dem Service extrahiert wird. Die Lösungen verschiedener API Technologien und Stile für ausgewählte Klassen an Problemen werden verglichen und mögliche Umwandlungen der verschiedenen API Technologien werden untersucht. Es wird ein Framework konzeptioniert und implementiert, das die Erstellung von Gateways durch Kombination von wiederverwendbaren Komponenten, wie das Zusammensetzen von Legosteinen, ermöglicht. Dieses Framework sorgt für eine hohe Flexibilität, während es den Aufwand bei der Erstellung von Gateways gering hält. Es werden ausgewählte wiederverwendbare Komponenten entworfen, um eine Referenzimplementierung des Ansatzes umzusetzen, bei der allgemeine Funktionalität in einen parallel laufenden Proxy ausgelagert wird. Dieser Ansatz wird evaluiert, indem Vor- und Nachteile anhand eines mit dem Framework erstellten Proxys identifiziert werden. Das Fazit dieser Arbeit ist, dass dieser Ansatz bei Systemen mit vielen Microservices mit unterschiedlichen Programmiersprachen Vorteile bringt, aber die Trennung eines Services in zwei Komponenten eine nicht unerhebliche Komplexität einführt

Performance Test Suite for MIT Kerberos

Tato práce se zaměřuje na vyvinutí nástrojů pro výkonnostní testování, které umožní otestovat infrastrukturu systému MIT Kerberos, zjistit její výkonnostní charakteristiky a detekovat potenciální problémy. Práce shrnuje teoretické základy protokolu Kerberos a analyzuje potenciální výkonnostní problémy v různých konfiguracích MIT Kerberosu. Dále práce obsahuje popis návrhu a implementace sady nástrojů pro distribuované testování. Pomocí implementovaných nástrojů bylo odhaleno několik výkonnostních problémů, které jsou v práci popsány spolu s návrhem jejich řešení.The aim of this thesis is to develop performance test suite, which will enable to test MIT Kerberos system infrastructure, assess gained performance characteristics and detect potential bottlenecks. This thesis summarizes necessary theoretical background of Kerberos protocol. Potential performance problems are analyzed on different MIT Kerberos configurations. This thesis describes distributed test suite design and implementation. Several performance problems were discovered using this test suite. These problems are described and some solutions are proposed.

An Autonomic Cross-Platform Operating Environment for On-Demand Internet Computing

The Internet has evolved into a global and ubiquitous communication medium interconnecting powerful application servers, diverse desktop computers and mobile notebooks. Along with recent developments in computer technology, such as the convergence of computing and communication devices, the way how people use computers and the Internet has changed people´s working habits and has led to new application scenarios. On the one hand, pervasive computing, ubiquitous computing and nomadic computing become more and more important since different computing devices like PDAs and notebooks may be used concurrently and alternately, e.g. while the user is on the move. On the other hand, the ubiquitous availability and pervasive interconnection of computing systems have fostered various trends towards the dynamic utilization and spontaneous collaboration of available remote computing resources, which are addressed by approaches like utility computing, grid computing, cloud computing and public computing. From a general point of view, the common objective of this development is the use of Internet applications on demand, i.e. applications that are not installed in advance by a platform administrator but are dynamically deployed and run as they are requested by the application user. The heterogeneous and unmanaged nature of the Internet represents a major challenge for the on demand use of custom Internet applications across heterogeneous hardware platforms, operating systems and network environments. Promising remedies are autonomic computing systems that are supposed to maintain themselves without particular user or application intervention. In this thesis, an Autonomic Cross-Platform Operating Environment (ACOE) is presented that supports On Demand Internet Computing (ODIC), such as dynamic application composition and ad hoc execution migration. The approach is based on an integration middleware called crossware that does not replace existing middleware but operates as a self-managing mediator between diverse application requirements and heterogeneous platform configurations. A Java implementation of the Crossware Development Kit (XDK) is presented, followed by the description of the On Demand Internet Computing System (ODIX). The feasibility of the approach is shown by the implementation of an Internet Application Workbench, an Internet Application Factory and an Internet Peer Federation. They illustrate the use of ODIX to support local, remote and distributed ODIC, respectively. Finally, the suitability of the approach is discussed with respect to the support of ODIC

Integrating heterogeneous open-source software into web browsers using AMICO:WEB

A web browser provides a uniform user interface to different types of information. Making this interface universally accessible and more interactive is a long term goal still far from being achieved. Universally accessible browsers require novel interaction modalities and additional functionalities, for which existing browsers tend to provide only partial solutions. Although functionality for web accessibility can be found as open-source and free software components, their reuse and integration is complex because they were developed in diverse implementation environments, following standards and conventions incompatible with the web. To enable the integration of existing partial solutions within a mainstream web browser environment, we have developed a middleware infrastructure, AMICO:WEB. This enables browser access to a wide variety of open source and free software components. The main contribution of AMICO:WEB is in enabling the syntactic interoperability between web extension mechanisms and a variety of integration mechanisms used by open-source and free software components. It also bridges the semantic differences between the high-level world of web XML-based APIs and the low-level APIs of the device-oriented world. We discuss the design decisions made during the development of AMICO:WEB in the context of web accessibility, using two typical usage scenarios: one describing a disabled user using a mainstream web browser with additional interaction modalities; another describing a non-disabled user browsing in a suboptimal interaction situation

VISOR: virtual machine images management service for cloud infarestructures

Cloud Computing is a relatively novel paradigm that aims to fulfill the computing as utility dream. It has appeared to bring the possibility of providing computing resources (such as servers, storage and networks) as a service and on demand, making them accessible through common Internet protocols. Through cloud offers, users only need to pay for the amount of resources they need and for the time they use them. Virtualization is the clouds key technology, acting upon virtual machine images to deliver fully functional virtual machine instances. Therefore, virtual machine images play an important role in Cloud Computing and their efficient management becomes a key concern that should be carefully addressed. To tackle this requirement, most cloud offers provide their own image repository, where images are stored and retrieved from, in order to instantiate new virtual machines. However, the rise of Cloud Computing has brought new problems in managing large collections of images. Existing image repositories are not able to efficiently manage, store and catalogue virtual machine images from other clouds through the same centralized service repository. This becomes especially important when considering the management of multiple heterogeneous cloud offers. In fact, despite the hype around Cloud Computing, there are still existing barriers to its widespread adoption. Among them, clouds interoperability is one of the most notable issues. Interoperability limitations arise from the fact that current cloud offers provide proprietary interfaces, and their services are tied to their own requirements. Therefore, when dealing with multiple heterogeneous clouds, users face hard to manage integration and compatibility issues. The management and delivery of virtual machine images across different clouds is an example of such interoperability constraints. This dissertation presents VISOR, a cloud agnostic virtual machine images management service and repository. Our work towards VISOR aims to provide a service not designed to fit in a specific cloud offer but rather to overreach sharing and interoperability limitations among different clouds. With VISOR, the management of clouds interoperability can be seamlessly abstracted from the underlying procedures details. In this way, it aims to provide users with the ability to manage and expose virtual machine images across heterogeneous clouds, throughout the same generic and centralized repository and management service. VISOR is an open source software with a community-driven development process, thus it can be freely customized and further improved by everyone. The conducted tests to evaluate its performance and resources usage rate have shown VISOR as a stable and high performance service, even when compared with other services already in production. Lastly, placing clouds as the main target audience is not a limitation for other use cases. In fact, virtualization and virtual machine images are not exclusively linked to cloud environments. Therefore and given the service agnostic design concerns, it is possible to adapt it to other usage scenarios as well.A Computação em Nuvem (”Cloud Computing”) é um paradigma relativamente novo que visa cumprir o sonho de fornecer a computação como um serviço. O mesmo surgiu para possibilitar o fornecimento de recursos de computação (servidores, armazenamento e redes) como um serviço de acordo com as necessidades dos utilizadores, tornando-os acessíveis através de protocolos de Internet comuns. Através das ofertas de ”cloud”, os utilizadores apenas pagam pela quantidade de recursos que precisam e pelo tempo que os usam. A virtualização é a tecnologia chave das ”clouds”, atuando sobre imagens de máquinas virtuais de forma a gerar máquinas virtuais totalmente funcionais. Sendo assim, as imagens de máquinas virtuais desempenham um papel fundamental no ”Cloud Computing” e a sua gestão eficiente torna-se um requisito que deve ser cuidadosamente analisado. Para fazer face a tal necessidade, a maioria das ofertas de ”cloud” fornece o seu próprio repositório de imagens, onde as mesmas são armazenadas e de onde são copiadas a fim de criar novas máquinas virtuais. Contudo, com o crescimento do ”Cloud Computing” surgiram novos problemas na gestão de grandes conjuntos de imagens. Os repositórios existentes não são capazes de gerir, armazenar e catalogar images de máquinas virtuais de forma eficiente a partir de outras ”clouds”, mantendo um único repositório e serviço centralizado. Esta necessidade torna-se especialmente importante quando se considera a gestão de múltiplas ”clouds” heterogéneas. Na verdade, apesar da promoção extrema do ”Cloud Computing”, ainda existem barreiras à sua adoção generalizada. Entre elas, a interoperabilidade entre ”clouds” é um dos constrangimentos mais notáveis. As limitações de interoperabilidade surgem do fato de as ofertas de ”cloud” atuais possuírem interfaces proprietárias, e de os seus serviços estarem vinculados às suas próprias necessidades. Os utilizadores enfrentam assim problemas de compatibilidade e integração difíceis de gerir, ao lidar com ”clouds” de diferentes fornecedores. A gestão e disponibilização de imagens de máquinas virtuais entre diferentes ”clouds” é um exemplo de tais restrições de interoperabilidade. Esta dissertação apresenta o VISOR, o qual é um repositório e serviço de gestão de imagens de máquinas virtuais genérico. O nosso trabalho em torno do VISOR visa proporcionar um serviço que não foi concebido para lidar com uma ”cloud” específica, mas sim para superar as limitações de interoperabilidade entre ”clouds”. Com o VISOR, a gestão da interoperabilidade entre ”clouds” é abstraída dos detalhes subjacentes. Desta forma pretende-se proporcionar aos utilizadores a capacidade de gerir e expor imagens entre ”clouds” heterogéneas, mantendo um repositório e serviço de gestão centralizados. O VISOR é um software de código livre com um processo de desenvolvimento aberto. O mesmo pode ser livremente personalizado e melhorado por qualquer pessoa. Os testes realizados para avaliar o seu desempenho e a taxa de utilização de recursos mostraram o VISOR como sendo um serviço estável e de alto desempenho, mesmo quando comparado com outros serviços já em utilização. Por fim, colocar as ”clouds” como principal público-alvo não representa uma limitação para outros tipos de utilização. Na verdade, as imagens de máquinas virtuais e a virtualização não estão exclusivamente ligadas a ambientes de ”cloud”. Assim sendo, e tendo em conta as preocupações tidas no desenho de um serviço genérico, também é possível adaptar o nosso serviço a outros cenários de utilização