Distributed Transactions using the SAGA pattern

Οι κατανεμημένες συναλλαγές αποτελούν κρίσιμο συστατικό των σύγχρονων κατανεμημένων συστημάτων, καθώς εξασφαλίζουν τη συνοχή και την ακεραιότητα των δεδομένων σε πολλαπλές βάσεις δεδομένων, διακομιστές ή υπηρεσίες. Είναι ιδιαίτερα σημαντικές στα σύγχρονα κατανεμημένα συστήματα, τα οποία είναι συχνά πολύπλοκα, δυναμικά με πολλά διασυνδεδεμένα στοιχεία, καθώς και σε συστήματα που περιλαμβάνουν ευαίσθητα ή κρίσιμα δεδομένα. Χωρίς κατανεμημένες συναλλαγές, θα ήταν δύσκολο να διασφαλιστεί ότι τα δεδομένα ενημερώνονται με συνέπεια και ορθότητα σε πολλαπλά συστήματα, γεγονός που θα μπορούσε να οδηγήσει σε ασυνέπειες και απώλεια δεδομένων. Ο στόχος μιας κατανεμημένης συναλλαγής είναι να διασφαλίσει ότι όλες οι λειτουργίες είτε όλες δεσμεύονται και τίθενται σε ισχύ είτε όλες ανακαλούνται και δεν έχουν κανένα αποτέλεσμα, ακόμη και σε περίπτωση αποτυχιών ή σφαλμάτων. Ωστόσο, οι παραδοσιακές προσεγγίσεις για την υλοποίηση κατανεμημένων συναλλαγών, όπως το Two-Phase Commit protocol (2PC), μπορεί να είναι πολύπλοκες και επιρρεπείς σε σφάλματα. Το πρότυπο SAGA είναι μια πολλά υποσχόμενη εναλλακτική προσέγγιση για την υλοποίηση κατανεμημένων συναλλαγών, καθώς επιτρέπει μεγαλύτερη ευελιξία και ανθεκτικότητα στα κατανεμημένα συστήματα. Σε ένα SAGA, κάθε βήμα αντιμετωπίζεται ως ξεχωριστή συναλλαγή, και εάν κάποιο βήμα αποτύχει, η διαδικασία ανατρέπεται σε μια προηγούμενη γνωστή καλή κατάσταση και ενεργοποιείται μια διαδικασία χειρισμού σφαλμάτων. Αυτό επιτρέπει τη μερική αποτυχία και τη δυνατότητα ανάκαμψης από αυτήν, καθιστώντας το πρότυπο SAGA ιδιαίτερα κατάλληλο για χρήση σε σύγχρονα κατανεμημένα συστήματα. Σε αυτή τη διπλωματική εργασία, θα διερευνήσουμε τη χρήση του προτύπου Saga για την υλοποίηση κατανεμημένων συναλλαγών σε κατανεμημένα συστήματα. Θα ξεκινήσουμε παρέχοντας μια επισκόπηση των κατανεμημένων συναλλαγών και των προκλήσεων που θέτουν στα σύγχρονα κατανεμημένα συστήματα. Στη συνέχεια θα παρουσιάσουμε το πρότυπο SAGA και θα συζητήσουμε τα οφέλη και τις προκλήσεις του. Αμέσως μετά, θα παρουσιάσουμε μια μελέτη περίπτωσης που υλοποιεί την «orchestrated» προσέγγισή μας - piSaga - για κατανεμημένες συναλλαγές σε Spring Boot microservices και θα πραγματοποιήσουμε ορισμένα πειράματα προκειμένου να μετρήσουμε την απόδοσή της σε σχέση με μια «choreographed» λύση. Τέλος, θα καταλήξουμε συνοψίζοντας τις βασικές ιδέες και τις συνεισφορές της ανάλυσής μας.Distributed transactions are a crucial component of modern distributed systems, as they ensure data consistency and integrity across multiple databases, servers, or services. They are essential in modern distributed systems, which are often complex and dynamic environments with many interconnected components, and in systems that involve sensitive or critical data. Without distributed transactions, ensuring that data is consistently and correctly updated across multiple systems would be challenging, which could lead to inconsistencies and data loss. The goal of a distributed transaction is to ensure that all operations either commit and take effect or roll back and have no effect, even in the case of failures or errors. However, traditional approaches to implementing distributed transactions, such as the Two-Phase commit (2PC) protocol, can be inflexible and prone to failure in complex and dynamic environments. The SAGA pattern is a promising alternative approach to implementing distributed transactions, as it allows for more flexibility and resilience in distributed systems. In a SAGA, each step in a long-running business process is treated as a separate transaction, and if any step fails, the process is rolled back to a previously known good state, and an error-handling process is triggered. This allows for partial failures and the ability to recover from them, making the SAGA pattern particularly well-suited for use in modern distributed systems. In this thesis, we will explore the use of the SAGA pattern for implementing distributed transactions in distributed systems. We will begin by providing an overview of distributed transactions and their challenges in modern distributed systems. We will then introduce the SAGA pattern and discuss its benefits and challenges. Next, we will present a case study that implements our orchestrated approach - piSaga - for distributed transactions in Spring Boot microservices. We will also conduct experiments to measure its performance against a choreographed solution. Finally, we will summarize the key insights and contributions of our analysis

Coordination between multiple microservices: a systematic mapping study

Mikropalveluarkkitehtuurin suosio on kasvanut huomattavisti viimeisen kymmenen vuoden aikana sen tarjoamien hyötyjen takia, jotka ovat osittain seurausta palveluiden vähentyneestä riippuvuudesta toisiinsa. Riippumattomuutta lisää esimerkiksi hajautettu tiedonhallinta, jonka mukaan jokaisen palvelun tulisi olla vastuussa omistamansa tiedon hallinasta käyttäen sopivinta tietokantateknologiaa. Vaikka tällä voidaan saavuttaa useita etuja, aiheuttaa se myös uusia ongelmia etenkin tiedon yhtenäisyyden hallinnassa kun usean palvelun hallitsemaa tietoa täytyy muokata yhteistyössä. Tämä ongelma voitaisiin välttää käyttämällä yhteistä tietokantaa palveluiden välillä, mutta se osittain poistaisi mikropalvelun hyödyt tuomalla lisää riippuvuusuhteita mikropalveluiden välille. Tästä syystä on tärkeää tarkastella muita vaihtoehtoja hajautetun tiedon hallintaan siten, että mikropalvelun hyötyjä on mahdollista ylläpitää. Tässä työssä toteutetaan systemaattinen kirjallisuuskartoitus, jonka tavoitteena on löytää sopivia malleja usean mikropalvelun väliseen koordinointiin. Aluksi työssä tunnistetaan koordinointimallit, joista käydään paljon keskustelua kirjallisuudessa. Tämän jälkeen jokaisesta valitusta mallista keskustellaan käyttäen yhteistä keskustelukaavaa, joka sisältää mallin määrittelyn sekä hyötyjen ja haittojen listaamisen. Kirjallisuuskartoituksessa saatujen tulosten perusteella huomattiin, että mikropalveluarkkitehtuurissa suositaan malleja jotka tarjoavat lopulta yhtenäistä tulosta (eng. eventual consistent). Tämä eroaa huomattavasti perinteisistä ohjelmistoista, joissa yhtenäisyyden täytyy olla ehdoton ja toteuttaa kaikki ACID periaatteet. Ero johtuu osittain siitä, että mallit joilla voidaan tarjota ehdoton johdonmukaisuus usean palvelun välillä vähentää mahdollisuutta rinnakkaisuudelle ja lisäksi vaikuttaa palveluiden saavutettavuuteen heikentävästi. Tästä syystä mikropalveluarkkitehtuurissa usein luovutaan ehdottomasta yhtenäisyydestä, koska sen seuraksena voidaan saavuttaa korkeampi suorituskyky ja lisääntynyt saavutettavuus. Etenkin saga-mallin havaittiin olevan suosittu yhtenäisyyden hallintaan mikropalveluiden välillä, koska siitä keskusteltiin ja siihen liittyviä parannusehdotuksia ja toteutustapoja ehdotettiin useissa töissä. Vaikkakin saga-malli on tällä hetkellä yleisesti käytetty tapa mikropalveluiden välisessä koordinoinnissa, valitusta kirjallisuudesta huomattiin myös tarve ehdottoman johdonmukaisuuden toteuttaville malleille. Useita uusia malleja ehdotettiin ratkaisemaan tämänhetkisissä ratkaisuissa olevia ongelmia, mutta myös ratkaisuja joilla voitaisiin poistaa tarve usean palvelun väliseen koordinointiin ehdotettiin. Vaikka ehdotetut mallit ovatkin lupaavia, ne ovat vasta suunnitteluvaiheessa eikä niitä voida käyttää luotettavasti tai helposti teollisuusympäristössä. Tästä syystä lisätutkimuksia tarvitaan näiden uusien mallien jalostamiseen tai kokonaan uusien mallien visiontiin.The popularity of microservice architecture has risen recently due to its multiple advantages partly related to the increased independence of services. One of the features that improve independence is decentralized data management, which outlines that each service should manage its own data with preferred data management technologies. However, the usage of decentralized data management brings problems, especially with data consistency when data owned by separate microservices must be modified in coordination. To alleviate this, a shared database between services could be used as it removes the need for coordination altogether, but then again, the usage of a single database could defeat some of the benefits of microservice architecture by increasing tight coupling between services. Therefore, it is important to consider other possibilities to manage the coordination while maintaining the independence of the services. We conducted a systematic mapping study to find out suitable design patterns to manage the coordination between multiple microservices. Firstly, design patterns that seemed widely discussed and adopted were identified. After this, these patterns were presented using a template that included advantages and disadvantages for each pattern. The results gathered in the systematic mapping study show that even though traditional systems pursue strict consistency with ACID guarantees, eventual consistency patterns, such as the saga pattern, seem to be more popular in the microservice environment. This is due to drawbacks within distributed transaction protocols including limited concurrency and reduced availability which makes developers choose loosened consistency as a trade-off for higher availability and increased performance. The prevalence of the saga pattern can be seen in the selected works as there are multiple articles proposing methods to manage different parts of the pattern. Also, implementation details were mainly related to the saga pattern in the selected works. Even though the saga pattern is currently the most prevalent option, there is still interest in highly consistent coordination methods in the research community. Multiple solutions have been proposed, which either propose new consistency protocols with strict consistency guarantees or entirely new solutions to remove the need for coordination completely. However, there are no novel solutions that could manage the requirements of microservice architecture reliably in the industry setting yet. Therefore, further research is still required to refine already proposed solutions or to vision new solutions for this problem

On-premise containerized, light-weight software solutions for Biomedicine

Bioinformatics software systems are critical tools for analysing large-scale biological data, but their design and implementation can be challenging due to the need for reliability, scalability, and performance. This thesis investigates the impact of several software approaches on the design and implementation of bioinformatics software systems. These approaches include software patterns, microservices, distributed computing, containerisation and container orchestration. The research focuses on understanding how these techniques affect bioinformatics software systems’ reliability, scalability, performance, and efficiency. Furthermore, this research highlights the challenges and considerations involved in their implementation. This study also examines potential solutions for implementing container orchestration in bioinformatics research teams with limited resources and the challenges of using container orchestration. Additionally, the thesis considers microservices and distributed computing and how these can be optimised in the design and implementation process to enhance the productivity and performance of bioinformatics software systems. The research was conducted using a combination of software development, experimentation, and evaluation. The results show that implementing software patterns can significantly improve the code accessibility and structure of bioinformatics software systems. Specifically, microservices and containerisation also enhanced system reliability, scalability, and performance. Additionally, the study indicates that adopting advanced software engineering practices, such as model-driven design and container orchestration, can facilitate efficient and productive deployment and management of bioinformatics software systems, even for researchers with limited resources. Overall, we develop a software system integrating all our findings. Our proposed system demonstrated the ability to address challenges in bioinformatics. The thesis makes several key contributions in addressing the research questions surrounding the design, implementation, and optimisation of bioinformatics software systems using software patterns, microservices, containerisation, and advanced software engineering principles and practices. Our findings suggest that incorporating these technologies can significantly improve bioinformatics software systems’ reliability, scalability, performance, efficiency, and productivity.Bioinformatische Software-Systeme stellen bedeutende Werkzeuge für die Analyse umfangreicher biologischer Daten dar. Ihre Entwicklung und Implementierung kann jedoch aufgrund der erforderlichen Zuverlässigkeit, Skalierbarkeit und Leistungsfähigkeit eine Herausforderung darstellen. Das Ziel dieser Arbeit ist es, die Auswirkungen von Software-Mustern, Microservices, verteilten Systemen, Containerisierung und Container-Orchestrierung auf die Architektur und Implementierung von bioinformatischen Software-Systemen zu untersuchen. Die Forschung konzentriert sich darauf, zu verstehen, wie sich diese Techniken auf die Zuverlässigkeit, Skalierbarkeit, Leistungsfähigkeit und Effizienz von bioinformatischen Software-Systemen auswirken und welche Herausforderungen mit ihrer Konzeptualisierungen und Implementierung verbunden sind. Diese Arbeit untersucht auch potenzielle Lösungen zur Implementierung von Container-Orchestrierung in bioinformatischen Forschungsteams mit begrenzten Ressourcen und die Einschränkungen bei deren Verwendung in diesem Kontext. Des Weiteren werden die Schlüsselfaktoren, die den Erfolg von bioinformatischen Software-Systemen mit Containerisierung, Microservices und verteiltem Computing beeinflussen, untersucht und wie diese im Design- und Implementierungsprozess optimiert werden können, um die Produktivität und Leistung bioinformatischer Software-Systeme zu steigern. Die vorliegende Arbeit wurde mittels einer Kombination aus Software-Entwicklung, Experimenten und Evaluation durchgeführt. Die erzielten Ergebnisse zeigen, dass die Implementierung von Software-Mustern, die Zuverlässigkeit und Skalierbarkeit von bioinformatischen Software-Systemen erheblich verbessern kann. Der Einsatz von Microservices und Containerisierung trug ebenfalls zur Steigerung der Zuverlässigkeit, Skalierbarkeit und Leistungsfähigkeit des Systems bei. Darüber hinaus legt die Arbeit dar, dass die Anwendung von SoftwareEngineering-Praktiken, wie modellgesteuertem Design und Container-Orchestrierung, die effiziente und produktive Bereitstellung und Verwaltung von bioinformatischen Software-Systemen erleichtern kann. Zudem löst die Implementierung dieses SoftwareSystems, Herausforderungen für Forschungsgruppen mit begrenzten Ressourcen. Insgesamt hat das System gezeigt, dass es in der Lage ist, Herausforderungen im Bereich der Bioinformatik zu bewältigen und stellt somit ein wertvolles Werkzeug für Forscher in diesem Bereich dar. Die vorliegende Arbeit leistet mehrere wichtige Beiträge zur Beantwortung von Forschungsfragen im Zusammenhang mit dem Entwurf, der Implementierung und der Optimierung von Software-Systemen für die Bioinformatik unter Verwendung von Prinzipien und Praktiken der Softwaretechnik. Unsere Ergebnisse deuten darauf hin, dass die Einbindung dieser Technologien die Zuverlässigkeit, Skalierbarkeit, Leistungsfähigkeit, Effizienz und Produktivität bioinformatischer Software-Systeme erheblich verbessern kann

A software architecture for electro-mobility services: a milestone for sustainable remote vehicle capabilities

To face the tough competition, changing markets and technologies in automotive industry, automakers have to be highly innovative. In the previous decades, innovations were electronics and IT-driven, which increased exponentially the complexity of vehicle’s internal network. Furthermore, the growing expectations and preferences of customers oblige these manufacturers to adapt their business models and to also propose mobility-based services. One other hand, there is also an increasing pressure from regulators to significantly reduce the environmental footprint in transportation and mobility, down to zero in the foreseeable future. This dissertation investigates an architecture for communication and data exchange within a complex and heterogeneous ecosystem. This communication takes place between various third-party entities on one side, and between these entities and the infrastructure on the other. The proposed solution reduces considerably the complexity of vehicle communication and within the parties involved in the ODX life cycle. In such an heterogeneous environment, a particular attention is paid to the protection of confidential and private data. Confidential data here refers to the OEM’s know-how which is enclosed in vehicle projects. The data delivered by a car during a vehicle communication session might contain private data from customers. Our solution ensures that every entity of this ecosystem has access only to data it has the right to. We designed our solution to be non-technological-coupling so that it can be implemented in any platform to benefit from the best environment suited for each task. We also proposed a data model for vehicle projects, which improves query time during a vehicle diagnostic session. The scalability and the backwards compatibility were also taken into account during the design phase of our solution. We proposed the necessary algorithms and the workflow to perform an efficient vehicle diagnostic with considerably lower latency and substantially better complexity time and space than current solutions. To prove the practicality of our design, we presented a prototypical implementation of our design. Then, we analyzed the results of a series of tests we performed on several vehicle models and projects. We also evaluated the prototype against quality attributes in software engineering

Building and monitoring an event-driven microservices ecosystem

Throughout the years, software architectures have evolved deeply to attempt to address the main issues that have been emerging, mainly due to the ever-changing market needs. The need to provide a way for organizations and teams to build applications independently and with greater agility and speed led to the adoption of microservices, particularly endorsing an asynchronous methodology of communication between them via events. Moreover, the evergrowing demands for high-quality resilient and highly available systems helped pave the path towards a greater focus on strict quality measures, particularly monitoring and other means of assuring the well-functioning of components in production in real-time. Although techniques like logging, monitoring, and alerting are essential to be employed for each microservice, it may not be enough considering an event-driven architecture. Studies have shown that although organizations have been adopting this type of software architecture, they still struggle with the lack of visibility into end-to-end business processes that span multiple microservices. This thesis explores how to guarantee observability over such architecture, thus keeping track of the business processes. It shall do so by providing a tool that facilitates the analysis of the current situation of the ecosystem, as well as allow to view and possibly act upon the data. Two solutions have been explored and are therefore presented thoroughly, alongside a detailed comparison with the purpose of drawing conclusions and providing some guidance to the readers. These outcomes that were produced by the thesis resulted in a paper published and registered to be presented at this year’s edition of the SEI hosted at ISEP.Ao longo dos últimos anos, as arquiteturas de software têm evoluído significativamente de forma a tentar resolver os principais problemas que têm surgindo, principalmente derivados nas necessidades do mercado que estão em constante mudança. A necessidade de providenciar uma forma das organizações e suas equipas construírem aplicações independentemente e com uma maior agilidade e rapidez levou à adoção de microserviços, geralmente aplicando uma metodologia de comunicação assíncrona através de eventos. Para além disso, a constante evolução da necessidade de ter sistemas de qualidade e altamente resilientes e disponíveis, ajudou a direcionar um maior foco para padrões de qualidade mais rigorosos, particularmente no que toca a monitorização e outros meios para assegurar o correto funcionamento de componentes em produção em tempo-real. Embora técnicas como a produção de logs, monitorização e alarmística sejam essenciais para ser aplicadas a cada microserviço, poderá não ser suficiente quando consideramos uma arquitetura baseada em eventos. Estudos recentes apontam para que organizações, apesar de estarem a adotar cada vez mais este tipo de arquiteturas de software, ainda encontram bastantes dificuldades devido à falta de visibilidade que possuem dos processos de negócio que envolvem e se propagam por diversos microserviços. Esta tese explora como garantir visibilidade sobre uma arquitetura como a descrita, e assim conseguir seguir os processos de negócio. O resultado da mesma deverá atender a isso providenciando uma ferramenta que facilita a análise da situação atual do ecossistema, e que possibilita a visualização e a intervenção sobre os dados que são disponibilizados. Foram desenvolvidas duas soluções que serão apresentadas detalhadamente juntamente com uma comparação entre as duas com o propósito de tirar mais conclusões e providenciar alguma orientação ao leitor. A tese originou a criação de um artigo submetido para ser apresentado na edição deste ano do SEI

Proyecto Docente e Investigador, Trabajo Original de Investigación y Presentación de la Defensa, preparado por Germán Moltó para concursar a la plaza de Catedrático de Universidad, concurso 082/22, plaza 6708, área de Ciencia de la Computación e Inteligencia Artificial

Este documento contiene el proyecto docente e investigador del candidato Germán Moltó Martínez presentado como requisito para el concurso de acceso a plazas de Cuerpos Docentes Universitarios. Concretamente, el documento se centra en el concurso para la plaza 6708 de Catedrático de Universidad en el área de Ciencia de la Computación en el Departamento de Sistemas Informáticos y Computación de la Universitat Politécnica de València. La plaza está adscrita a la Escola Técnica Superior d'Enginyeria Informàtica y tiene como perfil las asignaturas "Infraestructuras de Cloud Público" y "Estructuras de Datos y Algoritmos".También se incluye el Historial Académico, Docente e Investigador, así como la presentación usada durante la defensa.Germán Moltó Martínez (2022). Proyecto Docente e Investigador, Trabajo Original de Investigación y Presentación de la Defensa, preparado por Germán Moltó para concursar a la plaza de Catedrático de Universidad, concurso 082/22, plaza 6708, área de Ciencia de la Computación e Inteligencia Artificial. http://hdl.handle.net/10251/18903

An examination of public and private worlds in modern American literature

This thesis examines American novels written between 1890 and 1975 from the point of view of their formal and structural aspects and considers the implications arising from the manner in which they are deployed within the context of individual works. The phrase ’public and private worlds' was formulated to describe these aspects. The ’public world’ of a novel is the environment in which the characters have their existence and comprises places, physical objects and that area of tacitly agreed norms, society. Where groups of characters are instrumental in translating these norms into pressures upon the individual, then they too are considered as a part of the public world. 'Private worlds' are defined as those aspects of the fiction seen as belonging to a specific character) his words, thoughts, actions and emotions, and all these taken together as his consciousness, are considered as the private world. The phrase public and private worlds is a dichotomy only in the sense that it refers to two defined components of the novel; the 'worlds' are not assumed to be diametrically opposed. Indeed, this thesis is largely about the varied ways in which authors relate them. The weighting given to each side of the dichotomy in individual novels is considered and the resultant 'balance' described as the 'overall fictional reality’, a term which takes into account the effect which the combination of public and private worlds has upon the reader. Whilst the aim of this thesis is primarily analytic, the overall fictional reality is evaluated on the grounds of its coherence, subtlety and artistic merit, all factors which emerge naturally from the application of the dichotomy. The dichotomy may, of course, be used to analyse novels of any period or culture. American novels written over the past nine decades present a considerable challenge to its validity as a critical tool, for they vary widely in style, content and artistic worth and provide a 'fictional' mirror of the vast social changes occurring during this period. In order to present a further challenge to the dichotomy and to give the discussion extended scope, each chapter deals with different configurations of books, ranging from the works of a single author to those sharing nothing more than a coincidence of setting. Thus, it is the intention of this thesis both to test the worth of 'public and private worlds' as an aid to analysis and to show that this particular, formal relationship is crucial to American fiction

Pivot 2021: Dismantling/Reassembling

Pivot is a series of virtual conferences organized by the Pluriversal Design Special Interest Group (SIG) of the Design Research Society (DRS). Pivot’s first edition, PIVOT 2020: Designing a world of many centers, was hosted by the Phyllis M. Taylor Center for Social Innovation and Design Thinking at Tulane University. The 2021 edition was hosted by OCAD University (Toronto, Canada)