Data Management in Microservices: State of the Practice, Challenges, and Research Directions

We are recently witnessing an increased adoption of microservice architectures by the industry for achieving scalability by functional decomposition, fault-tolerance by deployment of small and independent services, and polyglot persistence by the adoption of different database technologies specific to the needs of each service. Despite the accelerating industrial adoption and the extensive research on microservices, there is a lack of thorough investigation on the state of the practice and the major challenges faced by practitioners with regard to data management. To bridge this gap, this paper presents a detailed investigation of data management in microservices. Our exploratory study is based on the following methodology: we conducted a systematic literature review of articles reporting the adoption of microservices in industry, where more than 300 articles were filtered down to 11 representative studies; we analyzed a set of 9 popular open-source microservice-based applications, selected out of more than 20 open-source projects; furthermore, to strengthen our evidence, we conducted an online survey that we then used to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 practitioners and researchers. Through this process, we were able to categorize the state of practice and reveal several principled challenges that cannot be solved by software engineering practices, but rather need system-level support to alleviate the burden of practitioners. Based on the observations we also identified a series of research directions to achieve this goal. Fundamentally, novel database systems and data management tools that support isolation for microservices, which include fault isolation, performance isolation, data ownership, and independent schema evolution across microservices must be built to address the needs of this growing architectural style

Technical Challenges of Microservices Migration

The microservices architecture is a recent trend in the software engineering community, with the number of research articles in the field increasing, and more companies adopting the architectural style every year. However, the migration of a monolith to the microservices architecture is an error-prone process with a lack of guidelines for its execution. Also, microservices introduce a lot of different challenges that are not faced when following a monolithic architecture. This work aims to fill some gaps in current microservices research by providing a catalogue of the currently most common challenges of adopting this architectural style, and possible solutions for them. For this reason, a systematic mapping study was executed analysing 54 different articles. Also, 30 industry professionals participated in a questionnaire regarding the topic. Furthermore, a participant observation experiment was performed to retrieve additional industry data. Moreover, one of the identified challenges – distributed transactions management – was further detailed and a solution implemented using the choreographed saga pattern. The solution is publicly available as an open-source project. Finally, multiple experts in the microservices field validated the results of the research and the distributed transactions solution and provided insights regarding the value of this work.A arquitetura de microserviços é uma tendência recente na comunidade de engenharia de software, com o número de artigos publicados sobre o tema a aumentar, assim como o número de empresas a adoptar o estilo arquitetural todos os anos. No entanto, o processo de migração de um monolito para uma arquitetura orientada a microserviços tem um alto potencial de erros, uma vez que existe falta de orientações sobre como conduzir o processo corretamente. Para além disso, os microserviços introduzem muitos desafios diferentes que não são enfrentados no desenvolvimento de um sistema monolitico. Este trabalho pretende preencher algumas destas lacunas na investigação da arquitetura de microserviços através da construção de um catalogo dos principais desafios enfrentados ao adoptar o estilo arquitetural e soluções possíveis para estes. Por este motivo, um systematic mapping study foi desenvolvido, analisando 54 artigos diferentes. Para além disso, 30 profissionais da industria responderam a questionario sobre o tema. Finalmente, para obter dados adicionais da indústria, uma experiência de migração foi realizada e observada de forma ativa. Ainda, um dos desafios identificados – gestão de transações distribuídas – foi detalhado e uma solução implementada usando o padrão de sagas coreografadas. A solução está publicamente disponível como um projecto open-source. Finalmente, vários peritos em microserviços avaliaram os resultados deste trabalho, incluindo a solução desenvolvida para gestão de transações distribuídas, e deram feedback relativamente ao valor deste trabalho