Understanding Sharded Caching Systems

Sharding is a method for allocating data items to nodes of a distributed caching or storage system based on the result of a hash function computed on the item identifier. It is ubiquitously used in key-value stores, CDNs and many other applications. Despite considerable work has focused on the design and the implementation of such systems, there is limited understanding of their performance in realistic operational conditions from a theoretical standpoint. In this paper we fill this gap by providing a thorough modeling of sharded caching systems, focusing particularly on load balancing and caching performance aspects. Our analysis provides important insights that can be applied to optimize the design and configuration of sharded caching systems

Efficient read monotonic data aggregation across shards on the cloud

Client-centric consistency models define the view of the data storage expected by a client in relation to the operations done by a client within a session. Monotonic reads is a client-centric consistency model which ensures that if a process has seen a particular value for the object, any subsequent accesses will never return any previous values. Monotonic reads are used in several applications like news feeds and social networks to ensure that the user always has a forward moving view of the data. The idea of Monotonic reads over multiple copies of the data and for lightly loaded systems is intuitive and easy to implement. For example, ensuring that a client session always fetches data from the same server automatically ensures that the user will never view old data. However, such a simplistic setup will not work for large deployments on the cloud, where the data is sharded across multiple high availability setups and there are several million clients accessing data at the same time. In such a setup it becomes necessary to ensure that the data fetched from multiple shards are logically consistent with each other. The use of trivial implementations, like sticky sessions, causes severe performance degradation during peak loads. This paper explores the challenges surrounding consistent monotonic reads over a sharded setup on the cloud and proposes an efficient architecture for the same. Performance of the proposed architecture is measured by implementing it on a cloud setup and measuring the response times for different shard counts. We show that the proposed solution scales with almost no change in performance as the number of shards increases

Load Imbalance and Caching Performance of Sharded Systems

Sharding is a method for allocating data items to nodes of a distributed caching or storage system based on the result of a hash function computed on the item’s identifier. It is ubiquitously used in key-value stores, CDNs and many other applications. Despite considerable work that has focused on the design and implementation of such systems, there is limited understanding of their performance in realistic operational conditions from a theoretical standpoint. In this paper we fill this gap by providing a thorough modeling of sharded caching systems, focusing particularly on load balancing and caching performance aspects. Our analysis provides important insights that can be applied to optimize the design and configuration of sharded caching systems

Rethinking Distributed Caching Systems Design and Implementation

Distributed caching systems based on in-memory key-value stores have become a crucial aspect of fast and efficient content delivery in modern web-applications. However, due to the dynamic and skewed execution environments and workloads, under which such systems typically operate, several problems arise in the form of load imbalance. This thesis addresses the sources of load imbalance in caching systems, mainly: i) data placement, which relates to distribution of data items across servers and ii) data item access frequency, which describes amount of requests each server has to process, and how each server is able to cope with it. Thus, providing several strategies to overcome the sources of imbalance in isolation. As a use case, we analyse Memcached, its variants, and propose a novel solution for distributed caching systems. Our solution revolves around increasing parallelism through load segregation, and solutions to overcome the load discrepancies when reaching high saturation scenarios, mostly through access re-arrangement, and internal replication.Os sistemas de cache distribuídos baseados em armazenamento de pares chave-valor em RAM, tornaram-se um aspecto crucial em aplicações web modernas para o fornecimento rápido e eficiente de conteúdo. No entanto, estes sistemas normalmente estão sujeitos a ambientes muito dinâmicos e irregulares. Este tipo de ambientes e irregularidades, causa vários problemas, que emergem sob a forma de desequilíbrios de carga. Esta tese aborda as diferentes origens de desequilíbrio de carga em sistemas de caching distribuído, principalmente: i) colocação de dados, que se relaciona com a distribuição dos dados pelos servidores e a ii) frequência de acesso aos dados, que reflete a quantidade de pedidos que cada servidor deve processar e como cada servidor lida com a sua carga. Desta forma, demonstramos várias estratégias para reduzir o impacto proveniente das fontes de desequilíbrio, quando analizadas em isolamento. Como caso de uso, analisamos o sistema Memcached, as suas variantes, e propomos uma nova solução para sistemas de caching distribuídos. A nossa solução gira em torno de aumento de paralelismo atraves de segregação de carga e em como superar superar as discrepâncias de carga a quando de sistema entra em grande saturação, principalmente atraves de reorganização de acesso e de replicação intern

Persistência Poliglota - Diferentes Necessidades de Armazenamento de Dados

A necessidade crescente de se armazenarem grandes quantidades de dados, de forma a prover serviços escaláveis, obrigou à procura de novas soluções, ao longo do tempo. No decurso da história podem enumerar-se diversos sistemas de dados: o hierárquico, o em rede, o relacional, o orientado a objetos, o objeto-relacional e, mais recentemente, o NoSQL. Todos estes sistemas tentaram dar resposta a diferentes realidades do armazenamento de dados, indo ao encontro dos problemas de cada época. Devido à necessidade de se aproveitarem as vantagens que cada tipo de base de dados oferece, surgiu o conceito de Persistência Poliglota, que traduz a integração de vários tipos de bases de dados num só sistema. Esta abordagem tem como objetivo retirar o melhor de cada base de dados, apresentando uma solução fiável e alternativa aos sistemas com apenas um tipo de base de dados. Como tal, este trabalho visa a análise da abordagem de Persistência Poliglota para comparar sistemas compostos por diversos sistemas de gestão de base de dados versus os que utilizam apenas um motor de base de dados, de modo a verificar se esta abordagem é útil e vantajosa. Desta forma, elaborou-se uma prova de conceito, com base num problema proposto, com o objetivo de se analisarem dois sistemas, um único e outro poliglota, tendo por base três volumes de dados diferentes. Para isto, foi necessário proceder-se à análise e escolha dos sistemas de gestão de base de dados a utilizar e montar os ambientes de teste, para ambos os sistemas. Com recurso a várias consultas individuais (a cada base de dados) e globais (conjunto das bases de dados que compõem o sistema poliglota), foram analisados os resultados obtidos com recurso à métrica de medição do desempenho relativa aos tempos de consulta. O trabalho e os resultados obtidos evidenciaram um aumento do desempenho, quanto à utilização individual das bases de dados. Perante o conjunto das bases de dados, apesar de um ligeiro aumento, nota-se que os resultados não são claros e que carecem de uma investigação mais profunda. Por fim, é possível afirmar que a abordagem poliglota é principalmente útil em sistemas complexos, onde o volume de dados é elevado, e onde se pretende armazenar diferentes tipos de dados.ABSTRACT: The increasing need to store large amounts of data in order to provide scalable services has forced the search for new solutions over time. In the course of history, several data systems can be enumerated: hierarchical, network, relational, object-oriented, object-relational, and, more recently, NoSQL. All these systems tried to respond to different realities of data storage, meeting the problems of each era. However, due to the need to take benefit from all the advantages that each type of database offers, the concept of Polyglot Persistence has emerged, which allows the integration of several types of databases in a single system. This approach aims to get the best out of each database, presenting a reliable and alternative solution to systems with only one type of database. As such, this work aims at the analysis of Polyglot Persistence approach to compare systems composed of several database management systems versus those using a single database engine, in order to verify if this approach is useful and advantageous. In this way, a proof of concept was elaborated, based on a proposed problem, with the objective of analyzing two systems, a single and another polyglot, based on three different data volumes. For this, it was necessary to proceed to the analysis and choice of the database management systems to be used and to assemble the test environments, for both systems. Using a number of individual queries (for each database) and global queries (set of databases that make up the polyglot system), the results obtained were analyzed using the performance metric relative to the query times. The work and the results obtained showed an increase in the performance, regarding the individual use of the databases. In spite of a slight increase, the results are not clear and need further investigation. Finally, it is possible to affirm that the polyglot approach is mainly useful in complex systems, where the volume of data is high, and it is intended to store different types of data

On the design of efficient caching systems

Content distribution is currently the prevalent Internet use case, accounting for the majority of global Internet traffic and growing exponentially. There is general consensus that the most effective method to deal with the large amount of content demand is through the deployment of massively distributed caching infrastructures as the means to localise content delivery traffic. Solutions based on caching have been already widely deployed through Content Delivery Networks. Ubiquitous caching is also a fundamental aspect of the emerging Information-Centric Networking paradigm which aims to rethink the current Internet architecture for long term evolution. Distributed content caching systems are expected to grow substantially in the future, in terms of both footprint and traffic carried and, as such, will become substantially more complex and costly. This thesis addresses the problem of designing scalable and cost-effective distributed caching systems that will be able to efficiently support the expected massive growth of content traffic and makes three distinct contributions. First, it produces an extensive theoretical characterisation of sharding, which is a widely used technique to allocate data items to resources of a distributed system according to a hash function. Based on the findings unveiled by this analysis, two systems are designed contributing to the abovementioned objective. The first is a framework and related algorithms for enabling efficient load-balanced content caching. This solution provides qualitative advantages over previously proposed solutions, such as ease of modelling and availability of knobs to fine-tune performance, as well as quantitative advantages, such as 2x increase in cache hit ratio and 19-33% reduction in load imbalance while maintaining comparable latency to other approaches. The second is the design and implementation of a caching node enabling 20 Gbps speeds based on inexpensive commodity hardware. We believe these contributions advance significantly the state of the art in distributed caching systems