7,572 research outputs found
Alpha Entanglement Codes: Practical Erasure Codes to Archive Data in Unreliable Environments
Data centres that use consumer-grade disks drives and distributed
peer-to-peer systems are unreliable environments to archive data without enough
redundancy. Most redundancy schemes are not completely effective for providing
high availability, durability and integrity in the long-term. We propose alpha
entanglement codes, a mechanism that creates a virtual layer of highly
interconnected storage devices to propagate redundant information across a
large scale storage system. Our motivation is to design flexible and practical
erasure codes with high fault-tolerance to improve data durability and
availability even in catastrophic scenarios. By flexible and practical, we mean
code settings that can be adapted to future requirements and practical
implementations with reasonable trade-offs between security, resource usage and
performance. The codes have three parameters. Alpha increases storage overhead
linearly but increases the possible paths to recover data exponentially. Two
other parameters increase fault-tolerance even further without the need of
additional storage. As a result, an entangled storage system can provide high
availability, durability and offer additional integrity: it is more difficult
to modify data undetectably. We evaluate how several redundancy schemes perform
in unreliable environments and show that alpha entanglement codes are flexible
and practical codes. Remarkably, they excel at code locality, hence, they
reduce repair costs and become less dependent on storage locations with poor
availability. Our solution outperforms Reed-Solomon codes in many disaster
recovery scenarios.Comment: The publication has 12 pages and 13 figures. This work was partially
supported by Swiss National Science Foundation SNSF Doc.Mobility 162014, 2018
48th Annual IEEE/IFIP International Conference on Dependable Systems and
Networks (DSN
Analysis of Trade-offs in Fault-Tolerant Distributed Computing and Replicated Databases
This paper examines fundamental trade-offs in fault-tolerant distributed systems and replicated databases built over the Internet. We discuss interplays between consistency, availability, and latency which are in the very nature of globally distributed computer systems and also analyse their interconnection with durability and energy efficiency. In this paper we put forward an idea that consistency, availability, latency, durability and other properties need to be viewed as more continuous than binary in contrast to the well-known CAP/PACELC theorems. We compare different consistency models and highlight the role of the application timeout, replication factor and other settings that essentially determine the interplay between above properties. Our findings may be of interest to software engineers and system architects who develop Internet-scale distributed computer systems and cloud solutions
The impact of microservices: an empirical analysis of the emerging software architecture
Dissertação de mestrado em Informatics EngineeringThe applications’ development paradigm has faced changes in recent years, with modern development being
characterized by the need to continuously deliver new software iterations. With great affinity with those principles,
microservices is a software architecture which features characteristics that potentially promote multiple quality
attributes often required by modern, large-scale applications. Its recent growth in popularity and acceptance in
the industry made this architectural style often described as a form of modernizing applications that allegedly
solves all the traditional monolithic applications’ inconveniences. However, there are multiple worth mentioning
costs associated with its adoption, which seem to be very vaguely described in existing empirical research, being
often summarized as "the complexity of a distributed system". The adoption of microservices provides the
agility to achieve its promised benefits, but to actually reach them, several key implementation principles have
to be honored. Given that it is still a fairly recent approach to developing applications, the lack of established
principles and knowledge from development teams results in the misjudgment of both costs and values of this
architectural style. The outcome is often implementations that conflict with its promised benefits. In order to
implement a microservices-based architecture that achieves its alleged benefits, there are multiple patterns and
methodologies involved that add a considerable amount of complexity. To evaluate its impact in a concrete and
empirical way, one same e-commerce platform was developed from scratch following a monolithic architectural
style and two architectural patterns based on microservices, featuring distinct inter-service communication and
data management mechanisms. The effort involved in dealing with eventual consistency, maintaining a communication
infrastructure, and managing data in a distributed way portrayed significant overheads not existent in the
development of traditional applications. Nonetheless, migrating from a monolithic architecture to a microservicesbased
is currently accepted as the modern way of developing software and this ideology is not often contested,
nor the involved technical challenges are appropriately emphasized. Sometimes considered over-engineering,
other times necessary, this dissertation contributes with empirical data from insights that showcase the impact
of the migration to microservices in several topics. From the trade-offs associated with the use of specific patterns,
the development of the functionalities in a distributed way, and the processes to assure a variety of quality
attributes, to performance benchmarks experiments and the use of observability techniques, the entire development
process is described and constitutes the object of study of this dissertation.O paradigma de desenvolvimento de aplicações tem visto alterações nos últimos anos, sendo o desenvolvimento
moderno caracterizado pela necessidade de entrega contínua de novas iterações de software. Com
grande afinidade com esses princípios, microsserviços são uma arquitetura de software que conta com características
que potencialmente promovem múltiplos atributos de qualidade frequentemente requisitados por aplicações
modernas de grandes dimensões. O seu recente crescimento em popularidade e aceitação na industria
fez com que este estilo arquitetural se comumente descrito como uma forma de modernizar aplicações que
alegadamente resolve todos os inconvenientes apresentados por aplicações monolíticas tradicionais. Contudo,
existem vários custos associados à sua adoção, aparentemente descritos de forma muito vaga, frequentemente
sumarizados como a "complexidade de um sistema distribuído". A adoção de microsserviços fornece a agilidade
para atingir os seus benefícios prometidos, mas para os alcançar, vários princípios de implementação
devem ser honrados. Dado que ainda se trata de uma forma recente de desenvolver aplicações, a falta de
princípios estabelecidos e conhecimento por parte das equipas de desenvolvimento resulta em julgamentos
errados dos custos e valores deste estilo arquitetural. O resultado geralmente são implementações que entram
em conflito com os seus benefícios prometidos. De modo a implementar uma arquitetura baseada em
microsserviços com os benefícios prometidos existem múltiplos padrões que adicionam considerável complexidade.
De modo a avaliar o impacto dos microsserviços de forma concreta e empírica, foi desenvolvida uma
mesma plataforma e-commerce de raiz segundo uma arquitetura monolítica e duas arquitetura baseadas em
microsserviços, contando com diferentes mecanismos de comunicação entre os serviços. O esforço envolvido
em lidar com consistência eventual, manter a infraestrutura de comunicação e gerir os dados de uma forma distribuída
representaram desafios não existentes no desenvolvimento de aplicações tradicionais. Apesar disso, a
ideologia de migração de uma arquitetura monolítica para uma baseada em microsserviços é atualmente aceite
como a forma moderna de desenvolver aplicações, não sendo frequentemente contestada nem os seus desafios
técnicos são apropriadamente enfatizados. Por vezes considerado overengineering, outras vezes necessário,
a presente dissertação visa contribuir com dados práticos relativamente ao impacto da migração para arquiteturas
baseadas em microsserviços em diversos tópicos. Desde os trade-offs envolvidos no uso de padrões
específicos, o desenvolvimento das funcionalidades de uma forma distribuída e nos processos para assegurar
uma variedade de atributos de qualidade, até análise de benchmarks de performance e uso de técnicas de
observabilidade, todo o desenvolvimento é descrito e constitui o objeto de estudo da dissertação
Recommended from our members
FutureGRID: A Program for long-term research into GRID systems architecture
Proceedings of the 2003 UK e-Science All Hands Meeting, 31st August - 3rd September, Nottingham UKThis is a project to carry out research into long-term GRID architecture, in the University of Cambridge
Computer Laboratory and the Cambridge eScience Center, with support from the Microsoft Research
Laboratory, Cambridge.
It is part of a larger vision for future systems architectures for public computing platforms, including
both scientitic GRID and commodity level computing such as games, peer2peer computing and storage
services and so forth, based on work in the laboratories in recent years into massively scaleable distributed systems for storage, computation, content distribution and collaboration[26]
Malleable Coding with Fixed Reuse
In cloud computing, storage area networks, remote backup storage, and similar
settings, stored data is modified with updates from new versions. Representing
information and modifying the representation are both expensive. Therefore it
is desirable for the data to not only be compressed but to also be easily
modified during updates. A malleable coding scheme considers both compression
efficiency and ease of alteration, promoting codeword reuse. We examine the
trade-off between compression efficiency and malleability cost-the difficulty
of synchronizing compressed versions-measured as the length of a reused prefix
portion. Through a coding theorem, the region of achievable rates and
malleability is expressed as a single-letter optimization. Relationships to
common information problems are also described
- …