4,910 research outputs found
Parallel Deferred Update Replication
Deferred update replication (DUR) is an established approach to implementing
highly efficient and available storage. While the throughput of read-only
transactions scales linearly with the number of deployed replicas in DUR, the
throughput of update transactions experiences limited improvements as replicas
are added. This paper presents Parallel Deferred Update Replication (P-DUR), a
variation of classical DUR that scales both read-only and update transactions
with the number of cores available in a replica. In addition to introducing the
new approach, we describe its full implementation and compare its performance
to classical DUR and to Berkeley DB, a well-known standalone database
Asynchronous Validity Resolution in Sequentially Consistent Shared Virtual Memory
Shared Virtual Memory (SVM) is an effort to provide a mechanism for a distributed system, such as a cluster, to execute shared memory parallel programs. Unfortunately, SVM has performance problems due to its underlying distributed architecture. Recent developments have increased performance of SVM by reducing communication. Unfortunately this performance gain was only possible by increasing programming complexity and by restricting the types of programs allowed to execute in the system. Validity resolution is the process of resolving the validity of a memory object such as a page. Current SVM systems use synchronous or deferred validity resolution techniques in which user processing is blocked during the validity resolution process. This is the case even when resolving validity of false shared variables. False-sharing occurs when two or more processes access unrelated variables stored within the same shared block of memory and at least one of the processes is writing. False sharing unnecessarily reduces overall performance of SVM systems?because user processing is blocked during validity resolution although no actual data dependencies exist. This thesis presents Asynchronous Validity Resolution (AVR), a new approach to SVM which reduces the performance losses associated with false sharing while maintaining the ease of programming found with regular shared memory parallel programming methodology. Asynchronous validity resolution allows concurrent user process execution and data validity resolution. AVR is evaluated by com-paring performance of an application suite using both an AVR sequentially con-sistent SVM system and a traditional sequentially consistent (SC) SVM system. The results show that AVR can increase performance over traditional sequentially consistent SVM for programs which exhibit false sharing. Although AVR outperforms regular SC by as much as 26%, performance of AVR is dependent on the number of false-sharing vs. true-sharing accesses, the number of pages in the program’s working set, the amount of user computation that completes per page request, and the internodal round-trip message time in the system. Overall, the results show that AVR could be an important member of the arsenal of tools available to parallel programmers
A taxonomy of the quality attributes for distributed applications
The software engineering community has paid little attention to non-functional requirements, or quality attributes, compared with studies performed on capture, analysis and validation of functional requirements.
This circumstance becomes more intense in the case of distributed applications. In these applications we have to take into account, besides the quality attributes such as correctness, robustness, extendibility, reusability, compatibility, efficiency, portability and ease of use, others like reliability, scalability, transparency, security, interoperability, concurrency, etc.
In this work we will show how these last attributes are related to different abstractions that coexist in the problem domain. To achieve this goal, we have established a taxonomy of quality attributes of distributed applications and have determined the set of necessary services to support such attributes
Partial replication with strong consistency
In response to the increasing expectations of their clients, cloud services exploit
geo-replication to provide fault-tolerance, availability and low latency when executing
requests. However, cloud platforms tend to adopt weak consistency semantics, in which
replicas may diverge in state independently. These systems offer good response times
but at the disadvantage of allowing potential data inconsistencies that may affect user
experience.
Some systems propose to adopt solutions with strong consistency, which are not as
efficient but simplify the development of correct applications by guaranteeing that all
replicas in the system maintain the same database state. Therefore, it is interesting to explore
a system that can offer strong consistency while minimizing its main disadvantage:
the impact in performance that results from coordinating every replica in the system. A
possible solution to reduce the cost of replica coordination is to support partial replication.
Partially replicating a database allows for each server to only be responsible for a
subset of the data - a partition - which means that when updating the database only some
of replicas have to be synchronized, improving response times.
In this dissertation, we propose an algorithm that implements a distributed replicated
database that offers strong consistency with support for partial replication. To achieve
strong consistency in a partially replicated scenario, our algorithm is in part based on the
Clock-SI[10] research, which presents an algorithm that implements a multi-versioned
database for strong consistency (snapshot-isolation) and performs the Two-Phase Commit
protocol when coordinating replicas during updates. The algorithm is supported by
an architecture that simplifies distributing partitions among datacenters and efficiently
propagating operations across nodes in the same partition, thanks to the ChainPaxos[27]
algorithm.Como forma de responder às expectativas cada vez maiores dos seus clientes, as
operadoras cloud tiram partido da geo-replicação para oferecer tolerância a falhas, disponibilidade
e baixa latência dos seus sistemas na resposta aos pedidos. No entanto, as
plataformas cloud tendem a adotar uma semântica de consistência fraca, na qual as réplicas
podem variar em estado de forma independente. Estes sistemas oferecem bons tempos
de resposta mas com a desvantagem de que têm de lidar com potenciais inconsistências
nos dados que podem ter impacto na experiência dos utilizadores.
Alguns sistemas propõem adotar soluções com consistência forte, as quais não são
tão eficientes mas simplificam o desenvolvimento de aplicações ao garantir que todas
as réplicas do sistema mantêm o mesmo estado da base de dados. É então interessante
explorar um sistema que garanta replicação forte mas que minimize a sua principal
desvantagem: o impacto de performance no momento de coordenar o estado das réplicas
nos sistema. Uma possível solução para reduzir o custo de coordenação das réplicas
durante transações é o suporte à replicação parcial. Replicar parcialmente uma base de
dados permite que cada servidor seja apenas responsável por uma parte dos dados - uma
partição - o que significa que quando são realizadas escritas apenas algumas das réplicas
têm de ser sincronizadas, melhorando os tempos de resposta.
Neste trabalho propomos um algoritmo que implementa um sistema de armazenamento
distríbuido replicado que oferece consistência forte com suporte a replicação parcial.
A fim de garantir consistência forte num cenário de replicação parcial, o nosso
algoritmo é em parte baseado no algoritmo Clock-SI[10], que implementa uma base de
dados parcial com multi-versões para garantir consistência forte (snapshot-isolation) e
que realiza o protocolo Two-Phase Commit para coordenar as réplicas no momento de
aplicar escritas. O algoritmo é suportado por uma arquitectura que torna simples distribuir
partições por vários centros de dados e propagar de forma eficiente operações entre
todos os nós numa mesma partição, através do algoritmo ChainPaxos[27]
A Peer-to-Peer Middleware Framework for Resilient Persistent Programming
The persistent programming systems of the 1980s offered a programming model
that integrated computation and long-term storage. In these systems, reliable
applications could be engineered without requiring the programmer to write
translation code to manage the transfer of data to and from non-volatile
storage. More importantly, it simplified the programmer's conceptual model of
an application, and avoided the many coherency problems that result from
multiple cached copies of the same information. Although technically
innovative, persistent languages were not widely adopted, perhaps due in part
to their closed-world model. Each persistent store was located on a single
host, and there were no flexible mechanisms for communication or transfer of
data between separate stores. Here we re-open the work on persistence and
combine it with modern peer-to-peer techniques in order to provide support for
orthogonal persistence in resilient and potentially long-running distributed
applications. Our vision is of an infrastructure within which an application
can be developed and distributed with minimal modification, whereupon the
application becomes resilient to certain failure modes. If a node, or the
connection to it, fails during execution of the application, the objects are
re-instantiated from distributed replicas, without their reference holders
being aware of the failure. Furthermore, we believe that this can be achieved
within a spectrum of application programmer intervention, ranging from minimal
to totally prescriptive, as desired. The same mechanisms encompass an
orthogonally persistent programming model. We outline our approach to
implementing this vision, and describe current progress.Comment: Submitted to EuroSys 200
BALANCING PRIVACY, PRECISION AND PERFORMANCE IN DISTRIBUTED SYSTEMS
Privacy, Precision, and Performance (3Ps) are three fundamental design objectives in distributed systems. However, these properties tend to compete with one another and are not considered absolute properties or functions. They must be defined and justified in terms of a system, its resources, stakeholder concerns, and the security threat model.
To date, distributed systems research has only considered the trade-offs of balancing privacy, precision, and performance in a pairwise fashion. However, this dissertation formally explores the space of trade-offs among all 3Ps by examining three representative classes of distributed systems, namely Wireless Sensor Networks (WSNs), cloud systems, and Data Stream Management Systems (DSMSs). These representative systems support large part of the modern and mission-critical distributed systems.
WSNs are real-time systems characterized by unreliable network interconnections and highly constrained computational and power resources. The dissertation proposes a privacy-preserving in-network aggregation protocol for WSNs demonstrating that the 3Ps could be navigated by adopting the appropriate algorithms and cryptographic techniques that are not prohibitively expensive.
Next, the dissertation highlights the privacy and precision issues that arise in cloud databases due to the eventual consistency models of the cloud. To address these issues, consistency enforcement techniques across cloud servers are proposed and the trade-offs between 3Ps are discussed to help guide cloud database users on how to balance these properties.
Lastly, the 3Ps properties are examined in DSMSs which are characterized by high volumes of unbounded input data streams and strict real-time processing constraints. Within this system, the 3Ps are balanced through a proposed simple and efficient technique that applies access control policies over shared operator networks to achieve privacy and precision without sacrificing the systems performance.
Despite that in this dissertation, it was shown that, with the right set of protocols and algorithms, the desirable 3P properties can co-exist in a balanced way in well-established distributed systems, this dissertation is promoting the use of the new 3Ps-by-design concept. This concept is meant to encourage distributed systems designers to proactively consider the interplay among the 3Ps from the initial stages of the systems design lifecycle rather than identifying them as add-on properties to systems
Asynchronous replication of metadata across multi-master servers in distributed data storage systems
In recent years, scientific applications have become increasingly data intensive. The increase in the size of data generated by scientific applications necessitates collaboration and sharing data among the nation\u27s education and research institutions. To address this, distributed storage systems spanning multiple institutions over wide area networks have been developed. One of the important features of distributed storage systems is providing global unified name space across all participating institutions, which enables easy data sharing without the knowledge of actual physical location of data. This feature depends on the ``location metadata\u27\u27 of all data sets in the system being available to all participating institutions. This introduces new challenges. In this thesis, we study different metadata server layouts in terms of high availability, scalability and performance. A central metadata server is a single point of failure leading to low availability. Ensuring high availability requires replication of metadata servers. A synchronously replicated metadata servers layout introduces synchronization overhead which degrades the performance of data operations. We propose an asynchronously replicated multi-master metadata servers layout which ensures high availability, scalability and provides better performance. We discuss the implications of asynchronously replicated multi-master metadata servers on metadata consistency and conflict resolution. Further, we design and implement our own asynchronous multi-master replication tool, deploy it in the state-wide distributed data storage system called PetaShare, and compare performance of all three metadata server layouts: central metadata server, synchronously replicated multi-master metadata servers and asynchronously replicated multi-master metadata servers
SUPPORTING MULTIPLE ISOLATION LEVELS IN REPLICATED ENVIRONMENTS
La replicación de bases de datos aporta fiabilidad y escalabilidad aunque hacerlo
de forma transparente no es una tarea sencilla. Una base de datos replicada es
transparente si puede reemplazar a una base de datos centralizada tradicional sin
que sea necesario adaptar el resto de componentes del sistema. La transparencia
en bases de datos replicadas puede obtenerse siempre que (a) la gestión de la
replicación quede totalmente oculta a dichos componentes y (b) se ofrezca la
misma funcionalidad que en una base de datos tradicional.
Para mejorar el rendimiento general del sistema, los gestores de bases de datos
centralizadas actuales permiten ejecutar de forma concurrente transacciones
bajo distintos niveles de aislamiento. Por ejemplo, la especificación del benchmark
TPC-C permite la ejecución de algunas transacciones con niveles de aislamiento
débiles. No obstante, este soporte todavía no está disponible en los
protocolos de replicación. En esta tesis mostramos cómo estos protocolos pueden
ser extendidos para permitir la ejecución de transacciones con distintos niveles
de aislamiento.Bernabe Gisbert, JM. (2014). SUPPORTING MULTIPLE ISOLATION LEVELS IN REPLICATED ENVIRONMENTS [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/36535TESI
- …