Superdatabases for Composition of Heterogeneous Databases

Superdatabases are designed to compose and extend databases. In particular, superdatabases allow consistent update across heterogeneous databases. The key idea of superdatabase is hierarchical composition of element databases. For global crash recovery, each element database must provide local recovery plus some kind of agreement protocol, such as two-phase commit. For global concurrency control, each element database must have local synchronization with an explicit serial order, such as two-phase locking, timestamps, or optimistic methods. Given element databases satisfying the above requirements, the superdatabase can certify the serializability of global transactions through a concatenation of local serial order. Combined with previous work on heterogeneous databases, including unified query languages and view integration, now we can build heterogeneous databases which are consistent, adaptable, and extensible by construction

A unified concurrency control algorithm for distributed database systems

We present a unified concurrency-control algorithm for distributed database systems in which each transaction may choose its own concurrency control protocol. Specifically, they integrate two-phase locking, timestamp ordering, and precedence agreement into one unified concurrency-control scheme. They show the correctness of the scheme and study the problem of selecting the best protocol for each transaction to optimize system performance.published_or_final_versio

An Update-intensive LSM-based R-tree Index

Many applications require update-intensive workloads on spatial objects, e.g., social-network services and shared-riding services that track moving objects. By buffering insert and delete operations in memory, the Log Structured Merge Tree (LSM) has been used widely in various systems because of its ability to handle write-heavy workloads. While the focus on LSM has been on key-value stores and their optimizations, there is a need to study how to efficiently support LSM-based {\em secondary} indexes (e.g., location-based indexes) as modern, heterogeneous data necessitates the use of secondary indexes. In this paper, we investigate the augmentation of a main-memory-based memo structure into an LSM secondary index structure to handle update-intensive workloads efficiently. We conduct this study in the context of an R-tree-based secondary index. In particular, we introduce the LSM RUM-tree that demonstrates the use of an Update Memo in an LSM-based R-tree to enhance the performance of the R-tree's insert, delete, update, and search operations. The LSM RUM-tree introduces new strategies to control the size of the Update Memo to make sure it always fits in memory for high performance. The Update Memo is a light-weight in-memory structure that is suitable for handling update-intensive workloads without introducing significant overhead. Experimental results using real spatial data demonstrate that the LSM RUM-tree achieves up to 9.6x speedup on update operations and up to 2400x speedup on query processing over existing LSM R-tree implementations

Consistent data aggregate retrieval for sensor network systems.

Lee Lok Hang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 87-93).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Sensors and Sensor Networks --- p.3Chapter 1.2 --- Sensor Network Deployment --- p.7Chapter 1.3 --- Motivations --- p.7Chapter 1.4 --- Contributions --- p.9Chapter 1.5 --- Thesis Organization --- p.10Chapter 2 --- Literature Review --- p.11Chapter 2.1 --- Data Cube --- p.11Chapter 2.2 --- Data Aggregation in Sensor Networks --- p.12Chapter 2.2.1 --- Hierarchical Data Aggregation --- p.13Chapter 2.2.2 --- Gossip-based Aggregation --- p.13Chapter 2.2.3 --- Hierarchical Gossip Aggregation --- p.13Chapter 2.3 --- GAF Algorithm --- p.14Chapter 2.4 --- Concurrency Control --- p.17Chapter 2.4.1 --- Two-phase Locking --- p.17Chapter 2.4.2 --- Timestamp Ordering --- p.18Chapter 3 --- Building Distributed Data Cubes in Sensor Network --- p.20Chapter 3.1 --- Aggregation Operators --- p.21Chapter 3.2 --- Distributed Prefix (PS) Sum Data Cube --- p.22Chapter 3.2.1 --- Prefix Sum (PS) Data Cube --- p.22Chapter 3.2.2 --- Notations --- p.24Chapter 3.2.3 --- Querying a PS Data Cube --- p.25Chapter 3.2.4 --- Building Distributed PS Data Cube --- p.27Chapter 3.2.5 --- Time Bounds --- p.32Chapter 3.2.6 --- Fast Aggregate Queries on Multiple Regions --- p.37Chapter 3.2.7 --- Simulation Results --- p.43Chapter 3.3 --- Distributed Local Prefix Sum (LPS) Data Cube --- p.50Chapter 3.3.1 --- Local Prefix Sum Data Cube --- p.52Chapter 3.3.2 --- Notations --- p.55Chapter 3.3.3 --- Querying an LPS Data Cube --- p.56Chapter 3.3.4 --- Building Distributed LPS Data Cube --- p.61Chapter 3.3.5 --- Time Bounds --- p.63Chapter 3.3.6 --- Fast Aggregate Queries on Multiple Regions --- p.67Chapter 3.3.7 --- Simulation Results --- p.68Chapter 3.3.8 --- Distributed PS Data Cube Vs Distributed LPS Data Cube --- p.74Chapter 4 --- Concurrency Control and Consistency in Sensor Networks --- p.76Chapter 4.1 --- Data Inconsistency in Sensor Networks --- p.76Chapter 4.2 --- Traditional Concurrency Control Protocols and Sensor Networks --- p.80Chapter 4.3 --- The Consistent Retrieval of Data from Distributed Data Cubes --- p.81Chapter 5 --- Conclusions --- p.85References --- p.87Appendix --- p.94A Publications --- p.9

P4言語を用いたパケット分類アルゴリズムに関する研究

パケット・クラシファイアとは、コンピュータネットワークにおいてネットワーク機器に到着したパケットをグループに分類するメカリズムである。特定の処理のためにパケットを区別して分離する必要があるサービス、例えば、ファイアウォールやサービス品質などのカスタマイズネットワークサービスなどを提供するためにルータでのパケットを分類するのは極めて重要である。パケット分類に関するアルゴリズムがいくつかの研究で提案されている。分類の性能を向上するため、決定木、ヒューリスティックなどを利用した提案がある。しかし、その性能評価は主にハードウェア実装に基づいていたので、アルゴリズムの設計方法、データ構造などソフトウェルーターに適用できない恐れがある。近年、ネットワークプロトコル、ターゲット非依存という特徴をあるP4言語が開発された。P4言語は幅広いのデータプレーンをプログラミングできるように、ネットワークの基本機能に関する表現力豊かな文法設計されています。仮想ネットワーク機能（VNF）に対する研究が流行っている背景のなか、P4言語用いてソフトウェアにおけるパケット分類の実装を研究する必要がある。本研究では、今までネットワークのパケット分類に関するアルゴリズムがP4言語文法による実装を検討する。P4抽象転送モデル中で利用可能なプログラミングフローを議論し、パケット分類の改善に適しているデータ構造を示した。また、異なるアルゴリズムとデータ構造を用いて、P4ソースコードからコンパイルされたソフトウェアルーターの性能評価を行った。電気通信大学201

Partial replication in the database state machine

Tese de Doutoramento em Informática - Ramo do Conhecimento em Tecnologias da ProgramaçãoEnterprise information systems are nowadays commonly structured as multi-tier architectures and invariably built on top of database management systems responsible for the storage and provision of the entire business data. Database management systems therefore play a vital role in today’s organizations, from their reliability and availability directly depends the overall system dependability. Replication is a well known technique to improve dependability. By maintaining consistent replicas of a database one can increase its fault tolerance and simultaneously improve system’s performance by splitting the workload among the replicas. In this thesis we address these issues by exploiting the partial replication of databases. We target large scale systems where replicas are distributed across wide area networks aiming at both fault tolerance and fast local access to data. In particular, we envision information systems of multinational organizations presenting strong access locality in which fully replicated data should be kept to a minimum and a judicious placement of replicas should be able to allow the full recovery of any site in case of failure. Our research departs from work on database replication algorithms based on group communication protocols, in detail, multi-master certification-based protocols. At the core of these protocols resides a total order multicast primitive responsible for establishing a total order of transaction execution. A well known performance optimization in local area networks exploits the fact that often the definitive total order of messages closely following the spontaneous network order, thus making it possible to optimistically proceed in parallel with the ordering protocol. Unfortunately, this optimization is invalidated in wide area networks, precisely when the increased latency would make it more useful. To overcome this we present a novel total order protocol with optimistic delivery for wide area networks. Our protocol uses local statistic estimates to independently order messages closely matching the definitive one thus allowing optimistic execution in real wide area networks. Handling partial replication within a certification based protocol is also particularly challenging as it directly impacts the certification procedure itself. Depending on the approach, the added complexity may actually defeat the purpose of partial replication. We devise, implement and evaluate two variations of the Database State Machine protocol discussing their benefits and adequacy with the workload of the standard TPC-C benchmark.Os sistemas de informação empresariais actuais estruturam-se normalmente em arquitecturas de software multi-nível, e apoiam-se invariavelmente sobre um sistema de gestão de bases de dados para o armazenamento e aprovisionamento de todos os dados do negócio. A base de dado desempenha assim um papel vital, sendo a confiabilidade do sistema directamente dependente da sua fiabilidade e disponibilidade. A replicação é uma das formas de melhorar a confiabilidade. Garantindo a coerência de um conjunto de réplicas da base de dados, é possível aumentar simultaneamente a sua tolerância a faltas e o seu desempenho, ao distribuir as tarefas a realizar pelas várias réplicas não sobrecarregando apenas uma delas. Nesta tese, propomos soluções para estes problemas utilizando a replicação parcial das bases de dados. Nos sistemas considerados, as réplicas encontram-se distribuídas numa rede de larga escala, almejando-se simultaneamente obter tolerância a faltas e garantir um acesso local rápido aos dados. Os sistemas propostos têm como objectivo adequarem-se às exigências dos sistemas de informação de multinacionais em que em cada réplica existe uma elevada localidade dos dados acedidos. Nestes sistemas, os dados replicados em todas as réplicas devem ser apenas os absolutamente indispensáveis, e a selecção criteriosa dos dados a colocar em cada réplica, deve permitir em caso de falha a reconstrução completa da base de dados. Esta investigação tem como ponto de partida os protocolos de replicação de bases de dados utilizando comunicação em grupo, em particular os baseados em certificação e execução optimista por parte de qualquer uma das réplicas. O mecanismo fundamental deste tipo de protocolos de replicação é a primitiva de difusão com garantia de ordem total, utilizada para definir a ordem de execução das transacções. Uma optimização normalmente utilizada pelos protocolos de ordenação total é a utilização da ordenação espontânea da rede como indicador da ordem das mensagens, e usar esta ordem espontânea para processar de forma optimista as mensagens em paralelo com a sua ordenação. Infelizmente, em redes de larga escala a espontaneidade de rede é praticamente residual, inviabilizando a utilização desta optimização precisamente no cenário em que a sua utilização seria mais vantajosa. Para contrariar esta adversidade propomos um novo protocolo de ordenação total com entrega optimista para redes de larga escala. Este protocolo utiliza informação estatística local a cada processo para "produzir" uma ordem espontânea muito mais coincidente com a ordem total obtida viabilizando a utilização deste tipo de optimizações em redes de larga escala. Permitir que protocolos de replicação de bases de dados baseados em certificação suportem replicação parcial coloca vários desafios que afectam directamente a forma com é executado o procedimento de certificação. Dependendo da abordagem à replicação parcial, a complexidade gerada pode até comprometer os propósitos da replicação parcial. Esta tese concebe, implementa e avalia duas variantes do protocolo da database state machine com suporte para replicação parcial, analisando os benefícios e adequação da replicação parcial ao teste padronizado de desempenho de bases de dados, o TPC-C.Fundação para a Ciência e a Tecnologia (FCT) - ESCADA (POSI/CHS/33792/2000)

Indexing for moving objects

Master'sMASTER OF SCIENC