Evolutionary Algorithms for Query Op-timization in Distributed Database Sys-tems: A review

Evolutionary Algorithms are bio-inspired optimization problem-solving approaches that exploit principles of biological evolution. , such as natural selection and genetic inheritance. This review paper provides the application of evolutionary and swarms intelligence based query optimization strategies in Distributed Database Systems. The query optimization in a distributed environment is challenging task and hard problem. However, Evolutionary approaches are promising for the optimization problems. The problem of query optimization in a distributed database environment is one of the complex problems. There are several techniques which exist and are being used for query optimization in a distributed database. The intention of this research is to focus on how bio-inspired computational algorithms are used in a distributed database environment for query optimization. This paper provides working of bio-inspired computational algorithms in distributed database query optimization which includes genetic algorithms, ant colony algorithm, particle swarm optimization and Memetic Algorithms

Distributed query optimization

The need for the distributed systems has been determined by the type of business developed by companies with offices geographically distributed where the specific organizational structure promotes a decentralized business model. This paper describes the techniques and concepts of system architecture for distributed database management systems, followed by the presentation of implementation phases involved when dealing with the distributed queries across distributed systems. The goal of query optimization is to determine the most efficient way to execute a query in a distributed environment, by obtaining a lower system response time and also by minimizing the query execution time. For this, we will analyze the factors that influence the ways to execute a query and we will also review the available strategies to optimize the distributed query execution.architecture, distributed queries, optimization, strategies.

Algebraic optimization of recursive queries

Over the past few years, much attention has been paid to deductive databases. They offer a logic-based interface, and allow formulation of complex recursive queries. However, they do not offer appropriate update facilities, and do not support existing applications. To overcome these problems an SQL-like interface is required besides a logic-based interface.\ud \ud In the PRISMA project we have developed a tightly-coupled distributed database, on a multiprocessor machine, with two user interfaces: SQL and PRISMAlog. Query optimization is localized in one component: the relational query optimizer. Therefore, we have defined an eXtended Relational Algebra that allows recursive query formulation and can also be used for expressing executable schedules, and we have developed algebraic optimization strategies for recursive queries. In this paper we describe an optimization strategy that rewrites regular (in the context of formal grammars) mutually recursive queries into standard Relational Algebra and transitive closure operations. We also describe how to push selections into the resulting transitive closure operations.\ud \ud The reason we focus on algebraic optimization is that, in our opinion, the new generation of advanced database systems will be built starting from existing state-of-the-art relational technology, instead of building a completely new class of systems

Dynamic Optimization and Migration of Continuous Queries Over Data Streams

Continuous queries process real-time streaming data and output results in streams for a wide range of applications. Due to the fluctuating stream characteristics, a streaming database system needs to dynamically adapt query execution. This dissertation proposes novel solutions to continuous query adaptation in three core areas, namely dynamic query optimization, dynamic plan migration and partitioned query adaptation. Runtime query optimization needs to efficiently generate plans that satisfy both CPU and memory resource constraints. Existing work focus on minimizing intermediate query results, which decreases memory and CPU usages simultaneously. However, doing so cannot assure that both resource constraints are being satisfied, because memory and CPU can be either positively or negatively correlated. This part of the dissertation proposes efficient optimization strategies that utilize both types of correlations to search the entire query plan space in polynomial time when a typical exhaustive search would take at least exponential time. Extensive experimental evaluations have demonstrated the effectiveness of the proposed strategies. Dynamic plan migration is concerned with on-the-fly transition from one continuous plan to a semantically equivalent yet more efficient plan. It is a must to guarantee the continuation and repeatability of dynamic query optimization. However, this research area has been largely neglected in the current literature. The second part of this dissertation proposes migration strategies that dynamically migrate continuous queries while guaranteeing the integrity of the query results, meaning there are no missing, duplicate or incorrect results. The extensive experimental evaluations show that the proposed strategies vary significantly in terms of output rates and memory usages given distinct system configurations and stream workloads. Partitioned query processing is effective to process continuous queries with large stateful operators in a distributed system. Dynamic load redistribution is necessary to balance uneven workload across machines due to changing stream properties. However, existing solutions generally assume static query plans without runtime query optimization. This part of the dissertation evaluates the benefits of applying query optimization in partitioned query processing and shows dramatic performance improvement of more than 300%. Several load balancing strategies are then proposed to consider the heterogeneity of plan shapes across machines caused by dynamic query optimization. The effectiveness of the proposed strategies is analyzed through extensive experiments using a cluster

Data pre-processing:Case of sensor data consistency based on Bi-temporal concepts

The volume, velocity, variety, veracity and value of data currently produced and consumed by different types of information systems turned big Data into a phenomena of study. For data variety, temporal data commonly represents a source of potential inconsistency. This paper reports on a research endeavor for treating the problem of how to minimize inconsistencies in temporal databases due to unavailability of big data. This problem often occurs in situations where a same query is executed on the same data set at different points in time. To address this issue, we propose query optimization strategies based on query transformation and rewriting rules, to amend data consistency in temporal databases. We validate these strategies proposed via case scenario in sensor data analysis, and via manual data input, both for local and distributed query environments

NEW DYNAMIC QUERY OPTIMIZATION TECHNIQUE IN RELATIONAL DATABASE MANAGEMENT SYSTEMS

Query optimizer is an important component in the architecture of relational data base management system. This component is responsible for translating user submitted query into an efficient query evolution program which can be executed against the database. The present query evolution existing algorithm tries to find the best possible plan to execute a query with a minimum amount of time using mostly semi accurate statistical information (e.g. sizes of temporary relations, selectivity factors, and availability of resources). It is a static approach for generating optimal or close to optimal execution plan. Which in turn increases the execution cost of the query to reduce the execution cost of the query; I propose a new dynamic query optimization algorithm which is based on greedy dynamic programming algorithm uses randomized strategies and reduces the execution cost of the queries and system resources and also it works efficiently with distributed and centralized databases

The role of expert systems in federated distributed multi-database systems/Ince Levent

A shared information system is a series of computer systems interconnected by some kind of communication network. There are data repositories residing on each computer. These data repositories must somehow be integrated. The purpose for using distributed and multi-database systems is to allow users to view collections of data repositories as if they were a single entity. Multidatabase systems, better known as heterogeneous multidatabase systems, are characterized by dissimilar data models, concurrency and optimization strategies and access methods. Unlike homogenous systems, the data models that compose the global database can be based on different types of data models. It is not necessary that all participant databases use the same data model. Federated distributed database systems are a special case of multidatabase systems. They are completely autonomous and do not rely on the global data dictionary to process distributed queries. Processing distributed query requests in federated databases is very difficult since there are multiple independent databases with their own rules for query optimization, deadlock detection, and concurrency. Expert systems can play a role in this type of environment by supplying a knowledge base that contains rules for data object conversion, rules for resolving naming conflicts, and rules for exchanging data.http://archive.org/details/theroleofexperts109459362Turkish Navy author.Approved for public release; distribution is unlimited

Cologne: A Declarative Distributed Constraint Optimization Platform

This paper presents Cologne, a declarative optimization platform that enables constraint optimization problems (COPs) to be declaratively specified and incrementally executed in distributed systems. Cologne integrates a declarative networking engine with an off-theshelf constraint solver. We have developed the Colog language that combines distributed Datalog used in declarative networking with language constructs for specifying goals and constraints used in COPs. Cologne uses novel query processing strategies for processing Colog programs, by combining the use of bottom-up distributed Datalog evaluation with top-down goal-oriented constraint solving. Using case studies based on cloud and wireless network optimizations, we demonstrate that Cologne (1) can flexibly support a wide range of policy-based optimizations in distributed systems, (2) results in orders of magnitude less code compared to imperative implementations, and (3) is highly efficient with low overhead and fast convergence times

Dynamic strategy and Bloom filters in distributed query optimization.

Distributed query optimization is an important issue in distributed database management systems, since it can greatly affect the performance of the system. Many query optimization strategies have been proposed to minimize either the total cost or the response time. Most strategies are static in nature in the sense that their construction is based on database statistics which are obtained prior to query execution. In this thesis we investigate the use of dynamic strategies and better estimation techniques in query optimization. Bloom filters are used to obtain better estimates for query processing. Based on the above concept three algorithms are proposed, first using a pure dynamic strategy, the second using Bloom filters and the third using a combination of both. The performance of these algorithms with respect to total cost is compared against the AHY algorithm. The algorithms are executed against a large number of synthetically generated databases and queries. The experiments show a significant improvement over the AHY algorithm. The dynamic strategy shows an improvement over the static strategy and the combination heuristic shows a marginal improvement over the dynamic strategy. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis1996 .K35. Source: Masters Abstracts International, Volume: 37-01, page: 0286. Adviser: Joan Morrissey. Thesis (M.Sc.)--University of Windsor (Canada), 1996