Expert system shell to reason on large amounts of data

The current data base management systems (DBMS's) do not provide a sophisticated environment to develop rule based expert systems applications. Some of the new DBMS's come with some sort of rule mechanism; these are active and deductive database systems. However, both of these are not featured enough to support full implementation based on rules. On the other hand, current expert system shells do not provide any link with external databases. That is, all the data are kept in the system working memory. Such working memory is maintained in main memory. For some applications the reduced size of the available working memory could represent a constraint for the development. Typically these are applications which require reasoning on huge amounts of data. All these data do not fit into the computer main memory. Moreover, in some cases these data can be already available in some database systems and continuously updated while the expert system is running. This paper proposes an architecture which employs knowledge discovering techniques to reduce the amount of data to be stored in the main memory; in this architecture a standard DBMS is coupled with a rule-based language. The data are stored into the DBMS. An interface between the two systems is responsible for inducing knowledge from the set of relations. Such induced knowledge is then transferred to the rule-based language working memory

Active database behaviour: the REFLEX approach

Modern day and new generation applications have more demanding requirements than traditional database management systems (DBMS) are able to support. Two of these requirements, timely responses to the change of database state and application domain knowledge stored within the database, are embodied within active database technology. Currently, there are a number of research prototype active database systems throughout the world. In order for an organisation to use any such prototype system, it may have to forsake existing products and resources and embark on substantial reinvestment in the new database products and associated resources and retraining costs. This approach would clearly be unfavourable as it is expensive both in terms of time and money. A more suitable approach would be to allow active behaviour to be added onto their existing systems. This scenario is addressed within this research. It investigates how best active behaviour can be augmented to existing DBMSs, so as to preserve the investments in an organisation's resources, by examining the following issues, (i.) what form the knowledge model should take, (ii.) should rules and events be modelled as first class objects, (iii.) how will the triggering events be specified, (iv.) how the user will interact with the system. Various design decisions were taken, which were investigated by implementation of a series of working prototypes, on the ONTOS DBMS platform. The resultant REFLEX model was successfully ported and adapted onto a second POET platform. The porting process uncovered some interesting issues regarding preconceived ideas about the portability of open systems

Photo annotation and retrieval through speech

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (p. 36).In this thesis I describe the development of a speech-based annotation and retrieval system for digital photographs. The system uses a client/server architecture which allows photographs to be captured and annotated on various clients, such as mobile camera phones or the web, and then processed, indexed and stored on networked servers. For speech-based retrieval we have developed a mixed grammar recognition approach which allows the speech recognition system to construct a single finite-state network combining context-free grammars for recognizing and parsing query carrier phrases and metadata phrases, with an unconstrained statistical n-gram model for recognizing free-form search terms. Experiments demonstrating successful retrieval of photographs using purely speech-based annotation and retrieval are presented.by Brennan P. Sherry.M.Eng

LogBase: A Scalable Log-structured Database System in the Cloud

Numerous applications such as financial transactions (e.g., stock trading) are write-heavy in nature. The shift from reads to writes in web applications has also been accelerating in recent years. Write-ahead-logging is a common approach for providing recovery capability while improving performance in most storage systems. However, the separation of log and application data incurs write overheads observed in write-heavy environments and hence adversely affects the write throughput and recovery time in the system. In this paper, we introduce LogBase - a scalable log-structured database system that adopts log-only storage for removing the write bottleneck and supporting fast system recovery. LogBase is designed to be dynamically deployed on commodity clusters to take advantage of elastic scaling property of cloud environments. LogBase provides in-memory multiversion indexes for supporting efficient access to data maintained in the log. LogBase also supports transactions that bundle read and write operations spanning across multiple records. We implemented the proposed system and compared it with HBase and a disk-based log-structured record-oriented system modeled after RAMCloud. The experimental results show that LogBase is able to provide sustained write throughput, efficient data access out of the cache, and effective system recovery.Comment: VLDB201

Stairwalker user manual

Geographical data are typically visualized using various information layers that are displayed over a map. Interactive exploration by zooming and panning actions needs real-time re-calculation. A common operation in calculating with multidimensional data is the computation of aggregates. For layers containing aggregated information derived from voluminous data sets, such real-time exploration is impossible using standard database technology. Calculations require too much time.\ud \ud The University of Twente has developed “Stairwalker”: database technology that accurately aggregates data so that they can geographically be explored in real-time. The technology is a plug-in to common open source technology.\ud \ud Its core is the pre-aggregate index: a database index that cleverly precalculates aggregation values such that it can obtain exact aggregation results from voluminous data with high performance. A fast calculation allows to fully recalculate the result for even the slightest movement of the map, such as a panning or zooming action, without loss of accuracy. Thanks to this indexing mechanism, we can provide a scalable real-time calculation: an order of magnitude larger dataset requires only one additional aggregation level.\ud \ud In geo data visualization, the ability to quickly develop new information layers is important. Although many solutions exist, there is a niche: the combination of visualizing aggregation information, interactive data exploration in real-time, Big Data, calculating exact numbers instead of approximations, and doing so with common open source technology. Our technology for the first time integrates all these features.\ud \ud Our research partners are the companies Arcadis and Nspyre. They both have struggled with this combination of requirements in many of their projects. Our database index technology is not specific to geographical data. It can be used with all types of multidimensional data. Visualization in business intelligence or eScience can also benefit from it.\ud \ud The company Arcadis developed an application for the DCMR Milieudienst Rijnmond based on the Stairwalk technology to investigate whether people send tweets about unpleasant odors as a possible signal of danger. This turns out not to be the case, probably because people think that nobody reads the tweets anyway. But if people have the idea that their complaining tweets are read, then tweets might be much more convenient than the reporting of unpleasant odors by telephone.\ud \ud This manual explains how to use Stairwalker. We first explain in Section 2 how to install the required components in order to have a basic running system. We then explain in Section 3 how to add databases and different kinds of datatypes to Geoserver, an open source server for sharing geospatial data.1 It is explained how to show and customize layers and views, but also how to adjust the system, for example, how to add dimensions or use different dimension types such as median. Finally, Section 4 explains how to extend the system

Integrity Control in Relational Database Systems - An Overview

This paper gives an overview of research regarding integrity control or integrity constraint handling in relational database management systems. The topic of constraint handling is discussed from two points of view. First, constraint handling is discussed by identifying a number of important research issues, and by treating each issue in detail. Second, a number of projects is described that have resulted in the realization of database management systems supporting integrity constraints; the various projects are compared with respect to a number of system characteristics. Together, both approaches give a broad overview of the state of the art in the field at this moment