A Markov-Based Update Policy for Constantly Changing Database Systems

In order to maximize the value of an organization\u27s data assets, it is important to keep data in its databases up-to-date. In the era of big data, however, constantly changing data sources make it a challenging task to assure data timeliness in enterprise systems. For instance, due to the high frequency of purchase transactions, purchase data stored in an enterprise resource planning system can easily become outdated, affecting the accuracy of inventory data and the quality of inventory replenishment decisions. Despite the importance of data timeliness, updating a database as soon as new data arrives is typically not optimal because of high update cost. Therefore, a critical problem in this context is to determine the optimal update policy for database systems. In this study, we develop a Markov decision process model, solved via dynamic programming, to derive the optimal update policy that minimizes the sum of data staleness cost and update cost. Based on real-world enterprise data, we conduct experiments to evaluate the performance of the proposed update policy in relation to benchmark policies analyzed in the prior literature. The experimental results show that the proposed update policy outperforms fixed interval update policies and can lead to significant cost savings

Addressing Timeliness/Accuracy/Cost Tradeoffs in Information Collection for Dynamic Environments

In this paper, we focus on addressing the tradeoffs between timeliness, accuracy and cost for applications requiring real-time information collection in distributed real-time environments. In this scenario, information consumers require data from information sources at varying levels of accuracy and timeliness. To accommodate the diverse characteristics of information sources and varying requirements from information consumers, we use an information mediator to coordinate and facilitate communication between information sources and consumers. We develop algorithms for real-time request scheduling and directory service maintenance and compare our techniques with several other proposed strategies. Our studies indicate that the judicious composition of our proposed intelligent policies can improve the overall efficiency of the system. Furthermore, the proposed policies perform very well as the system scales in the number of information sources and consumer requests

Data freshness and data accuracy :a state of the art

In a context of Data Integration Systems (DIS) providing access to large amounts of data extracted and integrated from autonomous data sources, users are highly concerned about data quality. Traditionally, data quality is characterized via multiple quality factors. Among the quality dimensions that have been proposed in the literature, this report analyzes two main ones: data freshness and data accuracy. Concretely, we analyze the various definitions of both quality dimensions, their underlying metrics and the features of DIS that impact their evaluation. We present a taxonomy of existing works proposed for dealing with both quality dimensions in several kinds of DIS and we discuss open research problems