Order dependency in the relational model

AbstractThe relational model is formally extended to include fixed orderings on attribute domains. A new constraint, called order dependency, is then introduced to incorporate semantic information involving these orderings. It is shown that this constraint can be applied to enhance the efficiency of an implemented database. The thrust of the paper is to study logical implication for order dependency. The main theoretical results consist in (i) introducing a formalism analogous to propositional calculus for analyzing order dependency, (ii) exhibiting a sound and complete set of inference rules for order dependency, and (iii) demonstrating that determining logical implication for order dependency is co-NP-complete. It is also shown that there are sets of order dependencies for which no Armstrong relations exist

Characterization of order-like dependencies with formal concept analysis

Functional Dependencies (FDs) play a key role in many fields of the relational database model, one of the most widely used database systems. FDs have also been applied in data analysis, data quality, knowl- edge discovery and the like, but in a very limited scope, because of their fixed semantics. To overcome this limitation, many generalizations have been defined to relax the crisp definition of FDs. FDs and a few of their generalizations have been characterized with Formal Concept Analysis which reveals itself to be an interesting unified framework for charac- terizing dependencies, that is, understanding and computing them in a formal way. In this paper, we extend this work by taking into account order-like dependencies. Such dependencies, well defined in the database field, consider an ordering on the domain of each attribute, and not sim- ply an equality relation as with standard FDs.Peer ReviewedPostprint (published version

characterization of order-like dependencies with formal concept analysis

Functional Dependencies (FDs) play a key role in many fields of the relational database model, one of the most widely used database systems. FDs have also been applied in data analysis, data quality, knowledge discovery and the like, but in a very limited scope, because of their fixed semantics. To overcome this limitation, many generalizations have been defined to relax the crisp definition of FDs. FDs and a few of their generalizations have been characterized with Formal Concept Analysis which reveals itself to be an interesting unified framework for characterizing dependencies, that is, understanding and computing them in a formal way. In this paper, we extend this work by taking into account order-like dependencies. Such dependencies, well defined in the database field, consider an ordering on the domain of each attribute, and not simply an equality relation as with standard FDsPostprint (published version

Contributions to the Formalization of Order-like Dependencies using FCA

Abstract. Functional Dependencies (FDs) play a key role in many fields of the relational database model, one of the most widely used database systems. FDs have also been applied in data analysis, data quality, knowledge discovery and the like, but in a very limited scope, because of their fixed semantics. To overcome this limitation, many generalizations have been defined to relax the crisp definition of FDs. FDs and a few of their generalizations have been characterized with Formal Concept Analysis which reveals itself to be an interesting unified framework for characterizing dependencies, that is, understanding and computing them in a formal way. In this paper, we extend this work by taking into account order-like dependencies. Such dependencies, well defined in the database field, consider an ordering on the domain of each attribute, and not simply an equality relation as with standard FDs

Characterization of Order-like Dependencies with Formal Concept Analysis

Abstract. Functional Dependencies (FDs) play a key role in many fields of the relational database model, one of the most widely used database systems. FDs have also been applied in data analysis, data quality, knowledge discovery and the like, but in a very limited scope, because of their fixed semantics. To overcome this limitation, many generalizations have been defined to relax the crisp definition of FDs. FDs and a few of their generalizations have been characterized with Formal Concept Analysis which reveals itself to be an interesting unified framework for characterizing dependencies, that is, understanding and computing them in a formal way. In this paper, we extend this work by taking into account order-like dependencies. Such dependencies, well defined in the database field, consider an ordering on the domain of each attribute, and not simply an equality relation as with standard FDs

Profiling relational data: a survey

Profiling data to determine metadata about a given dataset is an important and frequent activity of any IT professional and researcher and is necessary for various use-cases. It encompasses a vast array of methods to examine datasets and produce metadata. Among the simpler results are statistics, such as the number of null values and distinct values in a column, its data type, or the most frequent patterns of its data values. Metadata that are more difficult to compute involve multiple columns, namely correlations, unique column combinations, functional dependencies, and inclusion dependencies. Further techniques detect conditional properties of the dataset at hand. This survey provides a classification of data profiling tasks and comprehensively reviews the state of the art for each class. In addition, we review data profiling tools and systems from research and industry. We conclude with an outlook on the future of data profiling beyond traditional profiling tasks and beyond relational databases

Semantic Order Compatibilities and Their Discovery

Ordered domains such as numbers and dates are common in real-life datasets. The SQL standard includes an ORDER BY clause to sort the results, and there has been research work on formalizing, reasoning about, and automatically discovering order dependencies among columns in a table. However, a crucial assumption made in research and practice is that the order over a column is syntactic: numbers are ordered numerically, strings lexicographically and dates chronologically. To the best of our knowledge, this work is the first to relax this assumption. We present a generalized definition of order compatibilities that allows semantic orders such as (low, medium, high) or (excellent, very good, good, average, poor). We show that in general, validating whether there exists a semantic order relationship between columns is NP-complete, with some tractable special cases. We give an algorithm to automatically discover semantic order relationships in the data, we provide examples of interesting orders found by our algorithm that were missed by existing algorithms, and we show that the NP-complete validation cases do not occur frequently in practice