Relational Learning over Dirty Data Using Data Constraints

Abstract

Real-world datasets are dirty and contain many errors. Examples of these issues are violations of integrity constraints, duplicates, and inconsistencies in representing data values and entities. Applying machine learning on dirty databases may lead to inaccurate results. Users have to spend a lot of time and effort repairing data errors and creating a clean learning database. Moreover, as the information required to fix these errors is not often available, there may be numerous possible clean versions for a dirty database. We propose DLearn, a novel relational learning system that learns directly over dirty databases effectively and efficiently without any preprocessing. DLearn leverages database constraints, such as functional dependency and matching dependency, to learn accurate relational models over inconsistent and heterogeneous data. Its learned models using the unique data properties represent patterns over all possible clean instances of the data in a usable form. Our empirical study indicates that DLearn learns accurate models over large real-world databases efficiently

    Similar works