Fan, Wenfei

Li, Jianzhong

Ma, Shuai

Tang, Nan

Yu, Wenyuan

English

A variety of integrity constraints have been studied for data cleaning. While these constraints can detect the presence of errors, they fall short of guiding us to correct the errors. Indeed, data repairing based on these constraints may notfind certain fixes that are absolutely correct, and worse, may introduce new errors when repairing the data. We propose a method for finding certain fixes, based on master data, a notion of certain regions, and a class of editing rules. A certain region is a set of attributes that are assured correct by the users. Given a certain region and master data, editing rules tell us what attributes to fix and how to update them. We show how the method can be used in data monitoring and enrichment. We develop techniques for reasoning about editing rules, to decide whether they lead to a unique fix and whether they are able to fix all the attributes in a tuple, relative to master data and a certain region. We also provide an algorithm to identify minimal certain regions, such that a certain fix is warranted by editing rules and master data as long as one of the regions is correct. We experimentally verify the effectiveness and scalability of the algorithm

Edinburgh Research Explorer

Towards certain fixes with editing rules and master data

A variety of integrity constraints have been studied for data cleaning. While these constraints can detect the presence of errors, they fall short of guiding us to correct the errors. Indeed, data repairing based on these constraints may notfind certain xes that are absolutely correct, and worse, may introduce new errors when repairing the data. We propose a method for finding certain fixes, based on master data, a notion of certain regions, and a class of editing rules. Acertain region is a set of attributes that are assured correct by the users. Given a certain region and master data, editing rules tell us what attributes to fix and how to update them. We show how the method can be used in data monitoringand enrichment. We develop techniques for reasoning about editing rules, to decide whether they lead to a unique fix and whether they are able to fix all the attributes in a tuple, relative to master data and a certain region. We also provide an algorithm to identify minimal certain regions, such that a certain fix is warranted by editing rules and master data as long as one of the regions is correct. We experimentally verify the effectiveness and scalability of the algorithm

A costbased model and eﬀective heuristic for repairing constraints by value modiﬁcation.

A model for generalized edit and imputation of survey data.

A systematic approach to automatic edit and imputation.

Active database systems: triggers and rules for advanced database processing.

Approximating minimum keys and optimal substructure screens.

Approximation Algorithms.

Computers and Intractability: A Guide to the Theory of NP-Completeness.

Conditional functional dependencies for capturing data inconsistencies.

Consistent query answers in inconsistent databases.

Data cleansing as a transient service.

Data Quality and Record Linkage Techniques.

Data quality and the bottom line: Achieving business success through a commitment to high quality data. The Data Warehousing Institute,

Database repairing using updates.

Discovering data quality rules.

Extending dependencies with conditions.

Forecast: Data quality tools,

Improving data quality: Consistency and accuracy.

Information service patterns, part 3: Data cleansing pattern. IBM,

Minimal-change integrity maintenance using tuple deletions.

New algorithms for enumerating all maximal cliques.

On approximating optimum repairs for functional dependency violations.

On generating all maximal independent sets.

On generating near-optimal tableaux for conditional functional dependencies.

Reasoning about record matching rules.

Robust and eﬃcient fuzzy match for online data cleaning.

The complexity of enumeration and reliability problems.

The impact of poor data quality on the typical enterprise.

Towards Certain Fixes with Editing Rules and Master Data

https://www.pure.ed.ac.uk/ws/files/17895730/Fan_Li_ET_AL_2007Towards_certain_fixes_with_editing_rules_and_master_data.pdf

Towards Certain Fixes with Editing Rules and Master Data

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer

Edinburgh Research Explorer