Fan, Wenfei

Jia, Xibei

Li, Jianzhong

Ma, Shuai

English

To accurately match records it is often necessary to utilize the semantics of the data. Functional dependencies (FDs) have proven useful in identifying tuples in a clean relation, based on the semantics of the data. For all the reasons that FDs and their inference are needed, it is also important to develop dependencies and their reasoning techniques for matching tuples from unreliable data sources. This paper investigates dependencies and their reasoning for record matching. (a) We introduce a class of matching dependencies (MDs) for specifying the semantics of data in unreliable relations, defined in terms of similarity metrics and a dynamic semantics. (b) We identify a special case of MDs, referred to as relative candidate keys (RCKs), to determine what attributes to compare and how to compare them when matching records across possibly different relations. (c) We propose a mechanism for inferring MDs, a departure from traditional implication analysis, such that when we cannot match records by comparing attributes that contain errors, we may still find matches by using other, more reliable attributes. (d) We provide an O(n2) time algorithm for inferring MDs, and an effective algorithm for deducing a set of RCKs from MDs. (e) We experimentally verify that the algorithms help matching tools efficiently identify keys at compile time for matching, blocking or windowing, and that the techniques effectively improve both the quality and efficiency of various record matching methods

Edinburgh Research Explorer

A systematic approach to automatic edit and imputation.

A theory for record linkage.

Advances in record-linkage methodology as applied to matching the 1985 census of Tampa Florida.

Automating the approximate record-matching process.

BigMatch: A program for extracting probable matches from a large ﬁle.

Candidate keys for relations.

Computational problems related to the design of normal form relational schemas.

Constraint-based entity matching.

Data Quality: Concepts, Methodologies and Techniques.

Data tables with similarity relations: Functional dependencies, complete rules and non-redundant bases.

Declarative data cleaning: Language, model and algorithms.

Dependencies revisited for improving data quality.

Duplicate record detection: A survey.

Eliminating fuzzy duplicates in data warehouses.

Entity identiﬁcation in database integration.

Example-driven design of efﬁcient record matching queries.

Foundations of Databases.

iMAP: Discovering complex mappings between database schemas.

Industryscale duplicate detection.

Interactive deduplication using active learning.

Key issues for master data management.

Large-scale deduplication with constraints using Dedupalog. In

Learning to match and cluster large high-dimensional data sets for data integration.

Reasoning about Record Matching Rules

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer