15 research outputs found
SC-Block: Supervised Contrastive Blocking within Entity Resolution Pipelines
The goal of entity resolution is to identify records in multiple datasets
that represent the same real-world entity. However, comparing all records
across datasets can be computationally intensive, leading to long runtimes. To
reduce these runtimes, entity resolution pipelines are constructed of two
parts: a blocker that applies a computationally cheap method to select
candidate record pairs, and a matcher that afterwards identifies matching pairs
from this set using more expensive methods. This paper presents SC-Block, a
blocking method that utilizes supervised contrastive learning for positioning
records in the embedding space, and nearest neighbour search for candidate set
building. We benchmark SC-Block against eight state-of-the-art blocking
methods. In order to relate the training time of SC-Block to the reduction of
the overall runtime of the entity resolution pipeline, we combine SC-Block with
four matching methods into complete pipelines. For measuring the overall
runtime, we determine candidate sets with 99.5% pair completeness and pass them
to the matcher. The results show that SC-Block is able to create smaller
candidate sets and pipelines with SC-Block execute 1.5 to 2 times faster
compared to pipelines with other blockers, without sacrificing F1 score.
Blockers are often evaluated using relatively small datasets which might lead
to runtime effects resulting from a large vocabulary size being overlooked. In
order to measure runtimes in a more challenging setting, we introduce a new
benchmark dataset that requires large numbers of product offers to be blocked.
On this large-scale benchmark dataset, pipelines utilizing SC-Block and the
best-performing matcher execute 8 times faster than pipelines utilizing another
blocker with the same matcher reducing the runtime from 2.5 hours to 18
minutes, clearly compensating for the 5 minutes required for training SC-Block
The Battleship Approach to the Low Resource Entity Matching Problem
Entity matching, a core data integration problem, is the task of deciding
whether two data tuples refer to the same real-world entity. Recent advances in
deep learning methods, using pre-trained language models, were proposed for
resolving entity matching. Although demonstrating unprecedented results, these
solutions suffer from a major drawback as they require large amounts of labeled
data for training, and, as such, are inadequate to be applied to low resource
entity matching problems. To overcome the challenge of obtaining sufficient
labeled data we offer a new active learning approach, focusing on a selection
mechanism that exploits unique properties of entity matching. We argue that a
distributed representation of a tuple pair indicates its informativeness when
considered among other pairs. This is used consequently in our approach that
iteratively utilizes space-aware considerations. Bringing it all together, we
treat the low resource entity matching problem as a Battleship game, hunting
indicative samples, focusing on positive ones, through awareness of the latent
space along with careful planning of next sampling iterations. An extensive
experimental analysis shows that the proposed algorithm outperforms
state-of-the-art active learning solutions to low resource entity matching, and
although using less samples, can be as successful as state-of-the-art fully
trained known algorithms
Product Attribute Value Extraction using Large Language Models
E-commerce applications such as faceted product search or product comparison
are based on structured product descriptions like attribute/value pairs. The
vendors on e-commerce platforms do not provide structured product descriptions
but describe offers using titles or descriptions. To process such offers, it is
necessary to extract attribute/value pairs from textual product attributes.
State-of-the-art attribute/value extraction techniques rely on pre-trained
language models (PLMs), such as BERT. Two major drawbacks of these models for
attribute/value extraction are that (i) the models require significant amounts
of task-specific training data and (ii) the fine-tuned models face challenges
in generalizing to attribute values not included in the training data. This
paper explores the potential of large language models (LLMs) as a training
data-efficient and robust alternative to PLM-based attribute/value extraction
methods. We consider hosted LLMs, such as GPT-3.5 and GPT-4, as well as
open-source LLMs based on Llama2. We evaluate the models in a zero-shot
scenario and in a scenario where task-specific training data is available. In
the zero-shot scenario, we compare various prompt designs for representing
information about the target attributes of the extraction. In the scenario with
training data, we investigate (i) the provision of example attribute values,
(ii) the selection of in-context demonstrations, and (iii) the fine-tuning of
GPT-3.5. Our experiments show that GPT-4 achieves an average F1-score of 85% on
the two evaluation datasets while the best PLM-based techniques perform on
average 5% worse using the same amount of training data. GPT-4 achieves a 10%
higher F1-score than the best open-source LLM. The fine-tuned GPT-3.5 model
reaches a similar performance as GPT-4 while being significantly more
cost-efficient
Generative Benchmark Creation for Table Union Search
Data management has traditionally relied on synthetic data generators to
generate structured benchmarks, like the TPC suite, where we can control
important parameters like data size and its distribution precisely. These
benchmarks were central to the success and adoption of database management
systems. But more and more, data management problems are of a semantic nature.
An important example is finding tables that can be unioned. While any two
tables with the same cardinality can be unioned, table union search is the
problem of finding tables whose union is semantically coherent. Semantic
problems cannot be benchmarked using synthetic data. Our current methods for
creating benchmarks involve the manual curation and labeling of real data.
These methods are not robust or scalable and perhaps more importantly, it is
not clear how robust the created benchmarks are. We propose to use generative
AI models to create structured data benchmarks for table union search. We
present a novel method for using generative models to create tables with
specified properties. Using this method, we create a new benchmark containing
pairs of tables that are both unionable and non-unionable but related. We
thoroughly evaluate recent existing table union search methods over existing
benchmarks and our new benchmark. We also present and evaluate a new table
search methods based on recent large language models over all benchmarks. We
show that the new benchmark is more challenging for all methods than
hand-curated benchmarks, specifically, the top-performing method achieves a
Mean Average Precision of around 60%, over 30% less than its performance on
existing manually created benchmarks. We examine why this is the case and show
that the new benchmark permits more detailed analysis of methods, including a
study of both false positives and false negatives that were not possible with
existing benchmarks
Product Information Extraction using ChatGPT
Structured product data in the form of attribute/value pairs is the
foundation of many e-commerce applications such as faceted product search,
product comparison, and product recommendation. Product offers often only
contain textual descriptions of the product attributes in the form of titles or
free text. Hence, extracting attribute/value pairs from textual product
descriptions is an essential enabler for e-commerce applications. In order to
excel, state-of-the-art product information extraction methods require large
quantities of task-specific training data. The methods also struggle with
generalizing to out-of-distribution attributes and attribute values that were
not a part of the training data. Due to being pre-trained on huge amounts of
text as well as due to emergent effects resulting from the model size, Large
Language Models like ChatGPT have the potential to address both of these
shortcomings. This paper explores the potential of ChatGPT for extracting
attribute/value pairs from product descriptions. We experiment with different
zero-shot and few-shot prompt designs. Our results show that ChatGPT achieves a
performance similar to a pre-trained language model but requires much smaller
amounts of training data and computation for fine-tuning
SC-block: Supervised contrastive blocking within entity resolution pipelines
Millions of websites use the schema.org vocabulary to annotate structured data describing products, local businesses, or events within their HTML pages. Integrating schema.org data from the Semantic Web poses distinct requirements to entity resolution methods: (1) the methods must scale to millions of entity descriptions and (2) the methods must be able to deal with the heterogeneity that results from a large number of data sources. In order to scale to numerous entity descriptions, entity resolution methods combine a blocker for candidate pair selection and a matcher for the fine-grained comparison of the pairs in the candidate set. This paper introduces SC-Block, a blocking method that uses supervised contrastive learning to cluster entity descriptions in an embedding space. The embedding enables SC-Block to generate small candidate sets even for use cases that involve a large number of unique tokens within entity descriptions. To measure the effectiveness of blocking methods for Semantic Web use cases, we present a new benchmark, WDC-Block. WDC-Block requires blocking product offers from 3,259 e-shops that use the schema.org vocabulary. The benchmark has a maximum Cartesian product of 200 billion pairs of offers and a vocabulary size of 7 million unique tokens. Our experiments using WDC-Block and other blocking benchmarks demonstrate that SC-Block produces candidate sets that are on average 50% smaller than the candidate sets generated by competing blocking methods. Entity resolution pipelines that combine SC-Block with state-of-the-art matchers finish 1.5 to 4 times faster than pipelines using other blockers, without any loss in F1 score