8 research outputs found
The TUNA-REG challenge 2009 : overview and evaluation results
The TUNA-REGā09 Challenge was one
of the shared-task evaluation competitions
at Generation Challenges 2009. TUNAREGā
09 used data from the TUNA Corpus
of paired representations of entities
and human-authored referring expressions.
The shared task was to create systems
that generate referring expressions
for entities given representations of sets
of entities and their properties. Four
teams submitted six systems to TUNAREGā
09. We evaluated the six systems and
two sets of human-authored referring expressions
using several automatic intrinsic
measures, a human-assessed intrinsic evaluation
and a human task performance experiment.
This report describes the TUNAREG
task and the evaluation methods used,
and presents the evaluation results.peer-reviewe
Beyond DICE : measuring the quality of a referring expression
This paper discusses ways in which the similarity between the
contents of two referring expressions can be measured. Simi-
larity metrics of this kind are essential when expressions gen-
erated by an algoritm are compared against the ones produced
by human speakers, for example as part of an experiment in
which referring expressions are elicitated. We discuss argu-
ments for and against different metrics, taking our departure
from the well-known Dice metric.peer-reviewe
Reference and the facilitation of search in spatial domains
This is a pre-final version of the article, whose official publication is expected in the winter of 2013-14.Peer reviewedPreprin
The TUNA-REG Challenge 2009: Overview and Evaluation Results
The TUNA-REGā09 Challenge was one of the shared-task evaluation competitions at Generation Challenges 2009. TUNA-REGā09 used data from the TUNA Corpus of paired representations of entities and human-authored referring expressions. The shared task was to create systems that generate referring expressions for entities given representations of sets of entities and their properties. Four teams submitted six systems to TUNA-REGā09. We evaluated the six systems and two sets of human-authored referring expressions using several automatic intrinsic measures, a human-assessed intrinsic evaluation and a human task performance experiment. This report describes the TUNA-REG task and the evaluation methods used, and presents the evaluation results.