2 research outputs found
A Study of Realtime Summarization Metrics
Unexpected news events, such as natural disasters or other human tragedies, create a large volume of dynamic text data from official news media as well as less formal social media. Automatic real-time text summarization has become an important tool for quickly transforming this overabundance of text into clear, useful information for end-users including affected individuals, crisis responders, and interested third parties. Despite the importance of real-time summarization systems, their evaluation is not well understood as classic methods for text summarization are inappropriate for real-time and streaming conditions.
The TREC 2013-2015 Temporal Summarization (TREC-TS) track was one of the first evaluation campaigns to tackle the challenges of real-time summarization evaluation, introducing new metrics, ground-truth generation methodology and dataset. In this paper, we present a study of TREC-TS track evaluation methodology, with the aim of documenting its design, analyzing its effectiveness, as well as identifying improvements and best practices for the evaluation of temporal summarization systems
Recommended from our members
Development of a Novel Tablet-based Approach to Reduce HIV Stigma among Healthcare Staff in India.
Although stigma is considered to be one of the major barriers to reducing the AIDS epidemic in India, efforts to reduce stigma have not been sufficiently examined. In response, a partially computer-administered three-session stigma reduction intervention was developed and is currently being tested. This paper describes the technological design, development, implementation, and management of these in-person tablet-administered assessment and intervention sessions that are being used to evaluate the efficacy of this innovative stigma reduction intervention among nursing students and ward attendants in India