11 research outputs found
ShARe/CLEF eHealth evaluation lab 2014, task 3: user-centred health information retrieval
This paper presents the results of task 3 of the ShARe/CLEF eHealth Evaluation Lab 2014. This evaluation lab focuses on improving access to medical information on the web. The task objective was to investigate the effect of using additional information such as a related discharge summary and external resources such as medical ontologies on the IR effectiveness, in a monolingual and in a multilingual context. The participants were allowed to submit up to seven runs for each language, one mandatory run using no additional information or external resources, and three each using or not using discharge summaries
Overview of the CLEF 2018 Consumer Health Search Task
This paper details the collection, systems and evaluation
methods used in the CLEF 2018 eHealth Evaluation Lab, Consumer Health Search (CHS) task (Task 3). This task investigates the effectiveness of search engines in providing access to medical information present on the Web for people that have no or little medical knowledge. The task aims to foster advances in the development of search technologies for Consumer Health Search by providing resources and evaluation methods to test and validate search systems. Built upon the the 2013-17 series of CLEF eHealth Information Retrieval tasks, the 2018 task considers
both mono- and multilingual retrieval, embracing the Text REtrieval Conference (TREC) -style evaluation process with a shared collection of documents and queries, the contribution of runs from participants and the subsequent formation of relevance assessments and evaluation of the participants submissions.
For this year, the CHS task uses a new Web corpus and a new set of queries compared to the previous years. The new corpus consists of Web pages acquired from the CommonCrawl and the new set of queries consists of 50 queries issued by the general public to the Health on the Net (HON) search services. We then manually translated the 50 queries to
French, German, and Czech; and obtained English query variations of the 50 original queries.
A total of 7 teams from 7 different countries participated in the 2018 CHS task: CUNI (Czech Republic), IMS Unipd (Italy), MIRACL (Tunisia), QUT (Australia), SINAI (Spain), UB-Botswana (Botswana), and UEvora (Portugal)
Overview of the CLEF 2018 Consumer Health Search Task
This paper details the collection, systems and evaluation
methods used in the CLEF 2018 eHealth Evaluation Lab, Consumer
Health Search (CHS) task (Task 3). This task investigates the effectiveness of search engines in providing access to medical information present
on the Web for people that have no or little medical knowledge. The task
aims to foster advances in the development of search technologies for
Consumer Health Search by providing resources and evaluation methods
to test and validate search systems. Built upon the the 2013-17 series
of CLEF eHealth Information Retrieval tasks, the 2018 task considers
both mono- and multilingual retrieval, embracing the Text REtrieval
Conference (TREC) -style evaluation process with a shared collection of
documents and queries, the contribution of runs from participants and
the subsequent formation of relevance assessments and evaluation of the
participants submissions.
For this year, the CHS task uses a new Web corpus and a new set of
queries compared to the previous years. The new corpus consists of Web
pages acquired from the CommonCrawl and the new set of queries consists of 50 queries issued by the general public to the Health on the Net
(HON) search services. We then manually translated the 50 queries to
French, German, and Czech; and obtained English query variations of
the 50 original queries.
A total of 7 teams from 7 different countries participated in the 2018 CHS
task: CUNI (Czech Republic), IMS Unipd (Italy), MIRACL (Tunisia),
QUT (Australia), SINAI (Spain), UB-Botswana (Botswana), and UEvora
(Portugal)
Adapting SMT Query Translation Reranker to New Languages in Cross-Lingual Information Retrieval
We investigate adaptation of a supervised machine learning model for reranking of query translations to new languages in the context of cross-lingual information retrieval. The model is trained to rerank multiple translations produced by a statistical machine translation system and optimize retrieval quality. The model features do not depend on the source language and thus allow the model to be trained on query translations coming from multiple languages. In this paper, we explore how this affects the final retrieval quality. The experiments are conducted on medical-domain test collection in English and multilingual queries (in Czech, German, French) from the CLEF eHealth Lab series 2013--2015.
We adapt our method to allow reranking of query translations for four new languages (Spanish, Hungarian, Polish, Swedish). The baseline approach, where a single model is trained for each source language on query translations from that language, is compared with a model co-trained on translations from the three original languages
Overview of the CLEF eHealth Evaluation Lab 2018
In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues
our evaluation resource building efforts around the easing and support of
patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual
setting. This yearâs lab offered three tasks: Task 1 on multilingual information extraction to extend from last yearâs task on French and English
corpora to French, Hungarian, and Italian; Task 2 on technologically
assisted reviews in empirical medicine building on last yearâs pilot task in English; and Task 3 on Consumer Health Search (CHS) in mono- and
multilingual settings that builds on the 2013â17 Information Retrieval
tasks. In total 28 teams took part in these tasks (14 in Task 1, 7 in Task
2 and 7 in Task 3). Herein, we describe the resources created for these
tasks, outline our evaluation methodology adopted and provide a brief
summary of participants of this yearâs challenges and results obtained.
As in previous years, the organizers have made data and tools associated
with the lab tasks available for future research and development
Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020
On behalf of the Program Committee, a very warm welcome to the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020). This edition of the conference is held in Bologna and organised by the University of Bologna. The CLiC-it conference series is an initiative of the Italian Association for Computational Linguistics (AILC) which, after six years of activity, has clearly established itself as the premier national forum for research and development in the fields of Computational Linguistics and Natural Language Processing, where leading researchers and practitioners from academia and industry meet to share their research results, experiences, and challenges
CUNI at the ShARe/CLEF eHealth Evaluation Lab 2014
This report describes the participation of the team of Charles University in Prague at the ShARe/CLEF eHealth Evaluation Lab in 2014
European Language Grid
This open access book provides an in-depth description of the EU project European Language Grid (ELG). Its motivation lies in the fact that Europe is a multilingual society with 24 official European Union Member State languages and dozens of additional languages including regional and minority languages. The only meaningful way to enable multilingualism and to benefit from this rich linguistic heritage is through Language Technologies (LT) including Natural Language Processing (NLP), Natural Language Understanding (NLU), Speech Technologies and language-centric Artificial Intelligence (AI) applications. The European Language Grid provides a single umbrella platform for the European LT community, including research and industry, effectively functioning as a virtual home, marketplace, showroom, and deployment centre for all services, tools, resources, products and organisations active in the field. Today the ELG cloud platform already offers access to more than 13,000 language processing tools and language resources. It enables all stakeholders to deposit, upload and deploy their technologies and datasets. The platform also supports the long-term objective of establishing digital language equality in Europe by 2030 â to create a situation in which all European languages enjoy equal technological support. This is the very first book dedicated to Language Technology and NLP platforms. Cloud technology has only recently matured enough to make the development of a platform like ELG feasible on a larger scale. The book comprehensively describes the results of the ELG project. Following an introduction, the content is divided into four main parts: (I) ELG Cloud Platform; (II) ELG Inventory of Technologies and Resources; (III) ELG Community and Initiative; and (IV) ELG Open Calls and Pilot Projects