This paper details materials, methods, results, and analyses of the Consumer Health Search Task of the
CLEF eHealth 2021 Evaluation Lab. This task investigates the effectiveness of information retrieval (IR)
approaches in providing access to medical information to laypeople. For this a TREC-style evaluation
methodology was applied: a shared collection of documents and queries is distributed, participants’
runs received, relevance assessments generated, and participants’ submissions evaluated. The task generated a new representative web corpus including web pages acquired from a 2021 CommonCrawl and
social media content from Twitter and Reddit, along with a new collection of 55 manually generated
layperson medical queries and their respective credibility, understandability, and topicality assessments
for returned documents. This year’s task focused on three subtask: (i) ad-hoc IR, (ii) weakly supervised
IR, and (iii) document credibility prediction. In total, 15 runs were submitted to the three subtasks: eight
addressed the ad-hoc IR task, three the weakly supervised IR challenge, and 4 the document credibility
prediction challenge. As in previous years, the organizers have made data and tools associated with the
task available for future research and development
