1 research outputs found
The Claire French Dialogue Dataset
We present the Claire French Dialogue Dataset (CFDD), a resource created by
members of LINAGORA Labs in the context of the OpenLLM France initiative. CFDD
is a corpus containing roughly 160 million words from transcripts and stage
plays in French that we have assembled and publicly released in an effort to
further the development of multilingual, open source language models. This
paper describes the 24 individual corpora of which CFDD is composed and
provides links and citations to their original sources. It also provides our
proposed breakdown of the full CFDD dataset into eight categories of subcorpora
and describes the process we followed to standardize the format of the final
dataset. We conclude with a discussion of similar work and future directions