10 research outputs found
Advancing Italian Biomedical Information Extraction with Large Language Models: Methodological Insights and Multicenter Practical Application
The introduction of computerized medical records in hospitals has reduced
burdensome operations like manual writing and information fetching. However,
the data contained in medical records are still far underutilized, primarily
because extracting them from unstructured textual medical records takes time
and effort. Information Extraction, a subfield of Natural Language Processing,
can help clinical practitioners overcome this limitation, using automated
text-mining pipelines. In this work, we created the first Italian
neuropsychiatric Named Entity Recognition dataset, PsyNIT, and used it to
develop a Large Language Model for this task. Moreover, we conducted several
experiments with three external independent datasets to implement an effective
multicenter model, with overall F1-score 84.77%, Precision 83.16%, Recall
86.44%. The lessons learned are: (i) the crucial role of a consistent
annotation process and (ii) a fine-tuning strategy that combines classical
methods with a "few-shot" approach. This allowed us to establish methodological
guidelines that pave the way for future implementations in this field and allow
Italian hospitals to tap into important research opportunities