Reproducibility in Biomedical Natural Language Processing

International audienceTo assess the state of reproducibility in biomedical Natural Language Processing, we analyze the proceedings of the 2016 BioNLP workshop to check for code and data availability. We find that only 21% of papers report experiments where both data and code are available, with moderate to substantial inter-annotator agreement. This suggests that reproducibility in biomedical NLP would benefit from increased attention and understanding