8,794 research outputs found
How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering
Using deep learning models on small scale datasets would result in
overfitting. To overcome this problem, the process of pre-training a model and
fine-tuning it to the small scale dataset has been used extensively in domains
such as image processing. Similarly for question answering, pre-training and
fine-tuning can be done in several ways. Commonly reading comprehension models
are used for pre-training, but we show that other types of pre-training can
work better. We compare two pre-training models based on reading comprehension
and open domain question answering models and determine the performance when
fine-tuned and tested over BIOASQ question answering dataset. We find open
domain question answering model to be a better fit for this task rather than
reading comprehension model
Finding Answers from the Word of God: Domain Adaptation for Neural Networks in Biblical Question Answering
Question answering (QA) has significantly benefitted from deep learning
techniques in recent years. However, domain-specific QA remains a challenge due
to the significant amount of data required to train a neural network. This
paper studies the answer sentence selection task in the Bible domain and answer
questions by selecting relevant verses from the Bible. For this purpose, we
create a new dataset BibleQA based on bible trivia questions and propose three
neural network models for our task. We pre-train our models on a large-scale QA
dataset, SQuAD, and investigate the effect of transferring weights on model
accuracy. Furthermore, we also measure the model accuracies with different
answer context lengths and different Bible translations. We affirm that
transfer learning has a noticeable improvement in the model accuracy. We
achieve relatively good results with shorter context lengths, whereas longer
context lengths decreased model accuracy. We also find that using a more modern
Bible translation in the dataset has a positive effect on the task.Comment: The paper has been accepted at IJCNN 201
Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers
ChatGPT is a large language model developed by OpenAI. Despite its impressive
performance across various tasks, no prior work has investigated its capability
in the biomedical domain yet. To this end, this paper aims to evaluate the
performance of ChatGPT on various benchmark biomedical tasks, such as relation
extraction, document classification, question answering, and summarization. To
the best of our knowledge, this is the first work that conducts an extensive
evaluation of ChatGPT in the biomedical domain. Interestingly, we find based on
our evaluation that in biomedical datasets that have smaller training sets,
zero-shot ChatGPT even outperforms the state-of-the-art fine-tuned generative
transformer models, such as BioGPT and BioBART. This suggests that ChatGPT's
pre-training on large text corpora makes it quite specialized even in the
biomedical domain. Our findings demonstrate that ChatGPT has the potential to
be a valuable tool for various tasks in the biomedical domain that lack large
annotated data.Comment: Accepted by BioNLP@ACL 202
Evaluating BERT Embeddings for Text Classification in Bio-Medical Domain to Determine Eligibility of Patients in Clinical Trials
Clinical Trials are studies conducted by researchers in order to assess the impact of new medicine in terms of its efficacy and most importantly safety on human health. For any advancement in the field of medicine it is very important that clinical trials are conducted with right ethics supported by scientific evidence. Not all people who volunteer or participate in clinical trials are allowed to undergo the trials. Age, comorbidity and other health issues present in a patient can be a major factor to decide whether the profile is suitable or not for the trial. Profiles selected for clinical trials should be documented and also the profiles which were excluded. This research which took over a long time period conducted trials on 15,000 cancer drugs. Keeping track of so many trials, their outcomes and formulating a standard health guideline is easier said than done. In this paper, Text classification which is one of the primary assessment tasks in Natural Language Processing (NLP) is discussed. One of the most common problems in NLP, but it becomes complex when it is dealing with a specific domain like bio-medical which finds presence of quite a few jargons pertaining to the medical field. This paper proposes a framework with two major components comprising transformer architecture to produce embedding coupled with a text classifier. In the later section it is proved that pre-trained embeddings generated by BERT (Bidirectional Encoder Representations from Transformers) can perform as efficiently and achieve a better F1-score and accuracy than the current benchmark score which uses embeddings trained from the same dataset. The main contribution of this paper is the framework which can be extended to different bio-medical problems. The design can also be reused for different domains by fine-tuning. The framework also provides support for different optimization techniques like Mixed Precision, Dynamic Padding and Uniform Length Batching which improves performance by up to 3 times in GPU (Graphics Processing Unit) processors and by 60% in TPU (Tensor Processing Unit)
BAND: Biomedical Alert News Dataset
Infectious disease outbreaks continue to pose a significant threat to human
health and well-being. To improve disease surveillance and understanding of
disease spread, several surveillance systems have been developed to monitor
daily news alerts and social media. However, existing systems lack thorough
epidemiological analysis in relation to corresponding alerts or news, largely
due to the scarcity of well-annotated reports data. To address this gap, we
introduce the Biomedical Alert News Dataset (BAND), which includes 1,508
samples from existing reported news articles, open emails, and alerts, as well
as 30 epidemiology-related questions. These questions necessitate the model's
expert reasoning abilities, thereby offering valuable insights into the
outbreak of the disease. The BAND dataset brings new challenges to the NLP
world, requiring better disguise capability of the content and the ability to
infer important information. We provide several benchmark tasks, including
Named Entity Recognition (NER), Question Answering (QA), and Event Extraction
(EE), to show how existing models are capable of handling these tasks in the
epidemiology domain. To the best of our knowledge, the BAND corpus is the
largest corpus of well-annotated biomedical outbreak alert news with
elaborately designed questions, making it a valuable resource for
epidemiologists and NLP researchers alike
- …