2 research outputs found
Neural Summarization of Electronic Health Records
Hospital discharge documentation is among the most essential, yet
time-consuming documents written by medical practitioners. The objective of
this study was to automatically generate hospital discharge summaries using
neural network summarization models. We studied various data preparation and
neural network training techniques that generate discharge summaries. Using
nursing notes and discharge summaries from the MIMIC-III dataset, we studied
the viability of the automatic generation of various sections of a discharge
summary using four state-of-the-art neural network summarization models (BART,
T5, Longformer and FLAN-T5). Our experiments indicated that training
environments including nursing notes as the source, and discrete sections of
the discharge summary as the target output (e.g. "History of Present Illness")
improve language model efficiency and text quality. According to our findings,
the fine-tuned BART model improved its ROUGE F1 score by 43.6% against its
standard off-the-shelf version. We also found that fine-tuning the baseline
BART model with other setups caused different degrees of improvement (up to 80%
relative improvement). We also observed that a fine-tuned T5 generally achieves
higher ROUGE F1 scores than other fine-tuned models and a fine-tuned FLAN-T5
achieves the highest ROUGE score overall, i.e., 45.6. For majority of the
fine-tuned language models, summarizing discharge summary report sections
separately outperformed the summarization the entire report quantitatively. On
the other hand, fine-tuning language models that were previously instruction
fine-tuned showed better performance in summarizing entire reports. This study
concludes that a focused dataset designed for the automatic generation of
discharge summaries by a language model can produce coherent Discharge Summary
sections
Generative Benchmark Creation for Table Union Search
Data management has traditionally relied on synthetic data generators to
generate structured benchmarks, like the TPC suite, where we can control
important parameters like data size and its distribution precisely. These
benchmarks were central to the success and adoption of database management
systems. But more and more, data management problems are of a semantic nature.
An important example is finding tables that can be unioned. While any two
tables with the same cardinality can be unioned, table union search is the
problem of finding tables whose union is semantically coherent. Semantic
problems cannot be benchmarked using synthetic data. Our current methods for
creating benchmarks involve the manual curation and labeling of real data.
These methods are not robust or scalable and perhaps more importantly, it is
not clear how robust the created benchmarks are. We propose to use generative
AI models to create structured data benchmarks for table union search. We
present a novel method for using generative models to create tables with
specified properties. Using this method, we create a new benchmark containing
pairs of tables that are both unionable and non-unionable but related. We
thoroughly evaluate recent existing table union search methods over existing
benchmarks and our new benchmark. We also present and evaluate a new table
search methods based on recent large language models over all benchmarks. We
show that the new benchmark is more challenging for all methods than
hand-curated benchmarks, specifically, the top-performing method achieves a
Mean Average Precision of around 60%, over 30% less than its performance on
existing manually created benchmarks. We examine why this is the case and show
that the new benchmark permits more detailed analysis of methods, including a
study of both false positives and false negatives that were not possible with
existing benchmarks