Background: The quantity of documents being published requires researchers to
specialize to a narrower field, meaning that inferable connections between
publications (particularly from different domains) can be missed. This has given
rise to automatic literature based discovery (LBD). However, unless heavily
filtered, LBD generates more potential new knowledge than can be manually
verified and another form of selection is required before the results can be passed
onto a user. Since a large proportion of the automatically generated hidden
knowledge is valid but generally known, we investigate the hypothesis that non
trivial, interesting, hidden knowledge can be treated as an anomaly and identified
using anomaly detection approaches.
Results: Two experiments are conducted: (1) to avoid errors arising from
incorrect extraction of relations, the hypothesis is validated using manually
annotated relations appearing in a thesaurus, and (2) automatically extracted
relations are used to investigate the hypothesis on publication abstracts. These
allow an investigation of a potential upper bound and the detection of limitations
yielded by automatic relation extraction.
Conclusion: We apply one-class SVM and isolation forest anomaly detection
algorithms to a set of hidden connections to rank connections by identifying
outlying (interesting) ones and show that the approach increases the F1 measure
by a factor of 10 while greatly reducing the quantity of hidden knowledge to
manually verify. We also demonstrate the statistical significance of this result.
Keywords: literature based discovery; anomaly detection; unified medical
language syste
