1 research outputs found
A Search/Crawl Framework for Automatically Acquiring Scientific Documents
Despite the advancements in search engine features, ranking methods,
technologies, and the availability of programmable APIs, current-day
open-access digital libraries still rely on crawl-based approaches for
acquiring their underlying document collections. In this paper, we propose a
novel search-driven framework for acquiring documents for scientific portals.
Within our framework, publicly-available research paper titles and author names
are used as queries to a Web search engine. Next, research papers and sources
of research papers are identified from the search results using accurate
classification modules. Our experiments highlight not only the performance of
our individual classifiers but also the effectiveness of our overall
Search/Crawl framework. Indeed, we were able to obtain approximately 0.665
million research documents through our fully-automated framework using about
0.076 million queries. These prolific results position Web search as an
effective alternative to crawl methods for acquiring both the actual documents
and seed URLs for future crawls.Comment: 8 pages with references, 2 figure