Routine development of objectively derived search strategies

BACKGROUND:Over the past few years, information retrieval has become more and more professionalized and information specialists are considered full members of a research team conducting systematic reviews. Research groups preparing systematic reviews and clinical practice guidelines have been the driving force in the development of search strategies, but open questions remain regarding the transparency of the development process and the available resources. An empirically guided approach to the development of a search strategy provides a way to increase transparency and efficiency.METHODS:The aim of this paper is to describe the empirically guided development process for search strategies as applied by the German Institute for Quality and Efficiency in Health Care (Institut fur Qualitat und Wirtschaftlichkeit im Gesundheitswesen, IQWiG). This strategy consists of the following steps: generation of a test set, as well as development, validation, and standardized documentation of the search strategy.RESULTS:We illustrated our approach by means of an example, i.e. a search for literature on brachytherapy in patients with prostate cancer. For this purpose, a test set was generated including a total of 38 references from 3 systematic reviews. The development set for the generation of the strategy included 25 references. After application of text-analytic procedures a strategy was developed including all references of the development set. In order to test the search strategy on an independent set of references, the remaining 13 references from the test set (validation set) were used; the validation set was also completely identified.DISCUSSION:Our conclusion is that an objectively derived approach, similar to that used in search filter development, is a feasible way to develop and validate reliable search strategies. Besides creating high-quality strategies, the widespread application of this approach would result in a substantial increase in the transparency of the development process of search strategies