Clarke, John

Wentz, Reinhard

PubMed

Clarke, JM

Wentz, R

UCL Discovery

Evaluating “payback” on biomedical researchBiomedical funding decisions should beauditedEditor—Grant et al note that organisationsthat fund biomedical research assume thatthe research they support will lead to aneventual improvement in health.1 Becauseclinical guidelines represent one of the finallinks between basic research and actions toimprove health, they looked at which studieswere cited in guidelines.Although their analysis is a valuablemove away from the naive use and abuse ofcitation counts and impact factors, they mayhave attempted to bridge too great adistance in assessing which publications inthe serial peer reviewed literature were citedin guidelines. Guidelines should be based onsystematic reviews of all the studies relevantto particular clinical questions. They shouldnot be based on the biased subsets of reportsof primary research included in biblio›graphic databases or those that are suffi›ciently concise to be published in serialjournals.2It would be helpful if Grant et al wouldindicate the extent to which references tosystematic reviews were cited in the guide›lines they studied. The “payback” fromprimary studies might then be studied byassessing their contribution to these system›atic reviews. For example, was a primarystudy judged to be of sufficiently high qualityto have been included in a systematic reviewat all? If so, what contribution did it make tothe totality of the relevant evidence?Grant et al suggest that an alternative tothe retrospective approach that they usedfor assessing payback would be “to identify abody of basic research published some timeago and follow its subsequent knowledgeflow.” A more informative approach wouldbe to identify a body of basic researchfunded some time ago. Payback couldthen be assessed not only in terms ofwhether it led to an eventual improvementin health but also whether it was completedand published.Failed research and failure to publishsuccessful research are costs to the public.Yet I am not aware of any public or charit›able organisation that funds biomedicalresearch that routinely publishes audits of itsinvestment decisions using criteria such asthese.Iain Chalmers directorUK Cochrane Centre, NHS Research andDevelopment Programme, Oxford OX2 7LGichalmers@cochrane.co.uk1 Grant J, Cottrell R, Cluzeau F, Fawcett G. Evaluating ‘‘pay›back” on biomedical research from papers cited in clinicalguidelines: applied bibliometric study. BMJ 2000;320:1107›11. (22 April.)2 Clarke M, Chalmers I. Discussion sections in reports ofcontrolled trials published in general medical journals:islands in search of continents? JAMA 1998;280: 280›2.Authors’ replyEditor—Chalmers asks what proportion ofreferences cited in the guidelines we studiedwere systematic reviews. The answer is68/2501 (2.7%). We made this calculation byusing the keywords “systematic” (for system›atic review) and “meta” (for meta›analysis) inthe title of the publication. Interestingly,there was no difference between publica›tions in peer reviewed journals (56/2043;2.7%) and the so called grey literature(12/458; 2.6%). Although Chalmers arguesthat clinical guidelines should be based onsystematic reviews of the literature, thesedata show that authors of guidelines are cit›ing the primary research.The Cochrane Database of SystematicReviews could provide a useful resource inundertaking applied bibliometric studies.Chalmers notes that the prospective tracingof funded research, as opposed to that onlypublished, may be more informative inauditing the outcomes of public or charit›able funding of biomedical research. Weagree.Jonathan Grant policy adviserj.grant@wellcome.ac.ukRobert Cottrell policy officerGail Fawcett policy officerPolicy Unit, Wellcome Trust, London NW1 2BEFrançoise Cluzeau lecturer in health servicesHealth Care Evaluation Unit, St George’s HospitalMedical School, London SW17 0REPragmatic approach is effectivein evidence based health careEditor—We agree with Guyatt et al thatproviding evidence based answers to clinicalquestions requires intensive study and appli›cation if the process is seen in the context ofa systematic review of the original literature.1The main stumbling block remains the diffi›culty of constructing complex searchesappropriate for a range of potential sources.We suggest that many typical clinicalquestions can be answered by using a limitedrange of extracting, evaluating databases,which can be interrogated with simple twostep or three step search formulations. Typi›cally these sources contain several thousandreferences, as opposed to several millionson large databases such as Medline orEMBASE with their unfavourable signal tonoise ratio.The three sources that in our experiencehave a high yield of material related toevidence based health care are the clinicalqueries option in PubMed2; the CochraneLibrary with its four sections (systematicreviews, the CRD (Centre for Reviews andDissemination) database of reviews of effec›tiveness, the register of controlled trials, andthe NHS economic evaluation database)3;and the TRIP (turning research intopractice) database from the Centre forResearch Support, Cardiff.4These three databases typically retrievefewer than 10 references provided that twoor at most three relevant and discriminatingterms are selected for a simple search. Weoften suggest to trainees that they shouldformulate their searches as if they weresending a telegram: which two or threewords would you transmit to a colleague toensure that he or she can imagine the clini›cal question? Thus the question “Howefficient is a single dose of a steroid for out›patient croup?” suggests the search “croupand outpatient,” which identifies small setsAdvice to authorsWe prefer to receive all responses electronically,sent either directly to our website or to theeditorial office as email or on a disk. Processingyour letter will be delayed unless it arrives in anelectronic form.We are now posting all direct submissions toour website within 24 hours of receipt and ourintention is to post all other electronicsubmissions there as well. All responses will beeligible for publication in the paper journal.Responses should be under 400 words andrelate to articles published in the precedingmonth. They should include <5 references, in theVancouver style, including one to the BMJ articleto which they relate.We welcome illustrations.Please supply each author’s currentappointment and full address, and a phone orfax number or email address for thecorresponding author.We ask authors to declareany competing interest. Please send a stampedaddressed envelope if you would like to knowwhether your letter has been accepted or rejected.Letters will be edited and may be shortened.bmj.comletters@bmj.comLettersWebsite: bmj.comEmail: letters@bmj.com566 BMJ VOLUME 321 2 SEPTEMBER 2000 bmj.com( < 10) containing a target reference5 on anyof the three databases mentioned above.This pragmatic approach, although nosubstitute for systematic reviews for thoseundertaking more extensive searches, isinfluenced by William of Occam’s principleof “if in doubt keep it simple” and is a validoption for busy clinicians.John Clarke librarianReinhard Wentz trial search coordinator, CochraneInjuries Groupr.wentz@ich.ucl.ac.ukInstitute of Child Health, London WC1N 1EH1 Guyatt GH, Meade MO, Jaeschke RZ, Cook DJ, Haynes RB.Practitioners of evidence based care. BMJ 2000;320:954›5.(8 April.)2 Pubmed. www.ncbi.nlm.nih.gov:80/entrez/query/static/clinical.html3 www.update›software.com/clibhome/clib.htm [passwordrequired].4 www.ceres.uwcm.ac.uk/frameset.cfm?section = Trip.5 Geelhoed GC, Turner J, Macdonald WBG. Efficacy of asmall single dose of oral dexamethasone for outpatientcroup: a double blind placebo controlled clinical trial. BMJ1996;313:140›2.Evidence should be accessibleas well as relevantEditor—National policy on research anddevelopment encourages the delivery ofhealth care that is of proved efficacy andbased on research.1 Barker and Gilbert saythat evidence of clinical effectiveness has tobe relevant to health professionals for it tobe incorporated into clinical practice.2 Weconducted a survey of community nursesand community based professionals allied tomedicine (PAMs) employed by one eastLondon community trust to identify theareas of their work where they see evidenceas important.We adapted a postal questionnairesurvey that we used in 1997 among generalpractitioners and practice staff in teachingpractices.3 In all, 124 completed question›naires were returned (51%). Respondentswere asked for their views on the role of evi›dence and the sources of evidence that theyaccessed and were also asked to list up tothree areas where research evidence wouldbe relevant to their work.Eighty one per cent of the respondentsshowed a positive attitude to the use ofresearch evidence in the daily managementof patients and clients and in the planning ofservices. This compares with 90% of generalpractitioners in the 1997 survey. There weredifferences between these two groups in thesources of evidence that were accessed(table).Colleagues were used as a source of evi›dence by a similar proportion in each group,but community nurses and PAMs were lesslikely to consult experts or journals. Nor didthese professionals access the internet forevidence: 64% (77/121) reported never hav›ing used the internet for any purpose. Com›puter projects such as PRODIGY arebringing evidence based decision makinginto clinical settings.4 However, half of ourrespondents (58/117) reported having noaccess to a clinical computer system.Community nurses and PAMs are inter›ested in having access to evidence that is rel›evant to their work, especially in areas suchas community mental health interventions,appropriate timing of developmental checksin children, prevention of falls amongelderly people, pain management, interven›tion of health visitors in postnatal depres›sion, foot and leg ulcer care, and music andart therapy. Training courses in evidencebased health care need to consider the effec›tiveness of interventions in these areas. Weagree with Barker and Gilbert that theimpact of evidence based health caredepends on its relevance to the work ofpractitioners in the field, but practitionersalso need access to such evidence.M Falshaw research coordinatorm.falshaw@qmw.ac.ukY H Carter professorR W Gray research assistantDepartment of General Practice and Primary Care,St Bartholomew’s and the Royal London School ofMedicine and Dentistry, Queen Mary and WestfieldCollege, London E1 4NS1 Department of Health. National research and developmentstrategy for primary care. London: DoH, 1997.2 Barker J, Gilbert D. Evidence produced in evidence basedmedicine needs to be relevant. BMJ 2000;320:515. (19February.)3 Hagdrup N, Falshaw M, Gray RW, Carter Y. All members ofprimary care team are aware of importance of evidencebased medicine. BMJ 1998;317:282.4 Purvess I. PRODIGY: implementing clinical guidelinesusing computers. Br J Gen Pract 1998;48:1552›3.Costs are as important asoutcomesEditor—History provides the frameworkwithin which meaning is generated in anysystem, and the issue of the BMJ focusing ondoctors and nurses in the NHS (15 April)showed the complex interplay of stakehold›ers and their motives. Against a backgroundof increasing demands on limited resources,however, decisions about the doctor›nurseskill mix should be made on the basis of costeffectiveness, not historical precedent.1 Tofacilitate this process, Venning et al showedthat there were no differences in outcomeswhether nurses or general practitioners pro›vided care in minor illness and that healthservice costs were similar.2The perspective of a study defines whichcosts to count. For questions that have impli›cations for long term skill mix, all NHS costsshould be considered irrespective of whobears them. For example, the estimated costof training annuitised over the expectedworking life is £4735 a year for a nurse and£21 215 a year for a doctor.3 Including thesevalues will alter this study’s conclusion andshow that nurses are more cost effective thangeneral practitioners for the treatment ofminor illness.The important lesson is that when inter›preting studies that influence health servicedelivery, researchers and decision makersmust not focus on a comprehensivemeasurement of outcomes at the expense ofan inadequate consideration of costs.D P Kernick general practitionerSt Thomas’ Health Centre, Exeter EX4 1HJsul832@eclipse.co.uk1 Kernick DP. Nurses and doctors in primary care: decisionsshould be based on maximising the cost effectiveness of asystem of primary care and not the dictates of historicalprecedent. Br J Gen Pract 1999;49:647›9.2 Venning P, Durie A, Roland M, Roberts C, Leese B.Randomised controlled trial comparing cost effectivenessof general practitioners and nurse practitioners in primarycare. BMJ 2000;320:1048›53. (15 April.)3 Netten A, Knight J. Annuitising the cost of health serviceprofessionals. Health Econ 1999;8:254›5.Breast cancer screeningScreening has to be combined with goodsurgical and oncological servicesEditor—Dickinson in her editorial statedthat there are serious doubts about the con›tribution of breast screening to a fall inbreast cancer deaths.1 This statement isbased on the controversial article byGotzsche and Olsen,2 which at the time of itspublication attracted much media attention.The repetition of this allegation without amore balanced view is erroneous, seems tosupport the findings of this article unequivo›cally, and has the potential to undermineconfidence in the breast screening pro›gramme.The article by Gotzsche and Olsen wasbased on a meta›analysis of eight ran›domised trials of breast screening includinghalf a million women. Because of therandomisation process and using age as amarker of imbalance, Gotzsche and Olsenbelieved that only the trials from Malmö andCanada were methodically correct, andbecause these two trials failed to show a sur›vival advantage in the screened group, theyconcluded that breast screening was not jus›tifiable. The article can be criticised onseveral points including its own methods,3the inclusion of the Canadian trial, whichwas a combined mammogram and physicalexamination compared with physical exam›ination and should therefore not have beenincluded in an analysis of trials of mammog›raphy alone, and the exclusion of the mostrecent data from the Malmö trial, which hadshown a relative risk of death from breastcancer of 0.81 and a 26% reduction in breastcancer mortality. The study is also at oddswith other meta›analyses such as that byWald et al, which showed a 24% lower mor›tality related to breast cancer in women agedover 50 years invited for mammography.4Sources of evidence accessed at least weekly. Values are percentages (numbers)SourceCommunity nurses and community based PAMs(n=119) General practitioners (n=129)Colleagues 63 (75) 77 (99)Experts 37 (44) 91 (117)Journals 35 (42) 95 (123)PAMs=professionals allied to medicine.Letters567BMJ VOLUME 321 2 SEPTEMBER 2000 bmj.com

Pragmatic approach is effective in evidence based health care

http://discovery.ucl.ac.uk/1347261/1/1347261.pdf

Pragmatic approach is effective in evidence based health care

Abstract

Similar works

Full text

Available Versions

UCL Discovery