Steurer, Johann

RERO DOC Digital Library

PERSPECTIVEThe Delphi method: an efficient procedureto generate knowledgeJohann SteurerPublished online: 11 June 2011# ISS 2011Physicians base the recommendations and decisions theymake in daily practice on the results of clinical researchfindings. Results from randomized trials and systematicreviews inform physicians about the effects of treatments,and cross-sectional studies inform them about the accuracyof diagnostic tests. Researchers and clinical experts strive toincrease the knowledge and to generate a solid knowledgebase of medicine to support physicians in caring forpatients. However, not all relevant questions in medicinecan be answered by conventional research methods basedon patient data, and in some instances there are moreeffective methods for creating knowledge that are pertinentto patient care. For example, conventional methods may notbe useful when there is conflicting scientific evidence,when accurate information is not available, and whenjudgmental information is required. An alternative for suchsituations is to ask experts what they think and collect theirknowledge in a structured and systematic way.During the last two decades, “evidence-based medicine”has become the new paradigm in medicine. In parallel,expertise has been discredited as a source of knowledge,ranking low in the hierarchy of evidence. Certainly some‘experts’ fail to resist temptations that influence what theysay and write. However, this does not justify mistrust ofexpert knowledge in general. Expert knowledge gleanedfrom years of experience in the field is a valuable andunderrated source of knowledge that should be madeavailable to other, less experienced physicians. There areseveral methods for garnering the knowledge of experts,one of which is the Delphi method.The term ‘Delphi method’ goes back to the oracle ofDelphi in ancient Greece, which was consulted on mattersthat ranged from public policy to personal affairs. In the1950s, the deity Apollo, who was thought to speak throughthis oracle, was replaced by secular experts after a series ofstudies that strove to reach the most reliable consensusamong a group of experts was conducted by the RANDcooperation [1]. “Project Delphi,” financed by the U.S. AirForce, was the name given to the first project forforecasting technological developments. Similar methodsinclude the nominal group technique and the consensusconference. The nominal technique and the consensusconference use highly structured meetings of experts tocollect information using a format that includes a discus-sion among the experts.The Delphi method consists of “a series of structuredgroup processes, each referred to as a round, to surveyexpert opinion and reach a group response,” [2]. Opinions,beliefs, and judgments are collected and organized in asystematic fashion that focuses primarily on consensus butalso on dissenting views. The core principles includeindividual feedback regarding a given topic or question-naire; assessment of the group’s judgment or views;opportunities for individuals to revise their views; andanonymity for individual responses.The Delphi method has two important advantagescompared to the nominal technique or the traditionalconsensus conference. One advantage is that experts fromall over the world today can participate via electroniccommunication and the second is the anonymous responseformat, which allows experts to express opinions or beliefswithout being influenced or governed by other experts. Thisanonymity reduces the effects of group interaction that leadJ. Steurer (*)Horten Centre for Patient-Oriented Research and KnowledgeTransfer,Raemistrasse 100,CH 8091, Zurich, Switzerlande-mail: johann.steurer@usz.chSkeletal Radiol (2011) 40:959–961DOI 10.1007/s00256-011-1145-zsome to follow the opinion of the majority or that of thegreatest assumed authority. Anonymity also protects expertsfrom losing face if their opinions are dissenting and allowsthem to change their positions during the survey, even ifthey had held those views for decades.The Delphi method is a survey technique that has beenapplied to health research within the fields of healthtechnology assessment, health education, diagnostic criteriadevelopment, indicator selection for quantifying medicalcare quality, opinion surveys for developing appropriate-ness criteria for medical interventions and establishment ofresearch agendas and research priorities for various disci-plines [3–6].A Delphi survey has three main tasks: First, defining anddescribing the topic and preparing one or more questions tosend to the experts; second, selection of a panel ofparticipating experts; and third, organizing and runningthe survey, which involves two or more rounds. After thesurvey topic is clarified, the next step is to formulate thequestions and determine the information the researcherswould like to obtain. For example, diagnosis of lumbarspinal stenosis is a challenging task—it is unclear whichradiological criteria should be used to describe anddiagnose this condition. The first round of the survey canbe designed at least two ways. One way is to ask expertswhich parameters and points they use for measuring thedistances or square areas in the spinal canal. The other wayis to prepare a list of published parameters and then ask theexperts if the list is comprehensive and whether they wouldinclude additional parameters.One of the most critical aspects of the Delphi survey isthe selection of qualified experts. It is common to contactone or two experts in the field and ask them to suggestother experts. This is a convenient approach, but can lead toselection bias, as the experts will most likely nominatecolleagues who share their views. Another more elaborateapproach is to include only experts with more than, forexample, five publications on the chosen topic in peer-reviewed journals during the last 3 years. Note that a Delphisurvey does not depend per se on a sample that isrepresentative of any particular population. That is, thereare no specified rules regarding the selection of participantsor the number of participants. Rather, the Delphi methoddepends on the information the researchers want to gather.For example, 20 panelists may be adequate for thedevelopment of diagnostic indicators, whereas the develop-ment of a research agenda for colorectal surgery can andshould involve inviting all members of the correspondingmedical society to participate. If the results of a survey willbe used to shape public policy regarding, for example,priorities in the health care system, a representative sampleof the population should participate as well.The third task is collection of participants’ opinions andbeliefs. This can either be done using paper and pencilresponse or via the Internet. There are several Web-basedapplications (e.g., www.surveymonkey.com) for conductingsurveys. In general, the survey is accomplished in two orthree rounds. In the first round, a single question, aquestionnaire, or, in the lumbar spinal stenosis example, alist of radiological parameters to describe stenosis is sent tothe participants. An answer, or completion of a list, isrequested from each participant. The feedback from theparticipants is collected and, depending on the topic,question(s) are reformulated, new questions are added or alist of items is updated and adapted.In the second round, participants rank their agreementwith statements or weigh the relevance of information. Inthe Delphi survey on radiological signs of lumbar spinalstenosis, for example, participants may get an updated listin the second round and rank the diagnostic relevance ofeach parameter on a visual analog scale that ranges from 0(completely irrelevant) to 10 (very relevant).Whether a third round is necessary depends on the topicand design of the survey and on the desired result. If the aim isto reach consensus among all of the experts, it is likely thatmore than three rounds will be necessary; however, most ofthe time, full consensus is not the aim of the survey. Delphisurveys can be used to measure the degree of consensus ordegree of dissent. The results of a Delphi survey are reportedin terms of medians or means and interquartile ranges orstandard deviations. Statistical measures, like Cronbach’s α,can be used to quantify the homogeneity or consistency ofopinion among the experts [3].Critics may argue that the results of such surveys arecollections of personal beliefs and opinions. This may be alimitation of such a survey; therefore careful selection ofparticipants is of greatest importance to obtain sensible andtrustworthy results. Another limitation may be that theresults themselves cannot be discussed within the group ofexperts. Sometimes such a discussion might lead todifferent results. However, as noted above, group discus-sions have their own shortcomings.Purists, always believing only in the results of empiricalstudies, will distrust the results of Delphi surveys. Ingeneral, the Delphi method is used only when scientificevidence is either absent or contradictory, and judgmentalinformation is necessary. For such situations, and there aremany of them in medicine, collective expert opinions andbeliefs are helpful and necessary to inform both patientsand physicians. One premise underlying the Delphi method960 Skeletal Radiol (2011) 40:959–961is that such collective beliefs are generally more trustwor-thy than the beliefs of a single person.References1. Dalkey N, Helmer O. An experimental application of the Delphimethod to the use of experts. Manage Sci. 1963;9:458–67.2. Burt CG, Cima RR, Koltun WA, Littlejohn CE, Ricciardi R,Temple LK, et al. Developing a research agenda for the AmericanSociety of Colon and Rectal Surgeons: results of a Delphiapproach. Dis Colon Rectum. 2009;52:898–905.3. Graham B, Regehr G, Wright JG. Delphi as a method toestablish consensus for diagnostic criteria. J Clin Epidemiol.2003;56:1150–6.4. Gagliardi AR, Fleshner N, Langer B, Stern H, Brown AD.Development of prostate cancer quality indicators: a modifiedDelphi approach. Can J Urol. 2005;12:2808–15.5. Park RE, Fink A, Brook RH, Chassin MR, Kahn KL, MerrickNJ, et al. Physician ratings of appropriate indications for sixmedical and surgical procedures. Am J Public Health.1986;76:766–72.6. Phitayakorn R, Delaney CP, Reynolds HL, Champagne BJ, HeriotAG, Neary P, et al. Standardized algorithms for management ofanastomotic leaks and related abdominal and pelvic abscesses aftercolorectal surgery. World J Surg. 2008;32:1147–56.Skeletal Radiol (2011) 40:959–961 961

The Delphi method: an efficient procedure to generate knowledge

http://doc.rero.ch/record/313525/files/256_2011_Article_1145.pdf

The Delphi method: an efficient procedure to generate knowledge

Abstract

Similar works

Full text

Available Versions

RERO DOC Digital Library