Objective: Examine the reproducibility of the RAND method for developing criteria for the appropriateness of medical procedures. Design: Comparison of two sets of explicit criteria for appropriateness of upper gastrointestinal (UGI) endoscopy, developed by separate expert panels from two countries.Setting: United States, Switzerland. Study participants: National experts from different medical specialties involved in the referral or application of UGI endoscopy.Interventions: Each panel was presented with about 500clinical scenarios (indications)that were rated on a nine-point scale as to the appropriateness of performing UGI endoscopy for a patient with that clinical presentation. Main outcome measurer: (1) distribution of appropriateness ratings and intrapanel agreement categories between the two panels, (2)between-panel agreement of assigning appropriateness for comparable indications and, (3) percentage of indications with major between-panel differences. Results: Ratings for 2/3 of indications could be compared. The Swiss panel showed higher intrapanel agreement (54.6% versus 46.2% p=0.002). Seventy-eight percent of comparable Indications were assigned to indentical categories of appropriateness by both panels (kappa=0.76,P <0.001). For 93% of the 376 comparable indications, there were no major interpanel differences. Conclusion: Separate expert panels in different countries, using a standardized methodology, produce criteria for appropriatenesof medical procedures that are similar. Given the resources being invested throught the world in devilping criteria and guidelines, international collaboration in seeking optimal use of limited health care resources should be intensifled. © 1997 Elsevier Science Ltd. All rights reserve

BROOK, ROBERT H.

BURNAND, BERNARD

DUBOIS, ROBERT W.

DUPRIEZ, KARINE

FROEHLICH, FLORIAN

GONVERS, JEAN-JACQUES

LAREQUILAUBER, TANIA

PACHE, ISABELLE

VADER, JOHN-PAUL

RERO DOC Digital Library

PcrgamonInternational Journal for Quality In Health Care, Vol. 9, No. 2, pp. 83-86,1997Copyright © 1997 PublUhed by Bjevier Science Ltd. AH rights reservedPrinted in Great BritainPH: S1353^»505(97)00006-9 1353-4505/97 sn.oo+o.ooEditorialEight Recommendations for Maximizing the Return on Investmentin External Quality OversightIn a previous editorial, Palmer addressed the evolvingimpact of managed care on health care qualitymanagement in the United States and concluded that ameltdown was transpiring that represented a movementaway from strict external quality of care oversight towarda paradigm of encouraging quality improvement throughprofessional and competitive incentives [1]. Whereasmany countries throughout the world may not have thequality improvement benefits (and liabilities) of theintense competition brought about by the evolution ofmanaged care [2], the US marketplace experienceprovides the opportunity for consideration at aninternational level of the value of external regulatoryoversight of quality of care. The purpose of this editorialis to consider a conceptual framework for an economicevaluation of external quality oversight to stimulatethinking at an international level regarding the returnon investment of resources allocated to external qualityoversight. These comments will be framed in reference tothe largest external quality oversight effort in the US, theHealth Care Financing Administration's (HCFA) HealthCare Quality Improvement Program (HCQIP) [3-5] thatcurrently allocates in excess of $220 million per year fromUS taxpayers for 37 million Medicare beneficiaries whoare the subject of the quality improvement efforts of thepeer review organization (PRO) program. There isconsiderable uncertainty regarding the return on thisinvestment from any economic frame of reference,including that of Medicare [4, 6,7]. This commentaryprovides a definition of quality of care that incorporatesthe conditions of perfect competition and concludes witheight recommendations that may be robust on aninternational basis with respect to maximizing thereturn on investment of resources in external qualityoversight programs.DEFINING QUALITY OF CAREIdentifying the implications and operation of an idealmarket for health care services using the paradigm ofperfect competition may inform an evaluation of thereturn on investment in health care quality assuranceefforts, such as the HCQIP. Under perfect competition,the quality of care would be optimal (or, if not optimal,dynamically self-adjusting toward the optimum) if thefollowing four conditions prevailed in the market forhealth services [8]:(1) large numbers of buyers and sellers,(2) free mobility of resources,(3) undifferentiated products, and(4) symmetric information among buyers and sellers.The market for health care services is remarkable inthat all four of these assumptions of the perfectlycompetitive paradigm may fail. However, the conditionthat is most critical to the study of quality of care, and theone that may be driving the overall structure of themarket for health care services (thereby causing thefailure of the other three assumptions), is the fourthcondition, symmetric information among buyers andsellers [9]. The market failure that results from theinability of consumers of health care services to assessthe quality of care accurately is the economic justificationfor the licensure of health care providers [10]. Profes-sional licensure, with its attendant limitation on thenumber of providers and mobility of resources, repre-sents a further deviation from the paradigm of perfectcompetition.The inability of consumers of health care to assess thequality of care accurately also sets up a principal-agentrelationship between the consumer and the provider. Asagents for the consumer, providers recommend, dispenseand evaluate the quality of health care services. Opti-mally, the providers (as agents for the consumers)recommend and dispense the type and level of healthservices that the consumer would choose had theconsumer been appropriately informed about the efficacyand quality of the available services. Since a provider mayhave a self-interest in some of the health care servicesrequired by the consumer, an imperfect agency developsbetween the consumer and the provider, and theconsumer may not be offered the services that he wouldotherwise choose had he been fully informed. Thisimperfect agency results in efficiency losses in terms ofvariations in both the quality and quantity of careprovided (the small area variations phenomenon). How-ever, the professionalism of the provider and variousexternal and internal quality assurance programs keepthe self-interest of the provider from dominating the bestinterests of the consumer.8384 D. J. Ballard and C. B. CangialoscIf the agency relationship between the consumer andthe health services provider was perfect, and in theabsence of other imperfections in any other relevantmarkets, the quality of care received by the patient wouldreflect the consumer's preference weighting for qualityhealth care in the face of other consumption alternatives.This suggests that the definition of quality of care shouldincorporate a reference to consumer preferences. Byconsidering the paradigm of a perfectly competitivemarket for health services with perfect agency, wewould restate the US Institute of Medicine's definitionof quality of care in the following way [11]:Quality of care is the degree to which health services forindividuals and populations are consistent with currentprofessional knowledge and reflect the preferences ofwell-informed consumers with regard to the trade-offbetween increasing desired health outcomes and reducingother consumption alternatives.This definition of quality of care has the added benefitof focusing on consumer preferences; thus, it is consistentwith the philosophy of "focusing on the customer" that iscentral to the total quality management/continuousquality improvement approach to quality assurance. Inaddition to perfect agency and its focus on consumerpreferences, the perfectly competitive market for healthcare services would have four additional characteristics:(1) there are few or no costs associated with operating themechanism (transactions costs are low); (2) the perfectlycompetitive market is self-regulating and self-improving;(3) strategic behavior on the part of an individual agentis not beneficial to that agent; and (4) the mechanism isnot excessively burdensome to the agents who areinfluenced.THE HCQIP AS AN EXAMPLE OFEXTERNAL QUALITY OVERSIGHTThe HCQIP is the fifth version of review or oversightof the Medicare program. From its inception in 1965through 1973, the Medicare program operated under noformal external review mechanism. In 1973, the USCongress enacted legislation to create the ProfessionalStandards Review Organizations (PSROs). From 1965through 1983, Medicare reimbursed providers based onbilled charges. The role of the more than 200 regionalPSROs throughout the US and its territories was tomonitor providers in order to identify and control healthcare providers who were possibly overutilizing resources.In 1983, in response to an ongoing increase in costs thatwas threatening the solvency of the Medicare program,the US Congress changed the Medicare reimbursementmechanism. Within the Tax Equity and Fiscal Respon-sibility Act (TEFRA) of 1983, the Prospective PaymentSystem (PPS) was established. Under the PPS, hospitalsare reimbursed through a fixed payment that is based onthe patient's Diagnostic-Related Group (DRG), andthere is an incentive for providers to "undertreat" orprovide too few resources in the care of patients. Tocounterbalance this problem, Congress reconstituted areview of the Medicare program by establishing thePhysician Review Organizations (PROs). An externalreview of the Medicare program was carried out by thePROs through successive contracts, called the First,Second, Third, and Fourth Scopes of Work [5]. PROsare now engaged in the Fifth Scope of Work from 1996 to1999 with a mandate to shift external oversight activitiesfrom an exclusive focus on hospital care to addressingquality of ambulatory care as well.THE HCQEP COMPARED TO THE IDEALQUALITY ASSURANCE METHODOLOGYJencks and Wilensky defined the goal and immediateobjective of the HCQIP as follows: "to move fromdealing with individual clinical errors to helping provi-ders to improve the mainstream of care," and to create asituation in which "PROs will focus primarily onpersistent differences between the observed and theachievable in both care and outcomes and less onoccasional, unusual deficiencies in care," respectively[3]. This explicit goal and immediate objective do notprovide an obvious basis for an economic evaluation ofthe HCQIP. The ideal quality assurance methodologywould attempt to restore those attributes of the perfectlycompetitive paradigm that are lost as a result of themarket failure described in the previous section. Thefollowing is an evaluation of the HCQIP from thestandpoint of each of the attributes of the perfectlycompetitive paradigm.Accounting for consumer preferencesUnder our definition of quality of care, an ideal qualityassurance methodology would account explicitly for thepreferences of the well-informed consumer. Since theHCQIP does not explicitly take into account thepreferences of consumers, it does not conform to theideal. Some HCQIP efforts are beginning to include therole of consumer preferences, such as projects that focuson the use of breast conserving therapy for breast cancerpatients.Accounting for professional knowledge and judgmentThe ideal quality assurance methodology would alsoaccount for professional knowledge and judgment. TheHCQIP seems to conform to this ideal since the HCQIPforesees using optimal clinical practice guidelines in anattempt to shift and tighten the distribution of treatmentmodalities around these optimal practice guidelines.However, because competing practice guidelines areavailable for the same medical conditions and treatmentinterventions, we must evaluate the alternative guidelineswith regard to optimal patient outcomes. The guidelinesdeveloped by medical subspecialties (as distinguishedEditorial 85from those derived from multidisciplinary efforts) maysuffer from a conflict of interest similar to the onepreviously described to explain the market failure thatresults from asymmetric information [12].Self-regulating and self-improvingThe perfectly competitive market for health serviceswould be self-regulating and self-improving. That is, anyprovider of health services that does not provide asufficiently high quality of care would be driven out ofthe market by superior competitors, and investment innew improved techniques and technologies would berewarded, and quickly adopted, to gain competitiveadvantage. Although the HCQIP is not self-regulating,it does require the specific and ongoing intervention ofHCFA to remain in place, and it does have the potentialto be self-improving. The cooperative improvementprojects with the PROs that are being developed throughthe HCQIP could provide competitive advantages tothose providers who are quick to assimilate successfulinitiatives. This could also result in residual benefits tohealth care consumers beyond the Medicare program. Itmay carry some degree of risk to those providers who donot effectively take up successful initiatives, as theseproviders may begin to lose patients to "higher quality"providers. One economic criterion for evaluating theHCQDP is the extent to which the cooperative improve-ment projects initiated under the HCQIP might providecompetitive advantages to the participating providers.Benefits from strategic behaviorUnder the paradigm of perfect competition, agents orconsortia of agents cannot benefit in the market throughstrategic behavior. It is unclear as to whether the HCQIPis capable of countering strategic behaviors on the part ofconsumers, providers, or PROs. Clearly, the HCQIPitself would be undermined by this type of behavior onthe part of PROs. One of the most obvious PRO strategicbehaviors would be the implementation of projects thatwould be relatively easy to complete.Invisibility of the interventionThe last characteristic of the perfectly competitiveparadigm that could be emulated by the HCQIP is thatit should not be excessively burdensome to the agentsbeing influenced. The perceived burden and intrusion ofthe previous PRO review programs substantially influ-enced the clinical community to press for a redesign of theMedicare quality oversight methodology. The successfulimplementation of the HCQIP has the potential to berelatively invisible to the providers being influenced. Ifthe HCQIP cooperative projects can convincinglyimprove medical practices and create competitive advan-tages for cooperating providers, these providers are likelyto embrace the program.MEASURING THE COST-EFFECTIVENESSOF EXTERNAL QUALITY OVERSIGHTPhelps and Mooney [13] provide important informa-tion for measuring the cost-effectiveness of externalquality oversight in the United States. They havedeveloped an index to generate a dollar-valued welfareloss that combines measures of resource use, thecoefficient of variation in use rates across regions, andthe rate at which the incremental value of a medicaltreatment changes as its rate of use changes. The top 25medical items in 1987 generated a total annual efficiencyloss of $7 billion, or approximately 15% of total healthcare expenditure for that year. Given this analysis(combined with the theoretical ideal quality assuranceand improvement mechanism), the most effective expen-diture of resources for the HCQIP is likely to come out ofanalyzing the effectiveness of PRO efforts with medicalinterventions that generate the greatest welfare loss. It isclear, for example, that HCFA made a good choice ininitiating the Cooperative Cardiovascular Project (CCP),as it addresses three of the top five items on the Phelpsand Mooney list [3,13].We believe that investment in external health carequality oversight should consider the four characteristicsof the paradigm of perfect competition. In order toimplement this concept, we recommend that this invest-ment be directed toward initiatives that incorporate thefollowing items into their project design:(1) A cost-accounting of each of the individualprojects.(2) Review and discussion of consumer preferences asthey relate to the specific topic.(3) Review and discussion of the clinical guidelinesthat pertain to the specific topic, with particular attentionto any competing guidelines that may be present (i.e.subspecialty vs multispecialty guidelines).(4) Attention and concern for strategic behaviors thatmay undermine the goals of the individual projects.(5) Minimization of the burden of the project on thevarious participants.(6) Selection of projects based on the list of conditionsor health care interventions that are associated with thelargest efficiency losses in the particular population thatis the setting for the project.(7) Implementation of projects in an experimentalmanner, ideally with random allocation, so that theeffects of the quality oversight intervention can bedisentangled from secular trends and to ensure thatconfounding factors are distributed by chance [14].(8) Cost-effectiveness assessment of each project.Returning to the framework provided by Palmer [1], asthe value of the various layers and components ofexternal health care quality management oversight areconsidered by governments and other health care fundersthroughout the world, we can all benefit from reflectingon the advice of White, "Have a little bit of statistical86 D. J. Ballard and C. B. Cangialose[and economic] compassion and take a look at thequantitative information before providing inadequatecare or wasting millions of dollars" [15].REFERENCES1. Palmer, R. H., Managed care and melting moments(Editorial). International Journal for Quality inHealth Care 1996; 8: 317-319.2. Chassin, M. R., Improving the quality of healthcare: What strategy works? Bulletin of the New YorkAcademy of Medicine 1996; 73: 81-91.3. Jencks, S. F. and Wilensky, G. R., The health carequality improvement initiative: a new approach toquality assurance in Medicare. Journal of the Amer-ican Medical Association 1992; 268: 900-903.4. Nash, D. B., Is the quality cart before the horse?Journal of the American Medical Association 1992;268: 917-918.5. Hayes, R. P., Lundberg, M. T. and Ballard, D. J.,Peer review organizations: scientific challenges inHCFA's Health Care Quality Improvement Initia-tive. Medical Care Review 1994; 51: 39-60.6. Nash, D. B., The cost of outcomes (Editorial).Journal of Outcomes Management 1996; 3: 2.7. Cangialose, C. B., Elward, K. S. and Ballard, D. J.,An economic evaluation of the Health Care QualityImprovement Program. Journal of Outcomes Man-agement 1996; 3: 4-9.8. Mansfield, E., Microeconomics. Norton and Com-pany, New York, 1994.9. Arrow, K. J., Uncertainty and the economics ofmedical care. American Economic Review 1963; 53:941-973.10. Akerlof, G., The market for 'lemons': qualitativeuncertainty and the market mechanism. QuarterlyJournal of Economics 1970; 84: 488-500.11. Institute of Medicine, Medicare: a strategy for qual-ity assurance. Vol. 1, p. 4, 1990.12. Brook R. H., The RAND/UCLA appropriatenessmethod. In: Agency for Health Care Policy andResearch. Clinical practice guideline development:methodologic perspectives, pp. 95-1009, 59-70. USDepartment of Health and Human Services,Washington, DC, 1994.13. Phelps, C. E. and Mooney, G, Correction andupdate on 'Priority Setting in Medical TechnologyAssessment*. Medical Care 1992; 30: 744-751.14. Ballard, J). J., Hips and knees. State of evidenceregarding effectiveness of quality improvement inter-ventions in orthopedic surgery. Mayo Clinic Proce-dures 1996; 71: 208-210.15. Ballard D. J., A little statistical compassion linked toan intense and creative look at healthcare evidence:the genius of Kerr White. Health Services Research.April 1997 (Editorial).David J. Ballard, Ph.D.Chair, ISQua Advisory CouncilEditorial Committee Member, IJQHCProfessor of Medicine and EpidemiologyDirector, Center for Clinical Evaluation SciencesEmory University, Atlanta, Georgia, USAPresident, Kerr L. White Institute for Health ServicesResearch, Decatur, Georgia, USACharles B. Cangialose, Pb.D.Assistant Professor, Center for Clinical EvaluationSciences, Emory University, Atlanta, Georgia, USA

Appropriateness of Upper Gastrointestinal Endoscopy: Comparison of American and Swiss Criteria

Appropriateness of Upper Gastrointestinal Endoscopy: Comparison of American and Swiss Criteria

Abstract

Similar works

Full text

Available Versions

RERO DOC Digital Library