Complementary medicine has a different philosophy from conventional medicine, presenting challenges to research methodology. Rigorous evaluation of complementary medicine could provide much needed evidence of its effectiveness. Good design of randomised controlled trials will avoid invalid results and misrepresentation of the holistic essence of complementary medicine. Practitioners need to be recognised as a component in or contributor to complementary treatment. Both specific and non-specific outcome measures with long follow up are needed to adequately encompass the essence of complementary medicine

Long, AF

Mason, S

Tovey, P

English

PubMed

S. Mason

Crossref

Evaluating complementary medicine: methodological challenges of randomised controlled trials

White Rose Research Online

This is a repository copy of Evaluating complementary medicine: methodological 
challenges of randomised controlled trials.
White Rose Research Online URL for this paper:
http://eprints.whiterose.ac.uk/166/
Article:
Mason, S, Tovey, P and Long, AF (2002) Evaluating complementary medicine: 
methodological challenges of randomised controlled trials. British Medical Journal, 325 
(7368). 832 - 834 . ISSN 0959-535X 
https://doi.org/10.1136/bmj.325.7368.832
eprints@whiterose.ac.uk
https://eprints.whiterose.ac.uk/
Reuse 
See Attached 
Takedown 
If you consider content in White Rose Research Online to be in breach of UK law, please notify us by 
emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. 
doi:10.1136/bmj.325.7368.832 
 2002;325;832-834 BMJ
  
Su Mason, Philip Tovey and Andrew F Long 
  
 controlled trials
methodological challenges of randomised 
Evaluating complementary medicine:
 http://bmj.com/cgi/content/full/325/7368/832
Updated information and services can be found at: 
 These include:
 References
  
 http://bmj.com/cgi/content/full/325/7368/832#BIBL
This article cites 8 articles, 4 of which can be accessed free at: 
Rapid responses
 http://bmj.com/cgi/eletter-submit/325/7368/832
You can respond to this article at: 
  
 http://bmj.com/cgi/content/full/325/7368/832#responses
free at: 
30 rapid responses have been posted to this article, which you can access for
 service
Email alerting
the top right corner of the article 
Receive free email alerts when new articles cite this article - sign up in the box at
Topic collections
 (195 articles) Complementary Medicine •
 (474 articles) Other Statistics and Research Methods: descriptions •
 (264 articles) Randomized Controlled Trials: descriptions •
  
Articles on similar topics can be found in the following collections 
 Notes   
 http://bmj.bmjjournals.com/cgi/reprintform
To order reprints of this article go to: 
 http://www.bmjjournals.com/subscriptions
 go to: BMJTo subscribe to 
 on 25 October 2004 bmj.comDownloaded from 
Education and debate
Evaluating complementary medicine: methodological
challenges of randomised controlled trials
Su Mason, Philip Tovey, Andrew F Long
Complementary medicine is increasingly popular for treating many different problems. Doctors and
patients need evidence about complementary treatments, but randomised controlled trials need to
be carefully designed to take holism into account and avoid invalid results Topic: 20;149;171
You think that by understanding one, you can
understand two, for one and one is two. But to under-
stand two, you must first understand “and.”
Sufi saying1
Complementary medicine should be evaluated as
rigorously as conventional medicine to protect the
public from charlatans and unsafe practices,2–5 but
many practitioners of complementary medicine are
reticent about evaluation of their practice. Sceptics
maintain that this is because of fear that investigations
will find treatments ineffective and threaten livelihoods.
In defence, many practitioners argue that research
methods dissect their practice in a reductionist manner
and fail to take into account complementary
medicine’s holistic nature leading to invalid evaluation.
Nature of complementary medicine
Complementary medicine comprises many different
disciplines, a wide spectrum of practices and philoso-
phies which differ from conventional medicine.
Conventional medicine traditionally aims to diagnose
illness and treat, cure, or alleviate symptoms. Many
complementary disciplines aim not only to relieve
symptoms and restore wellness but also to help
individuals in a process of self healing within a holistic
view of health. In this view, individuals are more than
just mind, body, and spirit in a social—family or work—
environment: as well as promoting wellness, some
complementary medicine contains a philosophy that
everything is interconnected, and consequently intrin-
sically bound in a therapeutic relationship between the
individual and practitioner.1
Linked to the idea of self healing is a spiritual com-
ponent: some complementary practitioners believe
that illness has a corrective purpose, showing the
underlying disharmony in people’s lives and enabling
identification of areas for change.6 Illness is, in addition,
recognised as complex, embracing factors such as
genetic predisposition, environment, and diet. Conven-
tional medical relief of symptoms or even “cure” may
be considered a temporary respite if the individual
does not tackle the underlying cause—for example, by
releasing suppressed anger, forgiving, or reducing
stress by changing job or being more accepting of cir-
cumstances. Complementary treatment aims to be tai-
lored to the individual’s stage in life, exploring different
underlying causes and solutions.
The challenge for research methodology is to fuse
the philosophical concerns of stakeholders with the
highest standards of methodological rigour. Taking
complementary practitioners’ criticisms of randomised
controlled trials seriously enables methods to be modi-
fied so that the concerns of all stakeholders are taken
into account. This ensures that as far as possible what is
being assessed under experimental conditions is
consistent with everyday complementary practice. In
this paper, we consider the core methodological
difficulties in assessing complementary medicine and
offer practical solutions.7–9 Some evidence from
unmodified trials can be rejected as invalid and is
therefore ignored; anecdotes from practice are rejected
as meaningless. Unless evidence is generated in a way
that satisfies all parties, it is unlikely to have an impact
Summary points
Complementary medicine has a different
philosophy from conventional medicine,
presenting challenges to research methodology
Rigorous evaluation of complementary medicine
could provide much needed evidence of its
effectiveness
Good design of randomised controlled trials will
avoid invalid results and misrepresentation of the
holistic essence of complementary medicine
Practitioners need to be recognised as a
component in or contributor to complementary
treatment
Both specific and non-specific outcome measures
with long follow up are needed to adequately
encompass the essence of complementary
medicine
Northern and
Yorkshire Clinical
Trials and Research
Unit, Academic
Unit of
Epidemiology and
Health Services
Research, University
of Leeds, Leeds
LS2 9NG
Su Mason
honorary principal
research fellow
School of
Healthcare Studies,
University of Leeds
Philip Tovey
principal research
fellow
Health Care
Practice Research
and Development
Unit, University of
Salford, Salford
M5 4WT
Andrew F Long
professor
Correspondence to:
S Mason
medsam@leeds.
ac.uk
BMJ 2002;325:832–4
832 BMJ VOLUME 325 12 OCTOBER 2002 bmj.com
 on 25 October 2004 bmj.comDownloaded from 
on the objective of having evidence based decision
making and practice.
Challenges in research methodology
Sampling
Separation of organisations—Complementary care
within European countries is generally separate to
conventional medicine. Complementary medicine is
not usually integrated into national health systems, and
individuals pay for treatment. Funding for trials should
include the additional costs of complementary
treatment. Additional complexities require attention;
these include legal indemnity for the practitioners tak-
ing part and assurances of their professional and tech-
nical competence.
Generalisability—Because complementary and con-
ventional care are separate, patients recruited to a trial
comparing the two may differ from typical comple-
mentary patients since they themselves did not decide
which treatment they would receive. For example,
belief systems or coping mechanisms might differ,
affecting treatment outcomes. Incorporating prefer-
ence in the design of the trial might help tackle and
investigate this problem.
Therapeutic expectation—Choosing or preferring a
particular complementary discipline carries an associ-
ated implicit belief in benefit. Such credibility and
expectation of therapeutic gain can bias the results of
trials, especially if they cannot be blinded (as in a trial
of yoga practice or tai chi). Expectation and perceived
credibility should be acknowledged and assessed at
randomisation. Several controlled trials have used a
credibility scale to check the equivalence of psychologi-
cal treatments.10 11
Wide range of symptoms—Complementary medicine
and subsequent diagnosis are often not used because
of a specific disease. Instead, users may feel tired, lack
energy, or be unhappy. A trial about the effectiveness of
an intervention based on a disease may not be relevant
to complementary treatments. Trials may need to focus
on problem areas which are independent of categories
of disease. Broad and appropriate outcome measures
should capture a wide range of symptoms.
Treatment
Standardisation—Within a complementary disci-
pline, treatment is individualised and includes care
associated with an individual’s “diagnosis” and the spe-
cific practitioner (skills, competence, manner, etc). The
practitioner is explicitly recognised as a component of
or contributor to the treatment. This may conflict with
ideas of standardisation in conventional trials. Com-
plex treatment interventions can be investigated prag-
matically. Standardised treatment can be interpreted as
treating to a standard regimen, defined in a protocol,
rather than exactly the same treatment. Randomisation
should include stratification by individual practitioner.
Influence of practitioner and user—Practitioners often
claim that they do not just carry out a set of
techniques—for example, the remedy in herbalism or
the needles in acupuncture—on the user. They claim to
understand how to improve the healing process, the
user’s role in the healing process, and the therapeutic
relationship between the user and the practitioner. All
aspects of complementary care should be assessed, and
the human experience should be retained as central
(not marginalised, as often occurs in reductionist
methods).12 If a trial design includes modified therapies
so that the holistic aspect is excluded, then this must be
explicitly stated.
Controls—Choosing an appropriate comparison
group in trials of complementary medicine trials is
often problematic. Sham controls—for example,
dummy acupuncture to create a placebo controlled
comparison—should be treated with caution as they
are artificial experiences and distort practice. Any ben-
efit over standard medical treatment should be
assessed and appropriate controls adopted. For some
complementary treatments—for example, homoeopa-
thy and herbalism—the traditional blinded randomised
controlled trial is appropriate. For treatments in which
blinding is not possible—for example, meditation—
emphasis should be on pragmatic trials with conven-
tional medical practice as controls.
Understanding complementary processes—It is difficult
to define precisely which aspect of a complementary
treatment provides benefit. During spiritual healing, is
it the intent to heal, the preparation, the channelling of
“energy,” the accompanying non-judgmental listening
to the user, the relationship with the healer, the healing
environment, users’ expectations or attitudes, or all of
these which are important? Complex interventions in
conventional medicine result in similar problems.13 Too
often trials provide both limited detail on the actual
process and little insight into what component of the
intervention leads to the effect. Qualitative methods
should be used to investigate the process. The design of
the trial should address the therapeutic relationship
and non-specific effects, in addition to any specific
treatment effects.14 At the same time, from the user’s
perspective, it is the beneficial effect itself that matters,
not how it was brought about.
Outcome measures
Appropriate outcome measures—Capturing the
essence of how complementary medicine might help
people is an important challenge. Noteworthy changes
after treatment, such as spiritual change or personal
growth, might not be measurable. Choice of outcome
criteria in the evaluation of complementary therapies
“may make a substantial difference to conclusions as to
Acupuncture is a popular complementary treatment
Education and debate
833BMJ VOLUME 325 12 OCTOBER 2002 bmj.com
 on 25 October 2004 bmj.comDownloaded from 
whether a particular intervention or therapy is effective
or not.”14 Holism, which is central to the philosophy
and practice of much complementary medicine, needs
to be better understood. Detailed qualitative studies,
accompanying a trial, could elicit these more elusive
outcomes and aim to take account of experiential out-
come criteria, such as increased self reflection.
Illness role—According to some complementary
philosophies, illness indicates an imbalance in the body
and symptoms of illness draw attention to this. For
example, individuals develop colds, backache, or
eczema when they are under stress or run down. If
complementary medicine helps people recognise
imbalance and adverse lifestyle factors that contribute
to illness and promotes change and resultant improve-
ments in health, then research needs to capture these
changes.
Chronic illness—Complementary treatment is often
sought by people with chronic illnesses who may not
have responded to conventional treatment or may
want to explore different forms of care. Any change is
likely to be gradual and subtle and to have a relapsing
and remitting pattern. As complementary interven-
tions tend to aim for long term healing, and not neces-
sarily cure, they are well suited to help people with
chronic illness. Funding bodies must understand that
long term follow up of outcomes (specific and
non-specific) is needed for results to be meaningful.
Randomised controlled trials work best for exploring
short term effects. Over time, factors other than the
trial treatments could bring about the observed effects.
Variations in experience—One person can have
different experiences of complementary medicine at
different times; different people can have widely differ-
ent experiences. Designing a trial to investigate a treat-
ment that might be effective immediately for one
person but take months to be effective for another is a
problem. A “healing moment” may occur at the time of
treatment or some time after. Appropriate experiential
outcome measures should take variations into account
along with adequately long follow up periods.
Sick role—Some users do not respond to treatment.
There are those who adapt to the sick role because they
have an investment in staying ill. Others may be
satisfied with the short fix, of feeling peaceful and
relaxed, that is offered by many complementary
treatments and may not take responsibility to make the
necessary changes to improve their health. In
principle, randomised controlled trials provide an effi-
cient way to deal with this problem. People adopting
the sick role should be distributed equally throughout
the arms of a trial through the randomisation process.
Conclusion
Researchers need to tackle the complex issues in
researching complementary medicine. Studies should
be designed to enable a valid evaluation of comple-
mentary medicine, on multiple levels, in the human
healing process.
People often turn to complementary treatments
because of frustration with conventional medicine.15
They seek that “more”—the essence of what is valued—
that complementary treatments offer. Research into
complementary medicine has a challenge not to miss
the “more” and not to design inappropriately to find it.
Contributors: SM conceived the idea for the paper and wrote
the first draft which was rewritten with extensive contributions
from PT and AL.
Funding: No additional funding.
Competing interests: SM is a spiritual healer.
1 Wright S, Sayre-Adams J. Sacred space: right relationship in health and
healing: not just what we do but who we are. In: Rankin-Box D. The nurses’
handbook of complementary therapies. 2nd ed. London: Baillière Tindall,
2001.
2 Ernst E, Pittler M, Stevinson C, White A, eds. The desktop guide to comple-
mentary and alternative medicine: an evidence-based approach. London:
Mosby, 2001.
3 Cornbleet MA, Ross CSK. Research in complementary medicine is
essential. BMJ 2001:322:736-7.
4 House of Lords Select Committee on Science and Technology.
Complementary and alternative medicine. London: Stationery Office, 2000.
(6th report.)
5 Angell M, Kassirer JP. Alternative medicine: the risks of untested and
unregulated remedies [editorial]. N Engl J Med 1998;339:839-41.
6 Brown C. Optimum healing: a life-changing new approach to achieving good
health. London: Rider, 1998.
7 Long AF, Mercer G. Challenges in researching the effectiveness of com-
plementary therapies. J Contemp Health 1999;8:13-9.
8 Brown CK. Methodological problems of clinical research into spiritual
healing: perspective. J Altern Complement Med 2000;6:171-6.
9 Nahin RL, Straus SE. Research into complementary and alternative
medicine: problems and potential. BMJ 2001;322:161-4.
10 Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J Behav
Ther Exp Psychiatry 1972;3:257-60.
11 Morley S, Williams AcdeC, Eccleston C. Systematic review and
meta-analysis of randomised controlled trials of cognitive behaviour
therapy and behaviour therapy for chronic pain in adults, excluding
headache. Pain 1999;80:1-13.
12 Stevinson C. Why patients use complementary and alternative medicine.
In: Ernst E, Pittler M, Stevinson C, White A, eds. The desktop guide to com-
plementary and alternative medicine: an evidence-based approach. London:
Mosby, 2001
13 Medical Research Council. A framework for development and evaluation of
RCTs for complex interventions to improve health. London: MRC, 2000.
14 Long AF, Mercer G, Hughes K. Developing a tool to measure holistic
practice: a missing dimension in outcomes measurement within comple-
mentary therapies. Complement Ther Med 2000;8:26-31.
15 Rees L, Weil A. Integrated medicine. BMJ 2001;322:119-20.
(Accepted 10 April 2002)
Submitting articles to the BMJ
We are now inviting all authors who want to submit a paper to
the BMJ to do so via the web (http://submit.bmj.com).
Benchpress is a website where authors deposit their
manuscripts and editors go to read them and record their
decisions. Reviewers’ details are also held on the system, and
when asked to review a paper reviewers will be invited to access
the site to see the relevant paper. The system is secure, protected
by passwords, so that authors see only their own papers and
reviewers see only those they are meant to.
Anyone with an internet connection and a web browser can use
the system.
The system provides all our guidance and forms and allows
authors to suggest reviewers for their paper. Authors get an
immediate acknowledgement that their submission has been
received, and they can watch the progress of their manuscript.
The record of their submission, including editors’ and reviewers’
reports, remains on the system for future reference.
The system itself offers extensive help, and the BMJ ’s editorial
office will help authors and reviewers if they get stuck.
Benchpress is accessed via http://submit.bmj.com or via a link
from bmj.com
Education and debate
834 BMJ VOLUME 325 12 OCTOBER 2002 bmj.com
 on 25 October 2004 bmj.comDownloaded from 


This is a repository copy of Evaluating complementary medicine: methodological challenges of randomised controlled trials.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/166/Article:Mason, S, Tovey, P and Long, AF (2002) Evaluating complementary medicine: methodological challenges of randomised controlled trials. British Medical Journal, 325 (7368). 832 - 834 . ISSN 0959-535X https://doi.org/10.1136/bmj.325.7368.832eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse See Attached Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. doi:10.1136/bmj.325.7368.832  2002;325;832-834 BMJ  Su Mason, Philip Tovey and Andrew F Long    controlled trialsmethodological challenges of randomised Evaluating complementary medicine: http://bmj.com/cgi/content/full/325/7368/832Updated information and services can be found at:  These include: References   http://bmj.com/cgi/content/full/325/7368/832#BIBLThis article cites 8 articles, 4 of which can be accessed free at: Rapid responses http://bmj.com/cgi/eletter-submit/325/7368/832You can respond to this article at:    http://bmj.com/cgi/content/full/325/7368/832#responsesfree at: 30 rapid responses have been posted to this article, which you can access for serviceEmail alertingthe top right corner of the article Receive free email alerts when new articles cite this article - sign up in the box atTopic collections (195 articles) Complementary Medicine • (474 articles) Other Statistics and Research Methods: descriptions • (264 articles) Randomized Controlled Trials: descriptions •  Articles on similar topics can be found in the following collections  Notes    http://bmj.bmjjournals.com/cgi/reprintformTo order reprints of this article go to:  http://www.bmjjournals.com/subscriptions go to: BMJTo subscribe to  on 25 October 2004 bmj.comDownloaded from Education and debateEvaluating complementary medicine: methodologicalchallenges of randomised controlled trialsSu Mason, Philip Tovey, Andrew F LongComplementary medicine is increasingly popular for treating many different problems. Doctors andpatients need evidence about complementary treatments, but randomised controlled trials need tobe carefully designed to take holism into account and avoid invalid results Topic: 20;149;171You think that by understanding one, you canunderstand two, for one and one is two. But to under-stand two, you must first understand “and.”Sufi saying1Complementary medicine should be evaluated asrigorously as conventional medicine to protect thepublic from charlatans and unsafe practices,2–5 butmany practitioners of complementary medicine arereticent about evaluation of their practice. Scepticsmaintain that this is because of fear that investigationswill find treatments ineffective and threaten livelihoods.In defence, many practitioners argue that researchmethods dissect their practice in a reductionist mannerand fail to take into account complementarymedicine’s holistic nature leading to invalid evaluation.Nature of complementary medicineComplementary medicine comprises many differentdisciplines, a wide spectrum of practices and philoso-phies which differ from conventional medicine.Conventional medicine traditionally aims to diagnoseillness and treat, cure, or alleviate symptoms. Manycomplementary disciplines aim not only to relievesymptoms and restore wellness but also to helpindividuals in a process of self healing within a holisticview of health. In this view, individuals are more thanjust mind, body, and spirit in a social—family or work—environment: as well as promoting wellness, somecomplementary medicine contains a philosophy thateverything is interconnected, and consequently intrin-sically bound in a therapeutic relationship between theindividual and practitioner.1Linked to the idea of self healing is a spiritual com-ponent: some complementary practitioners believethat illness has a corrective purpose, showing theunderlying disharmony in people’s lives and enablingidentification of areas for change.6 Illness is, in addition,recognised as complex, embracing factors such asgenetic predisposition, environment, and diet. Conven-tional medical relief of symptoms or even “cure” maybe considered a temporary respite if the individualdoes not tackle the underlying cause—for example, byreleasing suppressed anger, forgiving, or reducingstress by changing job or being more accepting of cir-cumstances. Complementary treatment aims to be tai-lored to the individual’s stage in life, exploring differentunderlying causes and solutions.The challenge for research methodology is to fusethe philosophical concerns of stakeholders with thehighest standards of methodological rigour. Takingcomplementary practitioners’ criticisms of randomisedcontrolled trials seriously enables methods to be modi-fied so that the concerns of all stakeholders are takeninto account. This ensures that as far as possible what isbeing assessed under experimental conditions isconsistent with everyday complementary practice. Inthis paper, we consider the core methodologicaldifficulties in assessing complementary medicine andoffer practical solutions.7–9 Some evidence fromunmodified trials can be rejected as invalid and istherefore ignored; anecdotes from practice are rejectedas meaningless. Unless evidence is generated in a waythat satisfies all parties, it is unlikely to have an impactSummary pointsComplementary medicine has a differentphilosophy from conventional medicine,presenting challenges to research methodologyRigorous evaluation of complementary medicinecould provide much needed evidence of itseffectivenessGood design of randomised controlled trials willavoid invalid results and misrepresentation of theholistic essence of complementary medicinePractitioners need to be recognised as acomponent in or contributor to complementarytreatmentBoth specific and non-specific outcome measureswith long follow up are needed to adequatelyencompass the essence of complementarymedicineNorthern andYorkshire ClinicalTrials and ResearchUnit, AcademicUnit ofEpidemiology andHealth ServicesResearch, Universityof Leeds, LeedsLS2 9NGSu Masonhonorary principalresearch fellowSchool ofHealthcare Studies,University of LeedsPhilip Toveyprincipal researchfellowHealth CarePractice Researchand DevelopmentUnit, University ofSalford, SalfordM5 4WTAndrew F LongprofessorCorrespondence to:S Masonmedsam@leeds.ac.ukBMJ 2002;325:832–4832 BMJ VOLUME 325 12 OCTOBER 2002 bmj.com on 25 October 2004 bmj.comDownloaded from on the objective of having evidence based decisionmaking and practice.Challenges in research methodologySamplingSeparation of organisations—Complementary carewithin European countries is generally separate toconventional medicine. Complementary medicine isnot usually integrated into national health systems, andindividuals pay for treatment. Funding for trials shouldinclude the additional costs of complementarytreatment. Additional complexities require attention;these include legal indemnity for the practitioners tak-ing part and assurances of their professional and tech-nical competence.Generalisability—Because complementary and con-ventional care are separate, patients recruited to a trialcomparing the two may differ from typical comple-mentary patients since they themselves did not decidewhich treatment they would receive. For example,belief systems or coping mechanisms might differ,affecting treatment outcomes. Incorporating prefer-ence in the design of the trial might help tackle andinvestigate this problem.Therapeutic expectation—Choosing or preferring aparticular complementary discipline carries an associ-ated implicit belief in benefit. Such credibility andexpectation of therapeutic gain can bias the results oftrials, especially if they cannot be blinded (as in a trialof yoga practice or tai chi). Expectation and perceivedcredibility should be acknowledged and assessed atrandomisation. Several controlled trials have used acredibility scale to check the equivalence of psychologi-cal treatments.10 11Wide range of symptoms—Complementary medicineand subsequent diagnosis are often not used becauseof a specific disease. Instead, users may feel tired, lackenergy, or be unhappy. A trial about the effectiveness ofan intervention based on a disease may not be relevantto complementary treatments. Trials may need to focuson problem areas which are independent of categoriesof disease. Broad and appropriate outcome measuresshould capture a wide range of symptoms.TreatmentStandardisation—Within a complementary disci-pline, treatment is individualised and includes careassociated with an individual’s “diagnosis” and the spe-cific practitioner (skills, competence, manner, etc). Thepractitioner is explicitly recognised as a component ofor contributor to the treatment. This may conflict withideas of standardisation in conventional trials. Com-plex treatment interventions can be investigated prag-matically. Standardised treatment can be interpreted astreating to a standard regimen, defined in a protocol,rather than exactly the same treatment. Randomisationshould include stratification by individual practitioner.Influence of practitioner and user—Practitioners oftenclaim that they do not just carry out a set oftechniques—for example, the remedy in herbalism orthe needles in acupuncture—on the user. They claim tounderstand how to improve the healing process, theuser’s role in the healing process, and the therapeuticrelationship between the user and the practitioner. Allaspects of complementary care should be assessed, andthe human experience should be retained as central(not marginalised, as often occurs in reductionistmethods).12 If a trial design includes modified therapiesso that the holistic aspect is excluded, then this must beexplicitly stated.Controls—Choosing an appropriate comparisongroup in trials of complementary medicine trials isoften problematic. Sham controls—for example,dummy acupuncture to create a placebo controlledcomparison—should be treated with caution as theyare artificial experiences and distort practice. Any ben-efit over standard medical treatment should beassessed and appropriate controls adopted. For somecomplementary treatments—for example, homoeopa-thy and herbalism—the traditional blinded randomisedcontrolled trial is appropriate. For treatments in whichblinding is not possible—for example, meditation—emphasis should be on pragmatic trials with conven-tional medical practice as controls.Understanding complementary processes—It is difficultto define precisely which aspect of a complementarytreatment provides benefit. During spiritual healing, isit the intent to heal, the preparation, the channelling of“energy,” the accompanying non-judgmental listeningto the user, the relationship with the healer, the healingenvironment, users’ expectations or attitudes, or all ofthese which are important? Complex interventions inconventional medicine result in similar problems.13 Toooften trials provide both limited detail on the actualprocess and little insight into what component of theintervention leads to the effect. Qualitative methodsshould be used to investigate the process. The design ofthe trial should address the therapeutic relationshipand non-specific effects, in addition to any specifictreatment effects.14 At the same time, from the user’sperspective, it is the beneficial effect itself that matters,not how it was brought about.Outcome measuresAppropriate outcome measures—Capturing theessence of how complementary medicine might helppeople is an important challenge. Noteworthy changesafter treatment, such as spiritual change or personalgrowth, might not be measurable. Choice of outcomecriteria in the evaluation of complementary therapies“may make a substantial difference to conclusions as toAcupuncture is a popular complementary treatmentEducation and debate833BMJ VOLUME 325 12 OCTOBER 2002 bmj.com on 25 October 2004 bmj.comDownloaded from whether a particular intervention or therapy is effectiveor not.”14 Holism, which is central to the philosophyand practice of much complementary medicine, needsto be better understood. Detailed qualitative studies,accompanying a trial, could elicit these more elusiveoutcomes and aim to take account of experiential out-come criteria, such as increased self reflection.Illness role—According to some complementaryphilosophies, illness indicates an imbalance in the bodyand symptoms of illness draw attention to this. Forexample, individuals develop colds, backache, oreczema when they are under stress or run down. Ifcomplementary medicine helps people recogniseimbalance and adverse lifestyle factors that contributeto illness and promotes change and resultant improve-ments in health, then research needs to capture thesechanges.Chronic illness—Complementary treatment is oftensought by people with chronic illnesses who may nothave responded to conventional treatment or maywant to explore different forms of care. Any change islikely to be gradual and subtle and to have a relapsingand remitting pattern. As complementary interven-tions tend to aim for long term healing, and not neces-sarily cure, they are well suited to help people withchronic illness. Funding bodies must understand thatlong term follow up of outcomes (specific andnon-specific) is needed for results to be meaningful.Randomised controlled trials work best for exploringshort term effects. Over time, factors other than thetrial treatments could bring about the observed effects.Variations in experience—One person can havedifferent experiences of complementary medicine atdifferent times; different people can have widely differ-ent experiences. Designing a trial to investigate a treat-ment that might be effective immediately for oneperson but take months to be effective for another is aproblem. A “healing moment” may occur at the time oftreatment or some time after. Appropriate experientialoutcome measures should take variations into accountalong with adequately long follow up periods.Sick role—Some users do not respond to treatment.There are those who adapt to the sick role because theyhave an investment in staying ill. Others may besatisfied with the short fix, of feeling peaceful andrelaxed, that is offered by many complementarytreatments and may not take responsibility to make thenecessary changes to improve their health. Inprinciple, randomised controlled trials provide an effi-cient way to deal with this problem. People adoptingthe sick role should be distributed equally throughoutthe arms of a trial through the randomisation process.ConclusionResearchers need to tackle the complex issues inresearching complementary medicine. Studies shouldbe designed to enable a valid evaluation of comple-mentary medicine, on multiple levels, in the humanhealing process.People often turn to complementary treatmentsbecause of frustration with conventional medicine.15They seek that “more”—the essence of what is valued—that complementary treatments offer. Research intocomplementary medicine has a challenge not to missthe “more” and not to design inappropriately to find it.Contributors: SM conceived the idea for the paper and wrotethe first draft which was rewritten with extensive contributionsfrom PT and AL.Funding: No additional funding.Competing interests: SM is a spiritual healer.1 Wright S, Sayre-Adams J. Sacred space: right relationship in health andhealing: not just what we do but who we are. In: Rankin-Box D. The nurses’handbook of complementary therapies. 2nd ed. London: Baillière Tindall,2001.2 Ernst E, Pittler M, Stevinson C, White A, eds. The desktop guide to comple-mentary and alternative medicine: an evidence-based approach. London:Mosby, 2001.3 Cornbleet MA, Ross CSK. Research in complementary medicine isessential. BMJ 2001:322:736-7.4 House of Lords Select Committee on Science and Technology.Complementary and alternative medicine. London: Stationery Office, 2000.(6th report.)5 Angell M, Kassirer JP. Alternative medicine: the risks of untested andunregulated remedies [editorial]. N Engl J Med 1998;339:839-41.6 Brown C. Optimum healing: a life-changing new approach to achieving goodhealth. London: Rider, 1998.7 Long AF, Mercer G. Challenges in researching the effectiveness of com-plementary therapies. J Contemp Health 1999;8:13-9.8 Brown CK. Methodological problems of clinical research into spiritualhealing: perspective. J Altern Complement Med 2000;6:171-6.9 Nahin RL, Straus SE. Research into complementary and alternativemedicine: problems and potential. BMJ 2001;322:161-4.10 Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J BehavTher Exp Psychiatry 1972;3:257-60.11 Morley S, Williams AcdeC, Eccleston C. Systematic review andmeta-analysis of randomised controlled trials of cognitive behaviourtherapy and behaviour therapy for chronic pain in adults, excludingheadache. Pain 1999;80:1-13.12 Stevinson C. Why patients use complementary and alternative medicine.In: Ernst E, Pittler M, Stevinson C, White A, eds. The desktop guide to com-plementary and alternative medicine: an evidence-based approach. London:Mosby, 200113 Medical Research Council. A framework for development and evaluation ofRCTs for complex interventions to improve health. London: MRC, 2000.14 Long AF, Mercer G, Hughes K. Developing a tool to measure holisticpractice: a missing dimension in outcomes measurement within comple-mentary therapies. Complement Ther Med 2000;8:26-31.15 Rees L, Weil A. Integrated medicine. BMJ 2001;322:119-20.(Accepted 10 April 2002)Submitting articles to the BMJWe are now inviting all authors who want to submit a paper tothe BMJ to do so via the web (http://submit.bmj.com).Benchpress is a website where authors deposit theirmanuscripts and editors go to read them and record theirdecisions. Reviewers’ details are also held on the system, andwhen asked to review a paper reviewers will be invited to accessthe site to see the relevant paper. The system is secure, protectedby passwords, so that authors see only their own papers andreviewers see only those they are meant to.Anyone with an internet connection and a web browser can usethe system.The system provides all our guidance and forms and allowsauthors to suggest reviewers for their paper. Authors get animmediate acknowledgement that their submission has beenreceived, and they can watch the progress of their manuscript.The record of their submission, including editors’ and reviewers’reports, remains on the system for future reference.The system itself offers extensive help, and the BMJ ’s editorialoffice will help authors and reviewers if they get stuck.Benchpress is accessed via http://submit.bmj.com or via a linkfrom bmj.comEducation and debate834 BMJ VOLUME 325 12 OCTOBER 2002 bmj.com on 25 October 2004 bmj.comDownloaded from 

Mason, Su

Tovey, Philip

Long, Andrew F.

DigitalGeorgetown

Evaluating Complementary Medicine: Methodological Challenges of Randomised Controlled Trials

https://eprints.whiterose.ac.uk/166/1/masons3.pdf

Evaluating complementary medicine: methodological challenges of randomised controlled trials

Abstract

Similar works

Full text

Available Versions

Crossref

White Rose Research Online

White Rose Research Online

DigitalGeorgetown