Abstract There is growing recognition that health informatic

Andrew Georgiou

Catherine K Craven

Hannele Hyppönen

Michael Rigby

Nicolette F De Keizer

Philip J Scott

CiteSeerX

Why evidence-based health informatics should have theoretical foundations: Exploring 
the implications for policy and evaluation 
Philip J. Scotta, Nicolette F. de Keizerb, Andrew Georgiouc, Hannele Hyppönend, Catherine K. Cravene, 
Michael Rigbyf 
a Centre for Healthcare Modelling and Informatics, University of Portsmouth, Hampshire, United Kingdom 
b Academic Medical Center, University of Amsterdam, The Netherlands 
c Centre for Health Systems and Safety Research, Macquarie University, New South Wales, Australia 
d National Institute for Health and Welfare, Finland 
e MU Informatics Institute, University of Missouri, United States 
f School of Public Policy and Professional Practice, Keele University, United Kingdom 
 
 
 
Abstract 
There is growing recognition that health informatics should 
be an evidence-based discipline. We propose that a truly 
evidence-based field should also have sound theoretical 
foundations. In this workshop, jointly organized by the EFMI 
Working Group on Health IT Evaluation and the IMIA 
Working Group on Technology Assessment and Quality 
Development, we will explore the implications for policy 
makers and evaluators. Speakers will illustrate potential 
theoretical approaches in policy design and programme 
evaluation. Participants will work in groups to debate the 
areas where theoretical work is most useful. We aim to 
identify priority areas for further work to strengthen the 
theoretical foundations of health informatics. 
Keywords:  
Theory; Evidence; Health informatics; policy; evaluation. 
Introduction 
Medicine is a mature discipline with a strong scientific basis. 
Health informatics (HI) is not yet. The EFMI/IMIA campaign 
for evidence-based HI has led the way in promoting the scien-
tific maturity of the field [1]. This workshop forms part of that 
approach, arguing that HI should not only be evidence-based 
but also needs theoretical foundations. 
Evidence and theory in healthcare 
Evidence-based medicine (EBM) is defined as “the conscien-
tious, explicit, and judicious use of current best evidence in 
making decisions about the care of individual patients” [2]. 
EBM entails formal judgments about the quality of evidence 
for a particular therapy or technique, but only a minority of 
care guidelines are based on ‘strong’ evidence [3, 4]. There 
has been some resistance to evidence-based care protocols that 
can be perceived as insufficiently patient-centric [5], under-
mining clinical autonomy and expert diagnostic skill [6] or not 
keeping up with current clinical experience [7]. Concerns have 
also been expressed that the research data and the way it is 
packaged into guidelines are tainted by incomplete reporting, 
commercial conflicts of interest and publication bias [8]. 
However, the fundamental idea that healthcare should be sci-
entific rather than ‘magical’ seems to be uncontroversial. Re-
liable evidence in medicine is implicitly based on theory, 
whether in the natural sciences, statistics or psychology. 
Evidence and theory in health policy  
Health policy is a broad field that can encompass areas such as 
population health improvement, regulation and payment mod-
els. It has been argued by analogy from EBM that health poli-
cy should be evidence-based [9-11]. Efforts to inform policy 
with evidence include impact assessment, constructive tech-
nology assessment and monitoring implementation. Of course, 
policy by its nature is an expression of ideology. Policy is 
intrinsically about changing the world, not describing it. It is 
therefore subject to confirmation bias [12] and selective and 
tendentious use of data in support of a political aspiration [13]. 
Evidence and theory in health informatics 
A similar tension exists in HI. The aim is to change healthcare 
for the better by improving the use of information. Although 
numerous reports have demonstrated benefits to clinicians, 
patients and management in a wide variety of settings, sys-
tematic reviews have questioned the strength and sufficiency 
of the evidence base [14-16]. New information systems are 
sometimes proclaimed as ‘obviously better’ (the ‘you don’t 
need RCTs to tell you parachutes are a good idea’ argument 
[17]) or explicitly as performance of a political manifesto (the 
‘patients will be given a choice of hospital, whether they want 
it or not’ approach [18, 19]). Projects are naturally subject to 
optimism bias [12], but HI implementations have not been 
uniformly beneficial. Sadly, the record of HI is stained with 
programme failure, negative impacts on care and service de-
livery and (rarely) even increased mortality [20-22]. Para-
chutes are only a good idea if the most proven best design is 
used. There is also a lack of mature measurement science in 
HI [23]. This situation of  real risks as well as putative bene-
fits has led to calls for HI to become an evidence-based disci-
pline, which has had sustained leadership from EFMI and 
IMIA working groups [1]. There is evidence of latent practi-
tioner demand for theory in HI, but minimal current use [24]. 
The importance of theory in mature science 
Kuhn argued that the acquisition of theory was one of the 
markers of the maturity of a field of knowledge [25], as it 
shows that the given domain has moved beyond initial explo-
ration and has formed a solid understanding of its concepts, 
principles and relations. Theory is defined as “A scheme or 
system of ideas or statements held as an explanation or ac-
count of a group of facts or phenomena; a hypothesis that has 
been confirmed or established by observation or experiment, 
and is propounded or accepted as accounting for the known 
facts; a statement of what are held to be the general laws, 
principles, or causes of something known or observed.” [26]. 
This definition shows that theory can be explanatory or pre-
dictive and can be anywhere on the continuum from tentative 
hypothesis to reliable knowledge. “Theory” can incorporate 
both precise mathematical models and softer conceptual mod-
els or sense-making frameworks. Even when a theory has rig-
orous empirical support it is often still called “theory”, as it is 
not simply an unmediated observation of the world but is a 
construct, an abstract model that explains or predicts. Well-
known examples are ‘probability theory’ in statistics, ‘quan-
tum theory’ in physics and ‘rational choice theory’ in econom-
ics. 
What kind of theory is relevant in health 
informatics? 
Suitable theoretical approaches are not axiomatic but must be 
purposively selected by policy makers and evaluators. The 
fundamental question is one of framing: What is the policy 
goal? Whose worldview is being considered? (Government, 
payer/commissioner, industry, patient/citizen, carer, clinician, 
researcher, service manager). Therefore, what conceptual 
framework of ideas and principles is helpful? What are the 
theoretical mechanisms by which the proposed intervention 
will achieve the desired outcomes? [27] 
 
Theories operate at different levels of scale or abstraction. At 
the micro-level, the Theory of Planned Behavior [28] (for in-
dividual actions) has been used where the intervention is fo-
cused on behavioural choices. Mid-level normalization pro-
cess theory has been used to study team interventions [29]. HI 
studies have also employed macro-level theories, which oper-
ate at organizational or even societal level, including Diffu-
sion of Innovations [30] and Activity Theory [31]. Various 
other social science theories have been used in HI. 
 
Another contribution of theory is as an overarching conceptual 
framework for a evaluation design, such as the Theory of 
Change [32], or to explore phenomena and generate theory, as 
in the case of Grounded Theory [33]. 
Workshop 
To lead the group discussions, the following panel of HI 
policy and evaluation experts will facilitate the workshop: 
• Dr Philip Scott, Centre for Healthcare Modelling and 
Informatics, University of Portsmouth, UK 
• Assoc. Prof. Nicolet de Keizer, Academic Medical 
Center, University of Amsterdam, the Netherlands  
• Assoc. Prof. Andrew Georgiou,  Australian Institute 
of Health Innovation, Macquarie University, NSW, 
Australia 
• Assoc. Prof Catherine Craven, MU Informatics 
Institute, University of Missouri, United States 
General Organization of the Workshop Proposal 
Aim of the workshop 
The aim of the workshop is inform participants about existing 
theoretically-based work in HI and debate where theoretically-
informed approaches can add most value in HI. The panel 
speakers will seek to stimulate the group discussions. 
Workshop structure 
The planned duration of the workshop is 90 minutes:  
0-10 min.  PS will introduce the objectives and structure of 
the workshop and introduce the panel speakers. 
10-30 min. Speakers will briefly present on the topics: 
Using the the Friedman & Wyatt typology [34] 
Theory-based assessment of eHealth policies 
Application of Control Theory 
Explanation of group tasks 
Group organization 
30-60 min. Work in groups (4 – 5 participants each) 
Each group will be asked to nominate a group 
moderator (moderating the discussion, 
presenting the group results) and a secretary 
(documenting the key ideas  of the discussion).  
Groups will work on the following questions:  
1. What theoretical foundations does HI have? 
2. Which areas of HI policy and evaluation 
have greatest need of theoretical foundations? 
3. What kinds of theory are likely to be useful 
in these priority areas? 
60-80  min.  Group moderators summarize group 
discussions.  Discussion and conclusions on 
recommended priority areas for further work. 
80-90 min. Moderators will conclude on next steps for 
developing recommendations and will collect 
written documentation of all groups. 
Expected attendees 
• Academic researchers interested in evidence-based 
health informatics. 
• Policy makers responsible for health informatics 
programme planning and evaluation. 
• Practitioners who want to add to the evidence base for 
health informatics interventions. 
Conclusion 
It is hoped that this workshop will result in fruitful discussion 
that suggests priority areas for further work to strengthen the 
theoretical foundations of HI and informs future EFMI/IMIA 
activities in this area. We anticipate producing 
recommendations to be presented in a journal paper. 
Address for correspondence 
Dr Philip Scott, Centre for Healthcare Modelling and Informatics, 
University of Portsmouth, Lion Terrace, Portsmouth PO1 3HE, UK. 
Email: philip.scott@port.ac.uk   
References 
[1] Rigby M, Ammenwerth E, Beuscart-Zephir MC, 
Brender J, Hypponen H, Melia S, et al. Evidence Based Health 
Informatics: 10 Years of Efforts to Promote the Principle. 
Joint Contribution of IMIA WG EVAL and EFMI WG EVAL. 
Yearbook of medical informatics. 2013;8(1):34-46. 
[2] Sackett DL, Rosenberg WM, Gray JA, Haynes RB, 
Richardson WS. Evidence based medicine: what it is and what 
it isn't. BMJ. 1996;312(7023):71-2. 
[3] Garrow JS. What to do about CAM: How much of 
orthodox medicine is evidence based? BMJ. 
2007;335(7627):951. 
[4] Tricoci P, Allen JM, Kramer JM, Califf RM, Smith 
SC, Jr. Scientific evidence underlying the ACC/AHA clinical 
practice guidelines. JAMA. 2009;301(8):831-41. 
[5] Muhlhauser I, Meyer G. Evidence base in guideline 
generation in diabetes. Diabetologia. 2013;56(6):1201-9. 
[6] Kitto S, Villanueva EV, Chesters J, Petrovic A, 
Waxman BP, Smith JA. Surgeons' attitudes towards and usage 
of evidence-based medicine in surgical practice: a pilot study. 
wibble. 2007;77(4):231-6. 
[7] Hay MC, Weisner TS, Subramanian S, Duan N, 
Niedzinski EJ, Kravitz RL. Harnessing experience: exploring 
the gap between evidence-based medicine and clinical 
practice. J Eval Clin Pract. 2008;14(5):707-13. 
[8] Lenzer J. Why we can't trust clinical guidelines. 
BMJ. 2013;346:f3830. 
[9] Evans BA, Snooks H, Howson H, Davies M. How 
hard can it be to include research evidence and evaluation in 
local health policy implementation? Results from a mixed 
methods study. Implement Science. 2013;8:17. 
[10] Ham C, Hunter DJ, Robinson R. Evidence based 
policymaking. BMJ. 1995;310(6972):71-2. 
[11] Walshe K, Rundall TG. Evidence-based 
management: from theory to practice in health care. Milbank 
Q. 2001;79(3):429-57, IV-V. 
[12] Kahneman D. Thinking, fast and slow. London: 
Penguin; 2011. 
[13] Daneault S, Beaudry M, Godin G. Psychosocial 
determinants of the intention of nurses and dietitians to 
recommend breastfeeding. Canadian journal of public health = 
Revue canadienne de sante publique. 2004;95(2):151-4. 
[14] Black AD, Car J, Pagliari C, Anandan C, Cresswell 
K, Bokun T, et al. The impact of eHealth on the quality and 
safety of health care: a systematic overview. PLoS medicine. 
2011;8(1):e1000387. 
[15] Garg AX, Adhikari NK, McDonald H, Rosas-
Arellano MP, Devereaux PJ, Beyene J, et al. Effects of 
computerized clinical decision support systems on practitioner 
performance and patient outcomes: a systematic review. 
JAMA. 2005;293(10):1223-38. 
[16] Goldzweig CL, Orshansky G, Paige NM, Towfigh 
AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient 
portals: evidence on health outcomes, satisfaction, efficiency, 
and attitudes: a systematic review. Ann Intern Med. 
2013;159(10):677-87. 
[17] Halamka JD. Health information technology: shall 
we wait for the evidence? Ann Intern Med. 2006 May 
16;144(10):775-6. PubMed PMID: 16702595. 
[18] Sheppard MK. Fallacy or Functionality: Law and 
Policy of Patient Treatment Choice in the NHS. Health Care 
Analysis. 2014. 
[19] Harding AJ, Sanders F, Lara AM, van Teijlingen ER, 
Wood C, Galpin D, et al. Patient choice for older people in 
English NHS primary care: theory and practice. ISRN family 
medicine. 2014:742676. 
[20] Han YY, Carcillo JA, Venkataraman ST, Clark RS, 
Watson RS, Nguyen TC, et al. Unexpected increased mortality 
after implementation of a commercially sold computerized 
physician order entry system. Pediatrics. 2005 
Dec;116(6):1506-12. 
[21] Ammenwerth E, Shaw NT. Bad health informatics 
can kill--is evaluation the answer? Methods Inf Med. 
2005;44(1):1-3. 
[22] Cresswell KM, Bates DW, Sheikh A. Ten key 
considerations for the successful implementation and adoption 
of large-scale health information technology. Journal of the 
American Medical Informatics Association. 2013;20(e1):e9-
e13. 
[23] Malo C, Neveu X, Archambault PM, Emond M, 
Gagnon MP. Exploring nurses' intention to use a computerized 
platform in the resuscitation unit: development and validation 
of a questionnaire based on the theory of planned behavior. 
Interactive journal of medical research. 2012;1(2):e5. PubMed 
PMID: 23611903. Pubmed Central PMCID: PMC3626134. 
Epub 2012/01/01. eng. 
[24] Scott PJ, Briggs JS, Wyatt JC, Georgiou A. How 
Important is Theory in Health Informatics? A Survey of the 
UK Health Informatics Academic Community. Studies in 
Health Technology and Informatics. 2011;169:223-7. 
[25] Kuhn T. The structure of scientific revolutions. 
Chicago: University of Chicago Press; 1962. 
[26] Oxford English Dictionary. "theory, n.1": Oxford 
University Press; 2013. 
[27] Lilford RJ, Chilton PJ, Hemming K, Girling AJ, 
Taylor CA, Barach P. Evaluating policy and service 
interventions: framework to guide selection and interpretation 
of study end points. BMJ. 2010 2010-08-27 10:54:39;341. 
[28] Ajzen I. TPB Diagram. 2006 [cited 2014 11 
December]. Available from: 
http://people.umass.edu/aizen/tpb.diag.html. 
[29] May CR, Finch T, Ballini L, MacFarlane A, Mair F, 
Murray E, et al. Evaluating complex interventions and health 
technologies using normalization process theory: development 
of a simplified approach and web-enabled toolkit. BMC health 
services research. 2011;11:245. PubMed PMID: 21961827. 
Pubmed Central PMCID: PMC3205031. Epub 2011/10/04. 
eng. 
[30] Greenhalgh T, Robert G, Bate P, McFarlane F, 
Kyriakidou O. Diffusion of Innovations in Health Service 
Organisations: a systematic literature review. Oxford: 
Blackwell; 2005. 
[31] Frambach JM, Driessen EW, van der Vleuten CP. 
Using activity theory to study cultural complexity in medical 
education. Perspectives on medical education. 2014 
Jun;3(3):190-203. eng. 
[32] De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary 
N, Lund C, et al. Theory of Change: a theory-driven approach 
to enhance the Medical Research Council's framework for 
complex interventions. Trials. 2014;15. 
[33] Charmaz K. Constructing grounded theory - a 
practical guide through qualitative analysis. London: Sage; 
2006. 
[34] Friedman C, Wyatt J. Evaluation methods in 
biomedical informatics. 2nd ed. New York: Springer Verlag; 
2006. 
 
 


Why evidence-based health informatics should have theoretical foundations: Exploring the implications for policy and evaluation

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=0BD20E8163DE26D1473B2DA58EC0365C?doi=10.1.1.1046.3360&rep=rep1&type=pdf

Why evidence-based health informatics should have theoretical foundations: Exploring the implications for policy and evaluation

Abstract

Similar works

Full text

Available Versions

CiteSeerX