Usability analysis of contending electronic health record systems

In this paper, we report measured usability of two leading EHR systems during procurement. A total of 18 users participated in paired-usability testing of three scenarios: ordering and managing medications by an outpatient physician, medicine administration by an inpatient nurse and scheduling of appointments by nursing staff. Data for audio, screen capture, satisfaction rating, task success and errors made was collected during testing. We found a clear difference between the systems for percentage of successfully completed tasks, two different satisfaction measures and perceived learnability when looking at the results over all scenarios. We conclude that usability should be evaluated during procurement and the difference in usability between systems could be revealed even with fewer measures than were used in our study. © 2019 American Psychological Association Inc. All rights reserved.Peer reviewe

Generating Reliable and Responsive Observational Evidence: Reducing Pre-analysis Bias

A growing body of evidence generated from observational data has demonstrated the potential to influence decision-making and improve patient outcomes. For observational evidence to be actionable, however, it must be generated reliably and in a timely manner. Large distributed observational data networks enable research on diverse patient populations at scale and develop new sound methods to improve reproducibility and robustness of real-world evidence. Nevertheless, the problems of generalizability, portability and scalability persist and compound. As analytical methods only partially address bias, reliable observational research (especially in networks) must address the bias at the design stage (i.e., pre-analysis bias) including the strategies for identifying patients of interest and defining comparators. This thesis synthesizes and enumerates a set of challenges to addressing pre-analysis bias in observational studies and presents mixed-methods approaches and informatics solutions for overcoming a number of those obstacles. We develop frameworks, methods and tools for scalable and reliable phenotyping including data source granularity estimation, comprehensive concept set selection, index date specification, and structured data-based patient review for phenotype evaluation. We cover the research on potential bias in the unexposed comparator definition including systematic background rates estimation and interpretation, and definition and evaluation of the unexposed comparator. We propose that the use of standardized approaches and methods as described in this thesis not only improves reliability but also increases responsiveness of observational evidence. To test this hypothesis, we designed and piloted a Data Consult Service - a service that generates new on-demand evidence at the bedside. We demonstrate that it is feasible to generate reliable evidence to address clinicians’ information needs in a robust and timely fashion and provide our analysis of the current limitations and future steps needed to scale such a service

Using electronic health record data to evaluate the epidemiology and management of inflammatory arthritis

In healthcare, there are opportunities to utilise the growth of routine data capture in developing real-world evidence of chronic disease. Inflammatory arthritis encompasses a number of chronic diseases including gout, rheumatoid arthritis (RA) and ankylosing spondylitis (AS), for which timely treatment is necessary to limit joint damage. The hypothesis underlying this thesis is that the epidemiology and management of inflammatory arthritis can be evaluated using routine electronic health record (EHR) data. This was investigated through literature reviews and retrospective studies using a population-based primary care dataset. Gout, AS and RA studies have used EHR data, and this thesis identified variation in methods that influenced reported trends in epidemiology and management. For future studies, considerations were raised for improving the reporting and assessment of EHR-pertinent biases. Incidence and prevalence are uncertain in AS, and have not been investigated in RA in recent years following the incentivisation of diagnostic recording. Between 1998 and 2017, this thesis identified that AS incidence declined for ten years before it stabilised, while RA incidence trends were unclear, and prevalence rose in older patients. In an ageing population, managing these diseases is important and studies should consider changes in coding practice in the study period. There have been efforts to reduce diagnostic delay in AS. This thesis found no improvement in time to diagnosis over two decades, largely driven by delay in rheumatology referral. This is concerning given the importance of treatment in early AS. In RA, shifts in management principles have increased DMARD prescribing over time. This thesis identified that the prescribing of potentially toxic corticosteroids and non-steroidal anti-inflammatories nonetheless persisted across the last 20 years with suboptimal prophylactic therapy. This thesis provides evidence of, and raises considerations for further improving, the use of EHR data in evaluating the epidemiology and management of inflammatory arthritis

Ecology and emergence: Understanding factors that drive variation in process quality and clinical outcomes in general practice

Clinical practice variation (CPV), where differences in healthcare delivery do not reflect differences in patient preferences or clinical need, is considered a hallmark of poor quality care. 'Unwarranted' variation is the focus of mounting policy attention and a growing body of literature, but remains poorly explained and theorised, with ways of determining when variation is warranted only weakly developed. Many assertions around CPV remain under-explored and untested. Much of the literature operates on the assumption that the legitimacy of variation depends on its source or cause, and that variation in processes of care will lead to related variation in outcomes. This doctoral research focuses on two overarching questions relating to CPV in Australian general practice: (1) what is CPV, and how can it be best conceptualised and understood; and (2) what can routinely-collected clinical data tell us about the phenomenon of CPV in general practice? Accordingly, this thesis explores the operationalisation of CPV as a theoretical construct and also examines variation in a series of clinical performance measures for coronary heart disease (CHD) and diabetes. Together, these lines of inquiry constitute a mixed-methods 'sense-making' exercise that seeks an incremental interplay between literature and data, to shed light on the phenomenon of CPV. Data are drawn from a unique dataset of aggregate reporting metrics, using extracted electronic medical record data, among an affiliated group of 36 general practice clinics serving approximately 189,848 patients over a 5-year period. These data are examined descriptively and ultimately analysed using Qualitative Comparative Analysis (QCA) against an empirically derived explanatory framework. Theory development draws on complexity science, especially complex adaptive systems theory, and the disciplines of social epidemiology and health ecology. Results show that a series of discourses have strongly shaped thinking about CPV, converging around a normative 'bad apples' approach to understanding variation. However, CPV may also contribute to healthcare quality in ways that are not well considered, especially in primary care settings. I demonstrate that there may be unconventional but more illuminating ways to conceptualise variation that enable our collective understanding to progress. These include using an ecological framework to conceive CPV as an emergent property of coupled, complex adaptive systems, and employing an equity lens to distinguish between CPV in processes and outcomes of care. In descriptive analyses, I find that variation frequently behaves differently across different measures, with crucial system information contained in the interstices of the data. Contrary to common assumptions, relationships between processes and outcomes of care are not straightforward. Using a framework of factors associated with CPV in general practice management of diabetes and CHD, I confirm that causality is complex and multifactorial, operating at a number of levels. Employing the case-based configurational method of QCA, I show that there may be no single or primary cause for CPV. Instead, clinics can arrive at a particular outcome via multiple independent causal pathways which are themselves multifactorial. These multi-component causes may be defined as much by the interactions between component elements as by individual elements themselves. The same factor may have differential effects within different combinations, or at different scales. These findings suggest that relying on causal explanations to demarcate unwarranted variation may be insufficient. However, both theory and methods require continued development to ensure an adequate understanding of the role and representation of warranted and unwarranted variation in performance measurement systems. Case-based configurational methods such as QCA may have substantial utility in helping to explain and delineate these phenomena