Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety

The purpose of this paper is to argue for an integration of cognitive and socio-technical approaches to assessing the impact of health information systems. Historically, health informatics research has examined the cognitive and socio-technical aspects of health information systems separately. In this paper we argue that evaluations of health information systems should consider aspects related to cognition as well as socio-technical aspects including impact on workflow (i.e. an integrated view). Using examples from the study of technology-induced error in healthcare, we argue for the use of simulations to evaluate the cognitive-socio-technical impacts of health information technology [36]. Implications of clinical simulations and analysis of cognitive-social-technical impacts are discussed within the context of the system development life cycle to improve health information system design, implementation and evaluation

Usability evaluation of a clinical decision support tool for osteoporosis disease management

<p>Abstract</p> <p>Background</p> <p>Osteoporosis affects over 200 million people worldwide at a high cost to healthcare systems. Although guidelines are available, patients are not receiving appropriate diagnostic testing or treatment. Findings from a systematic review of osteoporosis interventions and a series of focus groups were used to develop a functional multifaceted tool that can support clinical decision-making in osteoporosis disease management at the point of care. The objective of our study was to assess how well the prototype met functional goals and usability needs.</p> <p>Methods</p> <p>We conducted a usability study for each component of the tool--the Best Practice Recommendation Prompt (BestPROMPT), the Risk Assessment Questionnaire (RAQ), and the Customised Osteoporosis Education (COPE) sheet--using the framework described by Kushniruk and Patel. All studies consisted of one-on-one sessions with a moderator using a standardised worksheet. Sessions were audio- and video-taped and transcribed verbatim. Data analysis consisted of a combination of qualitative and quantitative analyses.</p> <p>Results</p> <p>In study 1, physicians liked that the BestPROMPT can provide customised recommendations based on risk factors identified from the RAQ. Barriers included lack of time to use the tool, the need to alter clinic workflow to enable point-of-care use, and that the tool may disrupt the real reason for the visit. In study 2, patients completed the RAQ in a mean of 6 minutes, 35 seconds. Of the 42 critical incidents, 60% were navigational and most occurred when the first nine participants were using the stylus pen; no critical incidents were observed with the last six participants that used the touch screen. Patients thought that the RAQ questions were easy to read and understand, but they found it difficult to initiate the questionnaire. Suggestions for improvement included improving aspects of the interface and navigation. The results of study 3 showed that most patients were able to understand and describe sections of the COPE sheet, and all considered discussing the information with their physicians. Suggestions for improvement included simplifying the language and improving the layout.</p> <p>Conclusions</p> <p>Findings from the three studies informed changes to the tool and confirmed the importance of usability testing on all end users to reduce errors, and as an important step in the development process of knowledge translation interventions.</p

Group differences in physician responses to handheld presentation of clinical evidence: a verbal protocol analysis

<p>Abstract</p> <p>Background</p> <p>To identify individual differences in physicians' needs for the presentation of evidence resources and preferences for mobile devices.</p> <p>Methods</p> <p>Within-groups analysis of responses to semi-structured interviews. Interviews consisted of using prototypes in response to task-based scenarios. The prototypes were implemented on two different form factors: a tablet style PC and a pocketPC. Participants were from three user groups: general internists, family physicians and medicine residents, and from two different settings: urban and semi-urban. Verbal protocol analysis, which consists of coding utterances, was conducted on the transcripts of the testing sessions. Statistical relationships were investigated between staff physicians' and residents' background variables, self-reported experiences with the interfaces, and verbal code frequencies.</p> <p>Results</p> <p>47 physicians were recruited from general internal medicine, family practice clinics and a residency training program. The mean age of participants was 42.6 years. Physician specialty had a greater effect on device and information-presentation preferences than gender, age, setting or previous technical experience. Family physicians preferred the screen size of the tablet computer and were less concerned about its portability. Residents liked the screen size of the tablet, but preferred the portability of the pocketPC. Internists liked the portability of the pocketPC, but saw less advantage to the large screen of the tablet computer (F[2,44] = 4.94, p = .012).</p> <p>Conclusion</p> <p>Different types of physicians have different needs and preferences for evidence-based resources and handheld devices. This study shows how user testing can be incorporated into the process of design to inform group-based customization.</p

An interdisciplinary team communication framework and its application to healthcare 'e-teams' systems design

<p>Abstract</p> <p>Background</p> <p>There are few studies that examine the processes that interdisciplinary teams engage in and how we can design health information systems (HIS) to support those team processes. This was an exploratory study with two purposes: (1) To develop a framework for interdisciplinary team communication based on structures, processes and outcomes that were identified as having occurred during weekly team meetings. (2) To use the framework to guide 'e-teams' HIS design to support interdisciplinary team meeting communication.</p> <p>Methods</p> <p>An ethnographic approach was used to collect data on two interdisciplinary teams. Qualitative content analysis was used to analyze the data according to structures, processes and outcomes.</p> <p>Results</p> <p>We present details for team meta-concepts of structures, processes and outcomes and the concepts and sub concepts within each meta-concept. We also provide an exploratory framework for interdisciplinary team communication and describe how the framework can guide HIS design to support 'e-teams'.</p> <p>Conclusion</p> <p>The structures, processes and outcomes that describe interdisciplinary teams are complex and often occur in a non-linear fashion. Electronic data support, process facilitation and team video conferencing are three HIS tools that can enhance team function.</p

Development of a prototype clinical decision support tool for osteoporosis disease management: a qualitative study of focus groups

<p>Abstract</p> <p>Background</p> <p>Osteoporosis affects over 200 million people worldwide, and represents a significant cost burden. Although guidelines are available for best practice in osteoporosis, evidence indicates that patients are not receiving appropriate diagnostic testing or treatment according to guidelines. The use of clinical decision support systems (CDSSs) may be one solution because they can facilitate knowledge translation by providing high-quality evidence at the point of care. Findings from a systematic review of osteoporosis interventions and consultation with clinical and human factors engineering experts were used to develop a conceptual model of an osteoporosis tool. We conducted a qualitative study of focus groups to better understand physicians' perceptions of CDSSs and to transform the conceptual osteoporosis tool into a functional prototype that can support clinical decision making in osteoporosis disease management at the point of care.</p> <p>Methods</p> <p>The conceptual design of the osteoporosis tool was tested in 4 progressive focus groups with family physicians and general internists. An iterative strategy was used to qualitatively explore the experiences of physicians with CDSSs; and to find out what features, functions, and evidence should be included in a working prototype. Focus groups were conducted using a semi-structured interview guide using an iterative process where results of the first focus group informed changes to the questions for subsequent focus groups and to the conceptual tool design. Transcripts were transcribed verbatim and analyzed using grounded theory methodology.</p> <p>Results</p> <p>Of the 3 broad categories of themes that were identified, major barriers related to the accuracy and feasibility of extracting bone mineral density test results and medications from the risk assessment questionnaire; using an electronic input device such as a Tablet PC in the waiting room; and the importance of including well-balanced information in the patient education component of the osteoporosis tool. Suggestions for modifying the tool included the addition of a percentile graph showing patients' 10-year risk for osteoporosis or fractures, and ensuring that the tool takes no more than 5 minutes to complete.</p> <p>Conclusions</p> <p>Focus group data revealed the facilitators and barriers to using the osteoporosis tool at the point of care so that it can be optimized to aid physicians in their clinical decision making.</p

Capturing and testing perceptual-cognitive expertise: A comparison of stationary and movement response methods

Numerous methods have been used to study expertise and performance. In the present article, we compare the cognitive thought processes of skilled soccer players when responding to film-based simulations of defensive situations involving two different experimental conditions. Participants either remained stationary in a seated position (n = 10) or were allowed to move (n = 10) in response to life-size film sequences of 11 versus 11 open-play soccer situations viewed from a player’s perspective. Response accuracy and retrospective verbal reports of thinking were collected across the two task conditions. In the movement-based response group, participants generated a greater number of verbal report statements, including a higher proportion of evaluation, prediction, and action planning statements, than did participants in the stationary group. Findings suggest that the processing strategies employed during performance differ depending on the nature of the response required of participants. Implications for behavioral methods and experimental design are discussed

Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians

BACKGROUND: Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems. METHODS: We have developed a task-oriented questionnaire for evaluating EMR systems from the clinician's perspective. The key feature of the questionnaire is a list of 24 general clinical tasks. It is applicable to physicians of most specialties and covers essential parts of their information-oriented work. The task list appears in two separate sections, about EMR use and task performance using the EMR, respectively. By combining these sections, the evaluator may estimate the potential impact of the EMR system on health care delivery. The results may also be compared across time, site or vendor. This paper describes the development, performance and validation of the questionnaire. Its performance is shown in two demonstration studies (n = 219 and 80). Its content is validated in an interview study (n = 10), and its reliability is investigated in a test-retest study (n = 37) and a scaling study (n = 31). RESULTS: In the interviews, the physicians found the general clinical tasks in the questionnaire relevant and comprehensible. The tasks were interpreted concordant to their definitions. However, the physicians found questions about tasks not explicitly or only partially supported by the EMR systems difficult to answer. The two demonstration studies provided unambiguous results and low percentages of missing responses. In addition, criterion validity was demonstrated for a majority of task-oriented questions. Their test-retest reliability was generally high, and the non-standard scale was found symmetric and ordinal. CONCLUSION: This questionnaire is relevant for clinical work and EMR systems, provides reliable and interpretable results, and may be used as part of any evaluation effort involving the clinician's perspective of an EMR system

Modelling Conditions and Health Care Processes in Electronic Health Records : An Application to Severe Mental Illness with the Clinical Practice Research Datalink

BACKGROUND: The use of Electronic Health Records databases for medical research has become mainstream. In the UK, increasing use of Primary Care Databases is largely driven by almost complete computerisation and uniform standards within the National Health Service. Electronic Health Records research often begins with the development of a list of clinical codes with which to identify cases with a specific condition. We present a methodology and accompanying Stata and R commands (pcdsearch/Rpcdsearch) to help researchers in this task. We present severe mental illness as an example. METHODS: We used the Clinical Practice Research Datalink, a UK Primary Care Database in which clinical information is largely organised using Read codes, a hierarchical clinical coding system. Pcdsearch is used to identify potentially relevant clinical codes and/or product codes from word-stubs and code-stubs suggested by clinicians. The returned code-lists are reviewed and codes relevant to the condition of interest are selected. The final code-list is then used to identify patients. RESULTS: We identified 270 Read codes linked to SMI and used them to identify cases in the database. We observed that our approach identified cases that would have been missed with a simpler approach using SMI registers defined within the UK Quality and Outcomes Framework. CONCLUSION: We described a framework for researchers of Electronic Health Records databases, for identifying patients with a particular condition or matching certain clinical criteria. The method is invariant to coding system or database and can be used with SNOMED CT, ICD or other medical classification code-lists