Australian Commission on Safety and Quality in Health Care
Abstract
Paper-based observation charts are the principal means of monitoring changes to patients’ vital signs. There is considerable variation in the design of observation charts and a lack of empirical research on the performance of different designs. This report describes the results of a study carried out as part of a project funded by the Australian Commission for Safety and Quality in Health Care and Queensland Health to investigate the design and use of observation charts in recognising and managing patient deterioration, including the design and evaluation of a new adult observation chart that incorporated human factors principles. The first phase of this project involved using a procedure known as heuristic analysis to review 25 observation charts from Australia and New Zealand. 1,189 usability problems, which could lead to errors in recording data and identifying patient deterioration, were identified in the charts. The results from the heuristic analysis were used to design a new chart (the Adult Deterioration Detection System [ADDS] chart) based on human factors principles and current best practice. The study described in this report involved an empirical comparison of six charts (two versions of the ADDS chart, two existing charts rated as “well designed” in the heuristic analysis, one existing chart rated as being of “average design”, and one existing chart rated as “poorly designed”). Novices (individuals who were unfamiliar with using patient charts) and health professionals (doctors and nurses) were recruited as participants. Each chart design was shown to each participant four times displaying different physiological data with one abnormal vital sign (e.g. a high systolic blood pressure), and four times displaying different normal physiological data. After memorising the normal ranges for each vital sign, participants had to classify the physiological data on the charts as “normal” or “abnormal”. Error rates (the proportion of trials where participants made an incorrect normal/abnormal judgement) and response time (the time to read the chart and make the judgement) were measured. Results indicated that chart design had a statistically significant effect on both error rates and response time, with the charts identified as having better design tending to yield fewer errors and shorter decision times. Specifically, the two versions of the ADDS chart outperformed all the existing charts on both metrics, where the other charts yielded between 2.5 and 3.3 times as many errors as the ADDS chart. There was no significant difference between novices and health professionals in error rates for any chart, but the health professionals were significantly faster than novices at making their decisions for the charts rated as “average” and “poor”. There was no significant difference between doctors and nurses on either of the two performance measures for any of the charts. These data indicate that differences in the design of observation charts have a profound impact on chart users’ decisions regarding patients’ vital signs as well as the time it takes to make such decisions. Based on the current data, it appears that the ADDS chart is significantly better at signalling patient deterioration than other currently available charts
