OBJECTIVES: To examine the consistency of triage outcomes by nurses using four types of computerised
decision support software in NHS Direct.
METHODS: 119 scenarios were constructed based on calls to ambulance services that had been
assigned the lowest priority category by the emergency medical dispatch systems in use. These
scenarios were presented to nurses working in four NHS Direct call centres using different computerised
decision support software, including the NHS Clinical Assessment System.
RESULTS: The overall level of agreement between the nurses using the four systems was “fair” rather than
“moderate” or “good” (k=0.375, 95% CI: 0.34 to 0.41). For example, the proportion of calls triaged
to accident and emergency departments varied from 22% (26 of 119) to 44% (53 of 119). Between
21% (25 of 119) and 31% (37 of 119) of these low priority ambulance calls were triaged back to the
999 ambulance service. No system had both high sensitivity and specificity for referral to accident and
emergency services.
CONCLUSIONS: There were large differences in outcome between nurses using different software systems
to triage the same calls. If the variation is primarily attributable to the software then standardising on a
single system will obviously eliminate this. As the calls were originally made to ambulance services and
given the lowest priority, this study also suggests that if, in the future, ambulance services pass such
calls to NHS Direct then at least a fifth of these may be passed back unless greater sensitivity in the
selection of calls can be achieved
