Background: Triage is one of the core requirements for the provision of effective emergency care and has been shown to reduce patient mortality. However, in low- and middle-income countries this strategy is underused, under-resourced and poorly researched. OBJECTIVE: To assess the inter- and intra-rater reliability and accuracy of nurse triage ratings when using the South African Triage Scale (SATS) in an emergency department (ED) in Timergara, Pakistan. METHODS: Fifteen ED nurses assigned triage ratings to a set of 42 reference vignettes (written case reports of ED patients) under classroom conditions. Inter-rater reliability was assessed by comparing these triage ratings; intra-rater reliability was assessed by asking the nurses to re-triage 10 random vignettes from the original set of 42 vignettes and comparing these duplicate ratings. Accuracy of the nurse ratings was measured against the reference standard. RESULTS: Inter-rater reliability was substantial (intraclass correlation coefficient 0.77; 95% confidence interval (CI) 0.69 - 0.85). The intrarater agreement was also high with 87% exact agreement (95% CI 67 - 100) and 100% agreement allowing for a one-level discrepancy in triage ratings. Overall, the SATS had high specificity (97%) and moderate sensitivity (70%). Across all acuity levels the proportion of over-triage did not exceed the acceptable threshold of 30 - 50%. Under-triage was acceptable for all except emergency cases (66%). CONCLUSION: ED nurses in Pakistan can reliably use the SATS to assign triage acuity ratings. While the tool is accurate for 'very urgent' and 'routine' cases, importantly, it may under-triage 'emergency' cases requiring immediate attention. Approaches that will improve accuracy and validity are discussed
