Evaluating the Interrater Agreement and Acceptability of a New Reference Tool for Assessing Respiratory Rate in Children under Five with Cough and/or Difficulty Breathing.

BACKGROUND: Manual assessment of respiratory rate (RR) in children is unreliable, but remains the main method to diagnose pneumonia in low-resource settings. While automated RR counters offer a potential solution, there is currently no gold standard to validate these diagnostic aids. A video-based reference tool is proposed that allows users to annotate breaths and distortions including movement periods, allowing the exclusion of distortions from the computation of RR measures similar to how new diagnostic aids account for distortions automatically. This study evaluated the interrater agreement and acceptability of the new reference tool. METHODS: Annotations were based on previously recorded reference videos of children under five years old with cough and/or difficulty breathing (n = 50). Five randomly selected medical experts from a panel of ten annotated each video. RR measures (breaths per minute, bpm) were computed as the number of annotated certain breaths divided by the length of calm periods after removing annotated distorted periods. RESULTS: Reviewers showed good interrater agreement on continuous RR {standard error of measurement (SEM) [4.8 (95%CI 4.4-5.3)]} and substantial agreement on classification of fast breathing (Fleiss kappa, κ 0.71). Agreement was lowest in the youngest age group [< 2 months: SEM 6.2 (5.4-7.4) bpm, κ 0.48; 2-11 months: 4.7 (4.0-5.8) bpm, κ 0.84; 12-59 months: 2.6 (2.2-3.1) bpm, κ 0.8]. Reviewers found the functionalities of the tool helpful in annotating breaths, but remained uncertain about the validity of their annotations. CONCLUSIONS: Before the new tool can be considered a reference standard for RR assessments, interrater agreement in children younger than 2 months must be improved

Automated Respiratory Rate Counter to Assess Children for Symptoms of Pneumonia: Protocol for Cross-Sectional Usability and Acceptability Studies in Ethiopia and Nepal.

BACKGROUND: Manually counting a child's respiratory rate (RR) for 60 seconds using an acute respiratory infection timer is the World Health Organization (WHO) recommended method for detecting fast breathing as a sign of pneumonia. However, counting the RR is challenging and misclassification of an observed rate is common, often leading to inappropriate treatment. To address this gap, the acute respiratory infection diagnostic aid (ARIDA) project was initiated in response to a call for better pneumonia diagnostic aids and aimed to identify and assess automated RR counters for classifying fast breathing pneumonia when used by front-line health workers in resource-limited community settings and health facilities. The Children's Automated Respiration Monitor (ChARM), an automated RR diagnostic aid using accelerometer technology developed by Koninklijke Philips NV, and the Rad-G, a multimodal RR diagnostic and pulse oximeter developed by Masimo, were the two devices tested in these studies conducted in the Southern Nations, Nationalities, and Peoples' Region in Ethiopia and in the Karnali region in Nepal. OBJECTIVE: In these studies, we aimed to understand the usability of two new automated RR diagnostic aids for community health workers (CHWs; health extension workers [Ethiopia] and female community health volunteers [Nepal]) and their acceptability to CHWs in Ethiopia and Nepal, first-level health facility workers (FLHFWs) in Ethiopia only, and caregivers in both Ethiopia and Nepal. METHODS: This was a prospective, cross-sectional study with a mixed methods design. CHWs and FLHFWs were trained to use both devices and provided with refresher training on all WHO requirements to assess fast breathing. Immediately after training, CHWs were observed using ARIDA on two children. Routine pneumonia case management consultations for children aged 5 years and younger and the device used for these consultations between the first and second consultations were recorded by CHWs in their patient log books. CHWs were observed a second time after 2 months. Semistructured interviews were also conducted with CHWs, FLHFWs, and caregivers. The proportion of consultations with children aged 5 years and younger where CHWs using an ARIDA and adhered to all WHO requirements to assess fast breathing and device manufacturer instructions for use after 2 months will be calculated. Qualitative data from semistructured interviews will be analyzed using a thematic framework approach. RESULTS: The ARIDA project was funded in November 2015, and data collection was conducted between April and December 2018. Data analysis is currently under way and the first results are expected to be submitted for publication in 2020. CONCLUSIONS: This is the first time the usability and acceptability of automated RR counters in low-resource settings have been evaluated. Outcomes will be relevant for policy makers and are important for future research of this new class of diagnostic aids for the management of children with suspected pneumonia. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/14405

Determining the Agreement Between an Automated Respiratory Rate Counter and a Reference Standard for Detecting Symptoms of Pneumonia in Children: Protocol for a Cross-Sectional Study in Ethiopia.

BACKGROUND: Acute respiratory infections (ARIs), primarily pneumonia, are the leading infectious cause of under-5 mortality worldwide. Manually counting respiratory rate (RR) for 60 seconds using an ARI timer is commonly practiced by community health workers to detect fast breathing, an important sign of pneumonia. However, correctly counting breaths manually and classifying the RR is challenging, often leading to inappropriate treatment. A potential solution is to introduce RR counters, which count and classify RR automatically. OBJECTIVE: This study aims to determine how the RR count of an Automated Respiratory Infection Diagnostic Aid (ARIDA) agrees with the count of an expert panel of pediatricians counting RR by reviewing a video of the child's chest for 60 seconds (reference standard), for children aged younger than 5 years with cough and/or difficult breathing. METHODS: A cross-sectional study aiming to enroll 290 children aged 0 to 59 months presenting to pediatric in- and outpatient departments at a teaching hospital in Addis Ababa, Ethiopia, was conducted. Enrollment occurred between April and May 2017. Once enrolled, children participated in at least one of three types of RR evaluations: (1) agreement-measure the RR count of an ARIDA in comparison with the reference standard, (2) consistency-measure the agreement between two ARIDA devices strapped to one child, and (3) RR fluctuation-measure RR count variability over time after ARIDA attachment as measured by a manual count. The agreement and consistency of expert clinicians (ECs) counting RR for the same child with the Mark 2 ARI timer for 60 seconds was also measured in comparison with the reference standard. RESULTS: Primary outcomes were (1) mean difference between the ARIDA and reference standard RR count (agreement) and (2) mean difference between RR counts obtained by two ARIDA devices started simultaneously (consistency). CONCLUSIONS: Study strengths included the design allowing for comparison between both ARIDA and the EC with the reference standard RR count. A limitation is that exactly the same set of breaths were not compared between ARIDA and the reference standard since ARIDA can take longer than 60 seconds to count RR. Also, manual RR counting, even when aided by a video of the child's chest movements, is subject to human error and can result in low interrater reliability. Further work is needed to reach global consensus on the most appropriate reference standard and an acceptable level of agreement to provide ministries of health with evidence to make an informed decision on whether to scale up new automated RR counters. TRIAL REGISTRATION: ClinicalTrials.gov NCT03067558; https://clinicaltrials.gov/ct2/show/NCT03067558. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR1-10.2196/16531

A stitch in time: a cross-sectional survey looking at long lasting insecticide-treated bed net ownership, utilization and attrition in SNNPR, Ethiopia.

BACKGROUND: Since 2002/03, an estimated 4.7 million nets have been distributed in the Southern Nations, Nationalities and Peoples Region (SNNPR) among an at risk population of approximately 10 million people. Evidence from the region suggests that large-scale net ownership rapidly increased over a relatively short period of time. However, little is known about how coverage is being maintained given that the last mass distribution was in 2006/2007. This study sought to determine the status of current net ownership, utilization and rate of long lasting insecticide-treated nets (LLIN) loss in the previous three years in the context of planning for future net distribution to try to achieve sustainable universal coverage. METHODS: A total of 750 household respondents were interviewed across malarious, rural kebeles of SNNPR. Households were randomly selected following a two-stage cluster sampling design where kebeles were defined as clusters. Kebeles were chosen using proportional population sampling (PPS), and 25 households within 30 kebeles randomly chosen. RESULTS: Approximately 67.5% (95%CI: 64.1-70.8) of households currently owned at least one net. An estimated 31.0% (95%CI 27.9-34.4) of all nets owned in the previous three years had been discarded by owners, the majority of whom considered the nets too torn, old or dirty (79.9%: 95%CI 75.8-84.0). Households reported that one-third of nets (33.7%) were less than one year old when they were discarded. The majority (58.8%) of currently owned nets had 'good' structural integrity according to a proportionate Hole Index. Nearly two-thirds of households (60.6%) reported using their nets the previous night. The overriding reason for not using nets was that they were too torn (45.7%, 95% CI 39.1-50.7). Yet, few households are making repairs to their nets (3.7%, 95% CI: 2.4-5.1). CONCLUSIONS: Results suggest that the life span of nets may be shorter than previously thought, with little maintenance by their owners. With the global move towards malaria elimination it makes sense to aim for sustained high coverage of LLINs. However, in the current economic climate, it also makes sense to hark back to simple tools and messages on the importance of careful net maintenance, which could increase their lifespans

Childhood pneumonia diagnostics: community health workers’ and national stakeholders’ differing perspectives of new and existing aids

Background: Pneumonia heavily contributes to global under-five mortality. Many countries use community case management to detect and treat childhood pneumonia. Community health workers (CHWs) have limited tools to help them assess signs of pneumonia. New respiratory rate (RR) counting devices and pulse oximeters are being considered for this purpose. Objective: To explore perspectives of CHWs and national stakeholders regarding the potential usability and scalability of seven devices to aid community assessment of pneumonia signs. Design: Pile sorting was conducted to rate the usability and scalability of 7 different RR counting aids and pulse oximeters amongst 16 groups of participants. Following each pile-sorting session, a focus group discussion (FGD) explored participants’ sorting rationale. Purposive sampling was used to select CHWs and national stakeholders with experience in childhood pneumonia and integrated community case management (iCCM) in Cambodia, Ethiopia, Uganda and South Sudan. Pile-sorting data were aggregated for countries and participant groups. FGDs were audio recorded and transcribed verbatim. Translated FGDs transcripts were coded in NVivo 10 and analysed using thematic content analysis. Comparative analysis was performed between countries and groups to identify thematic patterns. Results: CHWs and national stakeholders across the four countries perceived the acute respiratory infection (ARI) timer and fingertip pulse oximeter as highly scalable and easy for CHWs to use. National stakeholders were less receptive to new technologies. CHWs placed greater priority on device acceptability to caregivers and children. Both groups felt that heavy reliance on electricity reduced potential scalability and usability in rural areas. Device simplicity, affordability and sustainability were universally valued. Conclusions: CHWs and national stakeholders prioritise different device characteristics according to their specific focus of work. The views of all relevant stakeholders, including health workers, policy makers, children and parents, should be considered in future policy decisions, research and development regarding suitable pneumonia diagnostic aids for community use

Usability of pulse oximeters used by community health and primary care workers as screening tools for severe illness in children under five in low resource settings: A cross-sectional study in Cambodia, Ethiopia, South Sudan, and Uganda.

Timely recognition and referral of severely ill children is especially critical in low-resource health systems. Pulse-oximeters can improve health outcomes of children by detecting hypoxaemia, a severity indicator of the most common causes of death in children. Cost-effectiveness of pulse-oximeters has been proven in low-income settings. However, evidence on their usability in community health settings is scarce.This study explores the usability of pulse-oximeters for community health and primary care workers in Cambodia, Ethiopia, South Sudan, and Uganda. We collected observational data, through a nine-task checklist, and survey data, using a five-point Likert scale questionnaire, capturing three usability aspects (effectiveness, efficiency, and satisfaction) of single-probe fingertip and multi-probe handheld devices. Effectiveness was determined by checklist completion rates and task completion rates per checklist item. Efficiency was reported as proportion of successful assessments within three attempts. Standardized summated questionnaire scores (min = 0, max = 100) determined health worker's satisfaction. Influencing factors on effectiveness and satisfaction were explored through hypothesis tests between independent groups (device type, cadre of health worker, country). Checklist completion rate was 78.3% [CI 72.6-83.0]. Choosing probes according to child age showed the lowest task completion rate of 68.7% [CI 60.3%-76.0%]. In 95.6% [CI 92.7%-97.4%] of assessments a reading was obtained within three attempts. The median satisfaction score was 95.6 [IQR = 92.2-99.0]. Significantly higher checklist completion rates were observed with single-probe fingertip devices (p<0.001) and children 12-59 months (p<0.001). We found higher satisfaction scores in South Sudan (p<0.001) and satisfaction varied slightly between devices. From a usability perspective single-probe devices for all age groups should be prioritized for scaled implementation. Further research on easy to use and accurate devices for infants is much needed

Effectiveness outcome: Frequency of full task completion, task completion rates with 95% confidence intervals and p-values.

Effectiveness outcome: Frequency of full task completion, task completion rates with 95% confidence intervals and p-values.</p

STROBE statement—checklist of items that should be included in reports of <i>cross-sectional studies</i>.

STROBE statement—checklist of items that should be included in reports of cross-sectional studies.</p

Description of checklist variables.

Timely recognition and referral of severely ill children is especially critical in low-resource health systems. Pulse-oximeters can improve health outcomes of children by detecting hypoxaemia, a severity indicator of the most common causes of death in children. Cost-effectiveness of pulse-oximeters has been proven in low-income settings. However, evidence on their usability in community health settings is scarce.This study explores the usability of pulse-oximeters for community health and primary care workers in Cambodia, Ethiopia, South Sudan, and Uganda. We collected observational data, through a nine-task checklist, and survey data, using a five-point Likert scale questionnaire, capturing three usability aspects (effectiveness, efficiency, and satisfaction) of single-probe fingertip and multi-probe handheld devices. Effectiveness was determined by checklist completion rates and task completion rates per checklist item. Efficiency was reported as proportion of successful assessments within three attempts. Standardized summated questionnaire scores (min = 0, max = 100) determined health worker’s satisfaction. Influencing factors on effectiveness and satisfaction were explored through hypothesis tests between independent groups (device type, cadre of health worker, country). Checklist completion rate was 78.3% [CI 72.6–83.0]. Choosing probes according to child age showed the lowest task completion rate of 68.7% [CI 60.3%-76.0%]. In 95.6% [CI 92.7%-97.4%] of assessments a reading was obtained within three attempts. The median satisfaction score was 95.6 [IQR = 92.2–99.0]. Significantly higher checklist completion rates were observed with single-probe fingertip devices (p</div