The Role of Monitoring and Evaluation in Assessing Progress of Operational Research in the EAPHLNP Study Sites in Kenya

Introduction: Studies on performance of Operational Research (OR) projects have outlined the various enabling factors leading to  favorable research outcomes. OR plays a key role in filling the gap between what we know from research and what we do with that  knowledge. This has been expressed over time, based on monitoring (progress indicators) and evaluation (performance indicators) results.Objective: To document the performance of OR activities based on Monitoring and Evaluation (M&E) indicators as well as highlighting lessons learned.Methodology:  M&E framework was developed for the OR studies in three thematic areas; Tuberculosis (TB), malaria enterics as well as components of capacity strengthening and administrative. That was done by KEMRI OR Secretariat in consultation with the East Central Southern African – Health Community (ECSA-HC) Secretariat and the principal investigators of each thematic area from East Africa Community partner states (namely Burundi, Rwanda, Tanzania and Uganda). The framework included outcome indicators for each study, target values defined in accordance with approved protocols. Reporting interval was set at quarterly per year. TB studies had 8 reporting indicators, Enterics had 5 reporting indicators, malaria had 11 indicators, while administrative and capacity building had 15 reporting indicators. The framework was then adopted by the region. In Kenya, the initial roll-out of the research in all three thematic areas wasdone in February 2013. The first quarter of M&E was conducted in the study sites, as defined in the editorial of this journal, in June 2013 while the second “quarter” was carried out in June 2014.Findings: Between February 2013 and June 2014, there was little progress in all the three thematic areas. During the first evaluation, the number of enrolled respondents presumed to have TB at the satellite facilities were 185(6.2%) of the expected target number 3,000 persons. Non-satellite sites enrolled 124(8.2%) of the expected 1,520 persons presumed to have TB as well. In the second evaluation, enrollment at the satellite sites was at 13.3% compared to 1.2% in the non-satellite sites. That represented a two-fold percentage increase in the satellite sites compared to nonsatellite sites. Using ZN outcome indicator, there were differences in the number of actual TB cases detected in both sites compared with the target values. Number of TB – cases detected using ZN at satellite sites rose from 7.8% (target of 784 cases) to 11.0% (target of 1725 cases). In the non-satellite sites, there was a decline from 3.1% (target of 508 cases) to 1.1% (target of 1118 cases). In the Enteric Study, there was a marginal decline in the number of patients recruited from 21.6% (target of 1440 patients) as at the first evaluation to 17.2% (target of 1800 patients) in the second evaluation. For malaria study, a total of 333 patients had been enrolled against a targeted of 300 patients into the study representing an over enrollment of 111% from one site. For the administrative indicators, the OR Secretariat had over attained in three target areas namely publication and sharing of OR findings in country and regional bulletins and held OR-Technical working group meetings.Lesson Learnt: Low performance in achieving indicators in both TB and enteric studies was resulted by including; high staff turnover particularly in the non-satellite sites, high workload and breakdown in communication among sites' personnel in regard to participation in research activities. In the satellite sites where study interventions were provided, better performance in achieving indicators was attributed to improved capacity in personnel and other non-financial motivational aspects, such as site exchange visits, refresher courses and frequent attendance of project workshops and meetings. Scientists from KEMRI assisting in patient recruitment, specimen collection and shipment alongside with study site staff resulted to over-attainment of performance indicators. Such was demonstrated in the malaria study. The observed lag time between the various M&E field visits by OR team could have partially contributed to the missed opportunities of identification and correction of any deviations from the project protocols. Changes in leadership especially at the project top management at the Ministry of Health affected the overall performance of OR activitie due to delays in disbursement of funds and delays in obtaining no-objection to incur expenditure on essential activities that were not originally in the approved annual workplans. Conclusion and Recommentdation: Itt was established that several factors, some of which could have been augmented if the M&E exercise was conducted in accordance with the framework, affected the achievement of study indicators. M&E component is a crucial activity especially in tracking research progress and should be conducted consistently within the stipulated timeline. This will subsequently provide opportunities of early identification and correction of any deviations from the protocol. Keywords: Operational Research, Monitoring & Evaluation, indicators, targets, performance, factor

Application of Barcode Technology to Enhance Electronic Quality-Assured Data Collection and Analysis in Operational Research EAPHLN project study sites in Kenya

Background: Barcode Technology is a replacement for the traditional keyboard data entry. The East Africa Public Health Laboratory Networking (EAPHLN) Project operational research activities anticipated enormous data generation from different geographical sites and health care site teams which necessitated the development of the system. This paper describes the use of barcode technology to enhance electronic quality assured data collection and analysis in operational research studies in Kenya. Methodology: Barcode labels consisting of an encoded 9-digit unique identification figures were generated and centralized at KEMRI for nine study sites. At the sites, the label placement was done in the following sequence: patient card, consent form, questionnaire and clinical forms by the clinicians. Specimens and shipment form from the same patient with two matching identifier labels by the laboratory staff. The specimen barcode label contained additional information including specimen type and collection date. On receipt at the KEMRI laboratories, the specimen barcodes were scanned in the reception module of the electronic data management system (eDMS). An  additional barcode label was generated with a laboratory number that was affixed to the specimen and scanned into the testing equipment that generated outputs. Findings:  Implementation of the barcode technology in the study sites, involved introduction of a new workflow methodology. This impacted positively on patient recruitment and sample collection process. The barcode labels served as identifiers when used during enrollment which provided an accurate patient and specimen tracking system. This was evident as all specimens delivered had complete accompanying documents with 92% of all barcodes being successfully scanned. Poor storage and handling of the barcode labels contributed to the inability to the scanning. Clinical, demographic and laboratory information to be viewed directly without the need to track down the patient’s source documents. The barcode system ensured the following: the confidentiality of patients was maintained;  Automation specimen identification on tests eliminating need for relabeling result output reports; fewer errors. Conclusion: Patients’ data linkages and verification from all study sites and the reference laboratory leading to increased efficiency and effectiveness in maintaining patient records. We recommend refresher trainings and supervisory visits to ensuring proper implementation and utilization of the barcode labels. Keywords: Barcode Scanning; Data Linkages; Patients; Operational Research, Datalogic Powersca

Inter / Intra Examiner Calibration of Clinical Signs and Symptoms in Comparison with Laboratory Results in the Diagnosis of Pulmonary Tuberculosis in Patients from EAPHLN Project Study Sites in Kenya

Introduction: The clinical diagnosis of pulmonary tuberculosis (PTB) is based on occurrence of four cardinal signs and symptoms which include current cough, night sweats, weight loss, or low grade fever. However few studies have determined the validity and reliability of these diagnoses by intra and inter-examiner calibration of clinicians for appropriateness of detection of tuberculosis (TB) in resource constrained settings. Objective: The study's aim was to determine the sensitivity of concordance and reliability (Kappa values) of inter-examiner and intra-examiner findings of clinicians in the EAPHLN project. Methodology: The study was a cross-sectional study in nine sites. It included 155 patients for intraexaminer and 57 patients for inter-examiner calibrations selected from eligible people with symptoms or signs suggestive of TB during the implementation of the East African Public Health Laboratory Network Project (EAPHLNP) in Kenya. TB clinical symptoms and signs were recorded in a structured medical form included the following: productive cough, weight loss, night sweats, low grade fever (classical cardinal signs and symptoms). Using quality assurance sampling for a total population of ten thousand people with symptoms or signs suggestive of TB from the sites with a minimum defective sample acceptable of 0 and a probability of defect accepted of 1% andan alpha of 5%, the sample size of repeatable samples is 262 for total patients for the sites per year. Intra-examiner calibration involved examination of the same patient independently by the same clinician within one day interval. Inter-examiner calibration involved examination of the same patient by two clinicians independently the same day. Calibration of the clinical tools used during examination of patients was done. TB laboratory diagnosis was first done by sputum  smear microscopy Ziehl–Neelsen stain.(ZN), secondly by optimized sputum smear microscopy with a Light Emitting Diode microscope ( LED) or fluorescent microscopy(FM), and thirdly by Gene-expert technique (Gene Xpert or Gx). The results from the clinicians and reference laboratory findings for these patients were entered in a computer, verified and analyzed in SPSS for reliability statistics. These unweighted Cohen Kappa scores were interpreted as follows: poor0.01–0.20, moderate 0.21–0.40, fair 0.41–0.60, good 0.61–0.80, or excellent 0.81–1.0 based on the agreement between the intraexaminer and inter-examiner findings Results: A significant difference was found between concordant diagnosis of a least 4 signs and symptoms of TB compared to fewer by the same examiner on the same patient in all TB test/HIV status categories except the ZN positive /HIV positives and GeneXpert negative /HIV positives and HIV negative categories. The highest sensitivity rate was 81.8 %( 95%CI=52.3-94.9) in the Gx+ve/HIV+ve category. The significance difference in senstivity results of TB/HIV test vs at least presence of the 4 signs and symptoms. However, This did not occur in ZN+ve/HIV+ve, FM+ve/HIV+ve , Gx+ve/HIV+ve Gx+ve/HIV-ve categories. Kappa values for cough and fever were consistently significantly higher than zero kappa. Conclusion: Excellent kappa can be achieved in low resource settings by clinician using all four cardinal signs and symptoms of TB with laboratory results. There is possibility of using the clinical diagnosis using the four signs and symptoms where laboratory diagnosis is not present but specificity is low. Good clinical practice would improve the specificity

Effects of Sputum Quality on Xpert® MTB/RIF Results in The Detection of Mycobacterium Tuberculosis from Persons Presumed To Have TB in EAPHLN Project Operational Research Study Sites in Kenya

Background: The common problem in tuberculosis (TB) management is mis-diagnosis or underdiagnosis of cases leading to high  morbidity and mortality. In order to reverse this, new diagnostic tools for detection of Mycobacterium tuberculosis (MTB) the causative agent of TB disease have been developed. However, in the evaluation process of these tools many studies have not considered attributes of sputum quality in their testing algorithm. Objectives: This study aimed at evaluating the effect of sputum quality in detection of MTB when using Xpert® MTB/RIF (GeneXpert) among patients presumed to have pulmonary TB. Methodology: Between February 2013 and August 2014 a total of 3585 Spot and morning sputum specimens were collected from 1918 persons presumed to have pulmonary TB enrolled in nine East Africa Public Health Laboratory Networking (EAPHLN) Project study sites in Kenya. The mean age was 40 (+17SD) years ranging between 18 and 95 years. Some of these specimens (512) were rejected and 3073 were analyzed. The specimens were appropriately packaged and transported to KEMRI Mycobacteriology research laboratory where they were macroscopically characterized into muco-purulent; mucoid, salivary or blood stained. The sputum specimens having reddish color was labeled as blood-stained sputum. Each specimen was processed for GeneXpert testing and culture. Results: Upon macroscopic characterization, out of the 3073 specimens received, 46.1% were mucoid, 44% salivary, 7.5% muco-purulent,  while 2.4% were blood stained. Bivariate analysis revealed that there was a significant association between sputum quality and gender  (p<0.001), age (p=0.022), specimen type (p<0.001), and HIV status (p=0.003). Performance of GeneXpert on the different specimen  categories, muco-purulent (85.7%; CI 95%, 67.4-100%) and mucoid (85.3%, 95%CI: 77.393.3%) specimens had higher sensitivity when  compared to salivary specimens (76.7%, 95%CI: 64.1-89.3). However when stratified by HIV status, GeneXpert detected more MTB on salivary specimens produced by HIV positive (85.7%, 95%CI: 67.4-100%) patients than those from HIV negative patients (71.4%, 95%CI: 52.1- 90.7%). Conclusion: By macroscopic characterization, any sputum specimen type from HIV Positive or Negative persons presumed to have  tuberculosis can be used in diagnosis of tuberculosis regardless of sputum quality classification. However, the sensitivity of GeneXpert was higher in morning sputum specimens that were muco-purulent and mucoid with high MTB yield than in spot sputum specimens which were salivary with low MTB yield. Also, GeneXpert sensitivity was higher, though not significant, in salivary specimens from HIV positive individuals than those of HIV negative individuals. Sputum specimen quality assessment should be considered as an integral part of routine laboratory diagnosis of TB especially in HIV negative individuals. Keywords: Specimen quality, Sputum, Tuberculosis, Xpert® MTB/RIF, GeneXper

Redefining the relevance of established cancer cell lines to the study of mechanisms of clinical anti-cancer drug resistance

Although in vitro models have been a cornerstone of anti-cancer drug development, their direct applicability to clinical cancer research has been uncertain. Using a state-of-the-art Taqman-based quantitative RT-PCR assay, we investigated the multidrug resistance (MDR) transcriptome of six cancer types, in established cancer cell lines (grown in monolayer, 3D scaffold, or in xenograft) and clinical samples, either containing >75% tumor cells or microdissected. The MDR transcriptome was determined a priori based on an extensive curation of the literature published during the last three decades, which led to the enumeration of 380 genes. No correlation was found between clinical samples and established cancer cell lines. As expected, we found up-regulation of genes that would facilitate survival across all cultured cancer cell lines evaluated. More troubling, however, were data showing that all of the cell lines, grown either in vitro or in vivo, bear more resemblance to each other, regardless of the tissue of origin, than to the clinical samples they are supposed to model. Although cultured cells can be used to study many aspects of cancer biology and response of cells to drugs, this study emphasizes the necessity for new in vitro cancer models and the use of primary tumor models in which gene expression can be manipulated and small molecules tested in a setting that more closely mimics the in vivo cancer microenvironment so as to avoid radical changes in gene expression profiles brought on by extended periods of cell culture