Data entry quality of double data entry vs automated form processing technologies: A cohort study validation of optical mark recognition and intelligent character recognition in a clinical setting

Background and Aims Patient-reported outcome measures (PROMs) are increasingly used in health services. Paper forms are still often used to register such data. Manual double data entry (DDE) has been defined as the gold standard for transferring data to an electronic format but is laborious and costly. Automated form processing (AFP) is an alternative, but validation in a clinical context is warranted. The study objective was to examine and validate a local hospital AFP setup. Methods Patients over 18 years of age who were scheduled for knee or hip replacement at Stavanger University Hospital from 2014 to 2017 who answered PROMs were included in the study and contributed PROM data. All paper PROMs were scanned using the AFP techniques of optical mark recognition (OMR) and intelligent character recognition (ICR) and were processed by DDE by health secretaries using a data entry program. OMR and ICR were used to capture different types of data. The main outcome was the proportion of correctly entered numbers, defined as the same response recorded in AFP and DDE or by consulting the original paper questionnaire at the data field, item, and PROM level. Results A total of 448 questionnaires from 255 patients were analyzed. There was no statistically significant difference in error proportions per 10 000 data fields between OMR and DDE for data from check boxes (3.52 95% confidence interval (CI) 2.17 to 5.72 and 4.18 (95% CI 2.68-6.53), respectively P = .61). The error proportion for ICR (nine errors) was statistically significantly higher than that for DDE (two errors), that is, 3.53 (95% CI 1.87-6.57) vs 0.78 (95% CI 0.22-2.81) per 100 data fields/items/questionnaires; P = .033. OMR (0.04% errors) outperformed ICR (3.51% errors; P < .001), Fisher's exact test. Conclusions OMR can produce an error rate that is comparable to that of DDE. In our setup, ICR is still problematic and is highly dependent on manual validation. When AFP is used, data quality should be tested and documented.publishedVersio

Assessing the content validity of the Manchester–Oxford Foot Questionnaire in surgically treated ankle fracture patients: a qualitative study

Background Roughly 10% of fractures in adults are ankle fractures. These injuries are found in both sexes and present with different fracture characteristics. The treatment varies with the patients’ biology and fracture type, and the goals are to restore stability, prevent pain and maintain ankle function. Clinicians generally use outcomes like assessment of radiography, pain level, or function. The use of patient-reported outcome measures is increasing, and the Manchester–Oxford Foot Questionnaire (MOXFQ) has been shown to have good measurement properties when validated in patients with foot and ankle disorders. However, the instrument has not been validated for ankle fracture patients. This study aims to assess the content validity of the items in MOXFQ in surgically treated ankle fracture patients. Methods A qualitative deductive design was used to investigate patients’ response process of the MOXFQ. Individual interviews were conducted using cognitive interviewing based on the theoretical framework of the 4-step model by Tourangeau. Adult patients that were surgically treated for an ankle fracture between four weeks and 18 months were purposively sampled, and interviews followed a semi-structured interview guide. The predetermined categories were comprehension, retrieval, judgement, and response. Results Seventeen respondents (65% females) were interviewed. Respondents’ age ranged from 27 to 76 years. Some of the respondents in the early recovery phase were limited by post-operative restrictions and did not find the items in the walking/standing domain relevant. Respondents that were allowed weight-bearing as tolerated (WBAT) were able to recall relevant information for most items. Respondents with time since surgery more than 12 months had less pain and remembered fewer relevant episodes in the recall period. Items in the social interaction domain contained ambiguous questions and were generally considered less important by respondents. The summary index score lacked important concepts in measuring overall quality of life. Conclusions Pain was a central concept in the post-operative recovery of ankle fracture patients. The MOXFQ-subscales for pain and walking/standing had acceptable content validity in patients that were allowed WBAT. The social interaction-subscale and the summary index score had insufficient content validity for this patient population.publishedVersio

Ankle fractures: a systematic review of patient‑reported outcome measures and their measurement properties

Purpose Ankle fractures are commonly occurring fractures, especially in the aging population, where they often present as fragility fractures. The disease burden and economic costs to the patient and society are considerable. Choosing accurate outcome measures for the evaluation of the management of ankle fractures in clinical trials facilitates better decision-making. This systematic review assesses the evidence for the measurement properties of patient-reported outcome measures (PROMs) used in the evaluation of adult patients with ankle fractures. Methods Searches were performed in CINAHL, EMBASE, Medline and Google Scholar from the date of inception to July 2021. Studies that assessed the measurement properties of a PROM in an adult ankle fracture population were included. The included studies were assessed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology for systematic reviews of PROMs. Results In total, 13 different PROMs were identified in the 23 included articles. Only the Ankle Fracture Outcome of Rehabilitation Measure (A-FORM) presented some evidence on content validity. The Olerud-Molander Ankle Score (OMAS) and Self-reported Foot and Ankle Score (SEFAS) displayed good evidence of construct validity and internal consistency. The measurement properties of the OMAS, LEFS and SEFAS were most studied. Conclusion The absence of validation studies covering all measurement properties of PROMs used in the adult ankle fracture population precludes the recommendation of a specific PROM to be used in the evaluation of this population. Further research should focus on validation of the content validity of the instruments used in patients with ankle fractures.publishedVersio

Improving osteoarthritis management in primary healthcare: results from a quasi-experimental study

Background To improve quality of care for patients with hip and knee osteoarthritis (OA), general practitioners (GPs) and physiotherapists (PTs) in a Norwegian municipality initiated an intervention. The intervention aimed to increase provision of core OA treatment (information, exercise, and weight control) prior to referral for surgery, rational use of imaging for assessing OA and improve communication between healthcare professionals. This study assessed the effectiveness of this intervention. Methods Forty-eight PTs and one hundred one GPs were invited to the intervention that included two interactive workshops outlining best practice and an accompanying template for PT discharge reports. Using interrupted time series research design, the study period was divided into three: pre-implementation, transition (implementation) and post-implementation. Comparing the change between pre- and post-implementation, the primary outcome was patient-reported quality of OA care measured with the OsteoArthritis Quality Indicator questionnaire. Secondary outcomes were number of PT discharge reports, information included in GP referral letters to orthopaedic surgeon, the proportion of GP referral letters indicating use of core treatment, and the use of imaging within OA assessment. Analyses involved linear mixed and logistic regression models. Results The PT workshop had 30 attendees, and 31 PTs and 33 GPs attended the multidisciplinary workshop. Two hundred eight and one hundred twenty-five patients completed the questionnaire during pre- and post-implementation, respectively. The adjusted model showed a small, statistically non-significant, increase in mean total score for quality of OA care (mean change = 4.96, 95% CI -0.18, 10.12, p:0.057), which was mainly related to items on OA core treatment. Patients had higher odds of reporting receipt of information on treatment alternatives (odds ratio (OR) 1.9, 95% CI 1.08, 3.24) and on self-management (OR 2.4, 95% CI 1.33, 4.32) in the post-implementation phase. There was a small, statistically non-significant, increase in the proportion of GP referral letters indicating prior use of core treatment modalities. There were negligible changes in the number of PT discharge reports, in the information included in the GP referral letters, and in the use of imaging for OA assessment. Conclusion This study suggests that a primary care intervention including two inter-active workshops can shift the quality of care towards best practice recommendations. Trial registration ClinicalTrials.gov: NCT02876120.publishedVersio

Pragmatiske studier – hva er det?

Konseptet pragmatiske studier ble foreslått av Schwartz og Lellouch i 1967 og problematiserte to viktige aspekter ved utprøving av ny behandling, nemlig forståelse og beslutning (1). Tradisjonelle forklarende effektstudier (explanatory trials) har som mål å øke vår forståelse ved å vise om en behandling virker per se, ofte under optimale forhold med nøye utvalgte studiedeltakere og utfallsmål. Pragmatiske studier skal derimot vise om en behandling virker i den kliniske hverdagen, helst på alle typer av aktuelle pasienter. Større bredde i pasientutvalget kan potensielt gjøre det vanskeligere å påvise klinisk relevante forskjeller, og studiene må av denne grunn ofte inkludere et stort antall pasienter. Den ideelle pragmatiske studien inkluderer en uselektert pasientgruppe som er aktuell for en type klinisk behandling, med endepunkter og oppfølging som i størst mulig grad foregår i den kliniske rutinen. Resultatene gir oss virkelighetsdata som kan understøtte beslutningen om å innføre en ny behandlingsform på generelt grunnlag. Pragmatiske studier er av interesse for beslutningstakere, fordi denne studieformen også tar sikte på å besvare spørsmål om kostnadseffekt av ny behandling som skal benyttes i alle deler av helsetjenesten

Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes

Background: The clinical and scientific usage of patient-reported outcome measures is increasing in the health services. Often paper forms are used. Manual double entry of data is defined as the definitive gold standard for transferring data to an electronic format, but the process is laborious. Automated forms processing may be an alternative, but further validation is warranted. Methods: 200 patients were randomly selected from a cohort of 5777 patients who had previously answered two different questionnaires. The questionnaires were scanned using an automated forms processing technique, as well as processed by single and double manual data entry, using the EpiData Entry data entry program. The main outcome measure was the proportion of correctly entered numbers at question, form and study level. Results: Manual double-key data entry (error proportion per 1000 fields = 0.046 (95 % CI: 0.001–0.258)) performed better than single-key data entry (error proportion per 1000 fields = 0.370 (95 % CI: 0.160–0.729), (p = 0.020)). There was no statistical difference between Optical Mark Recognition (error proportion per 1000 fields = 0.046 (95 % CI: 0.001–0.258)) and double-key data entry (p = 1.000). With the Intelligent Character Recognition method, there was no statistical difference compared to single-key data entry (error proportion per 1000 fields = 6.734 (95 % CI: 0.817–24.113), (p = 0.656)), as well as double-key data entry (error proportion per 1000 fields = 3.367 (95 % CI: 0.085–18.616)), (p = 0.319))

Data entry quality of double data entry vs automated form processing technologies: A cohort study validation of optical mark recognition and intelligent character recognition in a clinical setting

Background and Aims Patient-reported outcome measures (PROMs) are increasingly used in health services. Paper forms are still often used to register such data. Manual double data entry (DDE) has been defined as the gold standard for transferring data to an electronic format but is laborious and costly. Automated form processing (AFP) is an alternative, but validation in a clinical context is warranted. The study objective was to examine and validate a local hospital AFP setup. Methods Patients over 18 years of age who were scheduled for knee or hip replacement at Stavanger University Hospital from 2014 to 2017 who answered PROMs were included in the study and contributed PROM data. All paper PROMs were scanned using the AFP techniques of optical mark recognition (OMR) and intelligent character recognition (ICR) and were processed by DDE by health secretaries using a data entry program. OMR and ICR were used to capture different types of data. The main outcome was the proportion of correctly entered numbers, defined as the same response recorded in AFP and DDE or by consulting the original paper questionnaire at the data field, item, and PROM level. Results A total of 448 questionnaires from 255 patients were analyzed. There was no statistically significant difference in error proportions per 10 000 data fields between OMR and DDE for data from check boxes (3.52 95% confidence interval (CI) 2.17 to 5.72 and 4.18 (95% CI 2.68-6.53), respectively P = .61). The error proportion for ICR (nine errors) was statistically significantly higher than that for DDE (two errors), that is, 3.53 (95% CI 1.87-6.57) vs 0.78 (95% CI 0.22-2.81) per 100 data fields/items/questionnaires; P = .033. OMR (0.04% errors) outperformed ICR (3.51% errors; P < .001), Fisher's exact test. Conclusions OMR can produce an error rate that is comparable to that of DDE. In our setup, ICR is still problematic and is highly dependent on manual validation. When AFP is used, data quality should be tested and documented