Establishment of methods for extracting and analysing patient data from electronic practice management software systems used in first opinion veterinary practice in the UK

Examining patient records is a useful way to identify common conditions and treatment outcomes in veterinary practice and data gathered can be fed back to the profession to assist with clinical decision making. This research aimed to develop a method to extract clinical data from veterinary electronic patient records (EPRs) and to assess the value of the data extracted for use in practice-based research. The transfer of new research from continuing professional development (CPD) into practice was also considered. An extensible mark-up language (XML) schema was designed to extract information from a veterinary EPR. The analysis of free text was performed using a content analysis program and a clinical terms dictionary was created to mine the extracted data. Data collected by direct observation was compared to the extracted data. A review of research published in the proceedings of a popular veterinary CPD event, British Small Animal Veterinary Association (BSAVA) Congress, was appraised for evidence quality. All animal records were extracted and validation confirmed 100% accuracy. The content analysis produced results with a high specificity (100%) and the mined data analysis was successful in assessing the prevalence of a specific disease. On comparison, the data extracted from the EPR contained only 65% of all data recorded by direct observation. The review of BSAVA Congress abstracts found the majority of the clinical research abstracts (CRAs) presented to be case reports and case series, with differences in focus between CRAs and veterinary lecture stream abstracts. This study has demonstrated that data extraction using an XML schema is a viable method for the capture of patient data from veterinary EPRs. The next step will be to understand the differences found between data collected by observation and extraction, and to investigate how research presented as CPD is received, appraised and applied by the veterinary profession

Establishment of methods for extracting and analysing patient data from electronic practice management software systems used in first opinion veterinary practice in the UK

Examining patient records is a useful way to identify common conditions and treatment outcomes in veterinary practice and data gathered can be fed back to the profession to assist with clinical decision making. This research aimed to develop a method to extract clinical data from veterinary electronic patient records (EPRs) and to assess the value of the data extracted for use in practice-based research. The transfer of new research from continuing professional development (CPD) into practice was also considered. An extensible mark-up language (XML) schema was designed to extract information from a veterinary EPR. The analysis of free text was performed using a content analysis program and a clinical terms dictionary was created to mine the extracted data. Data collected by direct observation was compared to the extracted data. A review of research published in the proceedings of a popular veterinary CPD event, British Small Animal Veterinary Association (BSAVA) Congress, was appraised for evidence quality. All animal records were extracted and validation confirmed 100% accuracy. The content analysis produced results with a high specificity (100%) and the mined data analysis was successful in assessing the prevalence of a specific disease. On comparison, the data extracted from the EPR contained only 65% of all data recorded by direct observation. The review of BSAVA Congress abstracts found the majority of the clinical research abstracts (CRAs) presented to be case reports and case series, with differences in focus between CRAs and veterinary lecture stream abstracts. This study has demonstrated that data extraction using an XML schema is a viable method for the capture of patient data from veterinary EPRs. The next step will be to understand the differences found between data collected by observation and extraction, and to investigate how research presented as CPD is received, appraised and applied by the veterinary profession

Validation of text-mining and content analysis techniques using data collected from veterinary practice management software systems in the UK

Electronic patient records from practice management software systems have been used extensively in medicine for the investigation of clinical problems leading to the creation of decision support frameworks. To date, technologies that have been utilised for this purpose such as text mining and content analysis have not been employed significantly in veterinary medicine.The aim of this research was to pilot the use of content analysis and text-mining software for the synthesis and analysis of information extracted from veterinary electronic patient records. The purpose of the work was to be able to validate this approach for future employment across a number of practices for the purposes of practice based research. The approach utilised content analysis (Prosuite) and text mining (WordStat) software to aggregate the extracted text. Text mining tools such as Keyword in Context (KWIC) and Keyword Retrieval (KR) were employed to identify specific occurrences of data across the records. Two different datasets were interrogated, a bespoke test dataset that had been set up specifically for the purpose of the research, and a functioning veterinary clinic dataset that had been extracted from one veterinary practice.Across both datasets, the KWIC analysis was found to have a high level of accuracy with the search resulting in a sensitivity of between 85.3–100%, a specificity of between 99.1–99.7%, a positive predictive value between 93.5–95.8% and a negative predictive value between 97.7–100%. The KR search, based on machine learning, was utilised for the clinic-based dataset and was found to perform slightly better than the KWIC analysis.This study is the first to demonstrate the application of content analysis and text mining software for validation purposes across a number of different datasets for the purpose of search and recall of specific information across electronic patient records. This has not been demonstrated previously for small animal veterinary epidemiological research for the purposes of large scale analysis for practice-based research. Extension of this work to investigate more complex diseases across larger populations is required to fully explore the use of this approach in veterinary practice

The assessment and appraisal of regenerative medicines and cell therapy products : an exploration of methods for review, economic evaluation and appraisal

BACKGROUND: The National Institute for Health and Care Excellence (NICE) commissioned a 'mock technology appraisal' to assess whether changes to its methods and processes are needed. This report presents the findings of independent research commissioned to inform this appraisal and the deliberations of a panel convened by NICE to evaluate the mock appraisal. METHODS: Our research included reviews to identify issues, analysis methods and conceptual differences and the relevance of alternative decision frameworks, alongside the development of an exemplar case study of chimeric antigen receptor (CAR) T-cell therapy for treating acute lymphoblastic leukaemia. RESULTS: An assessment of previous evaluations of regenerative medicines found that, although there were a number of evidential challenges, none was unique to regenerative medicines or was beyond the scope of existing methods used to conceptualise decision uncertainty. Regarding the clinical evidence for regenerative medicines, the issues were those associated with a limited evidence base but were not unique to regenerative medicines: small non-randomised studies, high variation in response and the intervention subject to continuing development. The relative treatment effects generated from single-arm trials are likely to be optimistic unless it is certain that the historical data have accurately estimated the efficacy of the control agent. Pivotal trials may use surrogate end points, which, on average, overestimate treatment effects. To reduce overall uncertainty, multivariate meta-analysis of all available data should be considered. Incorporating indirectly relevant but more reliable (more mature) data into the analysis can also be considered; such data may become available as a result of the evolving regulatory pathways being developed by the European Medicines Agency. For the exemplar case of CAR T-cell therapy, target product profiles (TPPs) were developed, which considered the 'curative' and 'bridging to stem-cell transplantation' treatment approaches separately. Within each TPP, three 'hypothetical' evidence sets (minimum, intermediate and mature) were generated to simulate the impact of alternative levels of precision and maturity in the clinical evidence. Subsequent assessments of cost-effectiveness were undertaken, employing the existing NICE reference case alongside additional analyses suggested within alternative frameworks. The additional exploratory analyses were undertaken to demonstrate how assessments of cost-effectiveness and uncertainty could be impacted by alternative managed entry agreements (MEAs), including price discounts, performance-related schemes and technology leasing. The panel deliberated on the range of TPPs, evidence sets and MEAs, commenting on the likely recommendations for each scenario. The panel discussed the challenges associated with the exemplar and regenerative medicines more broadly, focusing on the need for a robust quantification of the level of uncertainty in the cost-effective estimates and the potential value of MEAs in limiting the exposure of the NHS to high upfront costs and loss associated with a wrong decision. CONCLUSIONS: It is to be expected that there will be a significant level of uncertainty in determining the clinical effectiveness of regenerative medicines and their long-term costs and benefits, but the existing methods available to estimate the implications of this uncertainty are sufficient. The use of risk sharing and MEAs between the NHS and manufacturers of regenerative medicines should be investigated further. FUNDING: The National Institute for Health Research Health Technology Assessment programme

Imaging tests for the detection of osteomyelitis : a systematic review

BACKGROUND: Osteomyelitis is an infection of the bone. Medical imaging tests, such as radiography, ultrasound, magnetic resonance imaging (MRI), single-photon emission computed tomography (SPECT) and positron emission tomography (PET), are often used to diagnose osteomyelitis. OBJECTIVES: To systematically review the evidence on the diagnostic accuracy, inter-rater reliability and implementation of imaging tests to diagnose osteomyelitis. DATA SOURCES: We conducted a systematic review of imaging tests to diagnose osteomyelitis. We searched MEDLINE and other databases from inception to July 2018. REVIEW METHODS: Risk of bias was assessed with QUADAS-2 [quality assessment of diagnostic accuracy studies (version 2)]. Diagnostic accuracy was assessed using bivariate regression models. Imaging tests were compared. Subgroup analyses were performed based on the location and nature of the suspected osteomyelitis. Studies of children, inter-rater reliability and implementation outcomes were synthesised narratively. RESULTS: Eighty-one studies were included (diagnostic accuracy: 77 studies; inter-rater reliability: 11 studies; implementation: one study; some studies were included in two reviews). One-quarter of diagnostic accuracy studies were rated as being at a high risk of bias. In adults, MRI had high diagnostic accuracy [95.6% sensitivity, 95% confidence interval (CI) 92.4% to 97.5%; 80.7% specificity, 95% CI 70.8% to 87.8%]. PET also had high accuracy (85.1% sensitivity, 95% CI 71.5% to 92.9%; 92.8% specificity, 95% CI 83.0% to 97.1%), as did SPECT (95.1% sensitivity, 95% CI 87.8% to 98.1%; 82.0% specificity, 95% CI 61.5% to 92.8%). There was similar diagnostic performance with MRI, PET and SPECT. Scintigraphy (83.6% sensitivity, 95% CI 71.8% to 91.1%; 70.6% specificity, 57.7% to 80.8%), computed tomography (69.7% sensitivity, 95% CI 40.1% to 88.7%; 90.2% specificity, 95% CI 57.6% to 98.4%) and radiography (70.4% sensitivity, 95% CI 61.6% to 77.8%; 81.5% specificity, 95% CI 69.6% to 89.5%) all had generally inferior diagnostic accuracy. Technetium-99m hexamethylpropyleneamine oxime white blood cell scintigraphy (87.3% sensitivity, 95% CI 75.1% to 94.0%; 94.7% specificity, 95% CI 84.9% to 98.3%) had higher diagnostic accuracy, similar to that of PET or MRI. There was no evidence that diagnostic accuracy varied by scan location or cause of osteomyelitis, although data on many scan locations were limited. Diagnostic accuracy in diabetic foot patients was similar to the overall results. Only three studies in children were identified; results were too limited to draw any conclusions. Eleven studies evaluated inter-rater reliability. MRI had acceptable inter-rater reliability. We found only one study on test implementation and no evidence on patient preferences or cost-effectiveness of imaging tests for osteomyelitis. LIMITATIONS: Most studies included < 50 participants and were poorly reported. There was limited evidence for children, ultrasonography and on clinical factors other than diagnostic accuracy. CONCLUSIONS: Osteomyelitis is reliably diagnosed by MRI, PET and SPECT. No clear reason to prefer one test over the other in terms of diagnostic accuracy was identified. The wider availability of MRI machines, and the fact that MRI does not expose patients to harmful ionising radiation, may mean that MRI is preferable in most cases. Diagnostic accuracy does not appear to vary with the potential cause of osteomyelitis or with the body part scanned. Considerable uncertainty remains over the diagnostic accuracy of imaging tests in children. Studies of diagnostic accuracy in children, particularly using MRI and ultrasound, are needed. STUDY REGISTRATION: This study is registered as PROSPERO CRD42017068511. FUNDING: This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 61. See the NIHR Journals Library website for further project information

A method for extracting electronic patient record data from practice management software systems used in veterinary practice

BackgroundData extracted from electronic patient records (EPRs) within practice management software systems are increasingly used in veterinary research. The use of real patient data gives the potential to generate research that can readily be applied to clinical practice. The use of veterinary EPRs for research in the United Kingdom is hindered by the number of different Practice Management System (PMS) providers used by practices, as obtaining and combining data from different systems electronically can be problematic. The use of extensible mark up language (XML) to extract clinical data for research would potentially resolve the compatibility issues between systems. The aim of this study was to establish and validate a method for the extraction of small animal patient records from a veterinary PMS that could potentially be used across multiple systems. An XML schema was designed to extract clinical information from EPRs. The schema was tested and validated in a test system, and was then tested in a real small animal practice where data was extracted for 16 weeks. A 10 % sample of the extracted records was then compared to paper copies provided by the practice.ResultsAll 21 fields encoded by the XML schema, from all of the records in the test system, were extracted with 100 % accuracy. Over the 18 week data collection period 4946 records, from 1279 patients, were extracted from the small animal practice. The 10 % printed records checked and compared with the XML extracted records demonstrated all required data was present. No unrequired, sensitive information e.g. costs or services/products or personal client information was extracted.ConclusionsThis is the first time a method for data extraction from EPRs in veterinary practice using an XML schema has been reported in the United Kingdom. This is an efficient and accurate way of extracting data which could be applied to all PMSs nationally and internationally

Accuracy of the electronic patient record in a first opinion veterinary practice

The use of electronic patient records (EPRs) in veterinary research is becoming more common place. To date no-one has investigated how accurately and completely they represent the clinical interactions that happen between veterinary professionals, and their clients and patients. The aim of this study was to compare data extracted from consultations within EPRs with data gathered by direct observation of the same consultation. A secondary aim was to establish the inter-rater reliability of two researchers who examined the data extracted from the EPRs. A convenience sample of 36 small animal consultations undertaken by 2 veterinary surgeons (83% by one veterinary surgeon) at a mixed veterinary practice in the United Kingdom was studied. All 36 consultations were observed by a single researcher using a standardised data collection tool. The information recorded in the EPRs was extracted from the Practice Management Software (PMS) systems using a validated XML schema. The XML extracted data was then converted into the same format as the observed data by two independent researchers who examined the extracted information and recorded their findings using the same tool as for the observation. The issues discussed and any action taken relating to those problems recorded in the observed and extracted datasets were then compared. In addition the inter-rater reliability of the two researchers who examined the extracted data was assessed. Only 64.4% of the observed problems discussed during the consultations were recorded in the EPR. The type of problem, who raised the problem and at what point in the consultation the problem was raised significantly affected whether the problem was recorded or not in the EPR. Only 58.3% of observed actions taken during the consultations were recorded in the EPR and the type of action significantly affected whether it would be recorded or not. There was moderate agreement between the two researchers who examined the extracted data. This is the first study that examines how much of the activity that occurs in small animal consultations is recorded in the EPR. Understanding the completeness, reliability and validity of EPRs is vital if they are to continue to be used for clinical research and the results to direct clinical care

Factors influencing common diagnoses made during first-opinion small-animal consultations in the United Kingdom

It is currently unclear how frequently a diagnosis is made during small-animal consultations or how much of a role making a diagnosis plays in veterinary decision-making. Understanding more about the diagnostic process will help direct future research towards areas relevant to practicing veterinary surgeons. The aim of this study was to determine the frequency with which a diagnosis was made, classify the types of diagnosis made (and the factors influencing these) and determine which specific diagnoses were made for health problems discussed during small-animal consultations. Data were gathered during real-time direct observation of small-animal consultations in eight practices in the United Kingdom. Data collected included characteristics of the consultation (e.g. consultation type), patient (e.g. breed), and each problem discussed (e.g. new or pre-existing problem). Each problem discussed was classified into one of the following diagnosis types: definitive; working; presumed; open; previous. A three-level multivariable logistic-regression model was developed, with problem (Level 1) nested within patient (Level 2) nested within consulting veterinary surgeon (Level 3). Problems without a previous diagnosis, in cats and dogs only, were included in the model, which had a binary outcome variable of definitive diagnosis versus no definitive diagnosis. Data were recorded for 1901 animals presented, and data on diagnosis were gathered for 3192 health problems. Previous diagnoses were the most common diagnosis type (n = 1116/3192; 35.0%), followed by open (n = 868/3192; 27.2%) then definitive (n = 660/3192; 20.7%). The variables remaining in the final model were patient age, problem history, consultation type, who raised the problem, and body system affected. New problems, problems in younger animals, and problems raised by the veterinary surgeon were more likely to result in a definitive diagnosis than pre-existing problems, problems in older animals, and problems raised by the owner. The most common diagnoses made were overweight/obese and periodontal disease (both n = 210; 6.6%). Definitive diagnoses are rarely made during small-animal consultations, with much of the veterinary caseload involving management of ongoing problems or making decisions around new problems prior to a diagnosis being made. This needs to be taken into account when considering future research priorities, and it may be necessary to conduct research focused on the approach to common clinical presentations, rather than purely on the common diagnoses made. Examining how making a diagnosis affects the actions taken during the consultation may shed further light on the role of diagnosis in the clinical decision-making process

The efficacy of systemic glucocorticosteroids for pain in rheumatoid arthritis: a systematic literature review and meta-analysis

Background: Glucocorticosteroids (GCs) are recommended to suppress inflammation in people with active RA. This systematic review and meta-analysis aimed to quantify the effects of systemic GCs on RA pain.Methods: A systematic literature review of randomised controlled trials (RCTs) in RA comparing systemic GCs to inactive treatment. Three databases were and spontaneous pain and evoked pain outcomes were extracted. Standardized mean differences (SMDs) and mean differences (MDs) were meta-analysed. Heterogeneity (I², tau statistics) and bias (funnel plot, Eggers test) were assessed. Subgroup analyses investigated sources of variation. This study was pre-registered (PROSPERO CRD42019111562). Results: 18903 titles, 880 abstracts and 226 full texts were assessed. Thirty three RCTs suitable for the meta-analysis included 2658 participants. Pain scores (spontaneous pain) decreased in participants treated with oral GCs; SMD= -0.65 (15 studies, 95% CI, -0.82, -0.49, p3 to 6 months, and -6mm (95% CI, -10mm, -2mm) at >6 months. Similar findings were obtained when evoked pain outcomes were examined. Data from 5 RCTs suggested improvement also in fatigue during GC treatment.Discussion. Oral GCs are analgesic in RA. The benefit is greatest shortly after initiation and GCs might not achieve clinically important pain relief beyond 3 months. Treatments other than anti-inflammatory GCs should be considered to reduce the long-term burden of pain in RA