Identifying and appraising promising sources of UK clinical, health and social care data for use by NICE

This report aimed to aid the National Institute of Health and Care Excellence (NICE) in identifying opportunities for greater use of real-world data within its work. NICE identified five key ways in which real-world data was currently informing its work, or could do so in the future through: (i) researching the effectiveness of interventions or practice in real-world (UK) settings (ii) auditing the implementation of guidance (iii) providing information on resource use and evaluating the potential impact of guidance (iv) providing epidemiological information (v) providing information on current practice to inform the development of NICE quality standards. This report took a broad definition of ‘real-world’ data and created a map of UK sources, informed by a number of experts in real-world data, as well as a literature search, to highlight where some of the opportunities may lie for NICE within its clinical, public health and social care remit. The report was commissioned by the NICE, although the findings are likely to be of wider interest to a range of stakeholders interested in the role of real-world data in informing clinical, social care and public health decision-making. Most of the issues raised surrounding the use and appraisal of real-world data are likely to be generic, although the choice of datasets that were profiled in-depth reflected the interests of NICE. We discovered 275 sources that were named as real-world data sources for clinical, social care or public health investigation, 233 of which were deemed as active. The real-world data landscape therefore is highly complex and heterogeneous and composed of sources with different purposes, structures and collection methods. Some real-world data sources are purposefully either set-up or re-developed to enhance their data linkages and to examine the presence/absence/effectiveness of integrated patient care; however, such sources are in the minority. Furthermore, the small number of real-world data sources that are designed to enable the monitoring of care across providers, or at least have the capability to do so at a national level, have been utilised infrequently for this purpose in the literature. Data that offer the capacity to monitor transitions between health and social care do not currently exist at a national level, despite the increasing recognition of the interdependency between these sectors. Among the data sources we included, it was clear that no one data source represented a panacea for NICE’s real world data needs. This does highlight the merits and importance of data linkage projects and is suggestive of a need to triangulate evidence across different data, particularly in order to understand the feasibility and impact of guidance. There exists no overall catalogue or repository of real-world data sources for health, public health and social care, and previous initiatives aimed at creating such a resource have not been maintained. As much as there is a need for enhanced usage of the data, there is also a need for taking stock, integration, standardisation, and quality assurance of different sources. This research highlights a need for a systematic approach to creating an inventory of sources with detailed metadata and the funding to maintain this resource. This would represent an essential first step to support future initiatives aimed at enhancing the use of real-world data

The application of process mining to care pathway analysis in the NHS

Background: Prostate cancer is the most common cancer in men in the UK and the sixth-fastest increasing cancer in males. Within England survival rates are improving, however, these are comparatively poorer than other countries. Currently, information available on outcomes of care is scant and there is an urgent need for techniques to improve healthcare systems and processes. Aims: To provide prostate cancer pathway analysis, by applying concepts of process mining and visualisation and comparing the performance metrics against the standard pathway laid out by national guidelines. Methods: A systematic review was conducted to see how process mining has been used in healthcare. Appropriate datasets for prostate cancer were identified within Imperial College Healthcare NHS Trust London. A process model was constructed by linking and transforming cohort data from six distinct database sources. The cohort dataset was filtered to include patients who had a PSA from 2010-2015, and validated by comparing the medical patient records against a Case-note audit. Process mining techniques were applied to the data to analyse performance and conformance of the prostate cancer pathway metrics to national guideline metrics. These techniques were evaluated with stakeholders to ascertain its impact on user experience. Results: Case note audit revealed 90% match against patients found in medical records. Application of process mining techniques showed massive heterogeneity as compared to the homogenous path laid out by national guidelines. This also gave insight into bottlenecks and deviations in the pathway. Evaluation with stakeholders showed that the visualisation and technology was well accepted, high quality and recommended to be used in healthcare decision making. Conclusion: Process mining is a promising technique used to give insight into complex and flexible healthcare processes. It can map the patient journey at a local level and audit it against explicit standards of good clinical practice, which will enable us to intervene at the individual and system level to improve care.Open Acces

An electronic application for rapidly calculating Charlson comorbidity score

BACKGROUND: Uncertainty regarding comorbid illness, and ability to tolerate aggressive therapy has led to minimal enrollment of elderly cancer patients into clinical trials and often substandard treatment. Increasingly, comorbid illness scales have proven useful in identifying subgroups of elderly patients who are more likely to tolerate and benefit from aggressive therapy. Unfortunately, the use of such scales has yet to be widely integrated into either clinical practice or clinical trials research. METHODS: This article reviews evidence for the validity of the Charlson Comorbidity Index (CCI) in oncology and provides a Microsoft Excel (MS Excel) Macro for the rapid and accurate calculation of CCI score. The interaction of comorbidity and malignant disease and the validation of the Charlson Index in oncology are discussed. RESULTS: The CCI score is based on one year mortality data from internal medicine patients admitted to an inpatient setting and is the most widely used comorbidity index in oncology. An MS Excel Macro file was constructed for calculating the CCI score using Microsoft Visual Basic. The Macro is provided for download and dissemination. The CCI has been widely used and validated throughout the oncology literature and has demonstrated utility for most major cancers. The MS Excel CCI Macro provides a rapid method for calculating CCI score with or without age adjustments. The calculator removes difficulty in score calculation as a limitation for integration of the CCI into clinical research. The simple nature of the MS Excel CCI Macro and the CCI itself makes it ideal for integration into emerging electronic medical records systems. CONCLUSIONS: The increasing elderly population and concurrent increase in oncologic disease has made understanding the interaction between age and comorbid illness on life expectancy increasingly important. The MS Excel CCI Macro provides a means of increasing the use of the CCI scale in clinical research with the ultimate goal of improving determination of optimal treatments for elderly cancer patients

Record Linkage Techniques: Exploring and developing data matching methods to create national record linkage infrastructure to support population level research

In a world where the growth in digital information and systems continues to expand, researchers have access to unprecedented amounts of data. These large and complex data reservoirs require creative, innovative and scalable tools to unlock the potential of this ‘big data’. Record linkage is a powerful tool in the ‘big data’ arsenal. This thesis demonstrates the value of national record linkage infrastructure and how this has been achieved for the Australian research community

Reengineering the clinical research enterprise to involve more community clinicians

<p>Abstract</p> <p>Background</p> <p>The National Institutes of Health has called for expansion of practice-based research to improve the clinical research enterprise.</p> <p>Methods</p> <p>This paper presents a model for the reorganization of clinical research to foster long-term participation by community clinicians.</p> <p>Based on the literature and interviews with clinicians and other stakeholders, we posited a model, conducted further interviews to test the viability of the model, and further adapted it.</p> <p>Results</p> <p>We propose a three-dimensional system of checks and balances to support community clinicians using research support organizations, community outreach, a web-based registry of clinicians and studies, web-based training services, quality audits, and a feedback mechanism for clinicians engaged in research.</p> <p>Conclusions</p> <p>The proposed model is designed to offer a systemic mechanism to address current barriers that prevent clinicians from participation in research. Transparent mechanisms to guarantee the safety of patients and the integrity of the research enterprise paired with efficiencies and economies of scale are maintained by centralizing some of the functions. Assigning other responsibilities to more local levels assures flexibility with respect to the size of the clinician networks and the changing needs of researchers.</p

Temporal variability analysis reveals biases in electronic health records due to hospital process reengineering interventions over seven years

[EN] Objective To evaluate the effects of Process-Reengineering interventions on the Electronic Health Records (EHR) of a hospital over 7 years. Materials and methods Temporal Variability Assessment (TVA) based on probabilistic data quality assessment was applied to the historic monthly-batched admission data of Hospital La Fe Valencia, Spain from 2010 to 2016. Routine healthcare data with a complete EHR was expanded by processed variables such as the Charlson Comorbidity Index. Results Four Process-Reengineering interventions were detected by quantifiable effects on the EHR: (1) the hospital relocation in 2011 involved progressive reduction of admissions during the next four months, (2) the hospital services re-configuration incremented the number of inter-services transfers, (3) the care-services re-distribution led to transfers between facilities (4) the assignment to the hospital of a new area with 80,000 patients in 2015 inspired the discharge to home for follow up and the update of the pre-surgery planned admissions protocol that produced a significant decrease of the patient length of stay. Discussion TVA provides an indicator of the effect of process re-engineering interventions on healthcare practice. Evaluating the effect of facilities¿ relocation and increment of citizens (findings 1, 3¿4), the impact of strategies (findings 2¿3), and gradual changes in protocols (finding 4) may help on the hospital management by optimizing interventions based on their effect on EHRs or on data reuse. Conclusions The effects on hospitals EHR due to process re-engineering interventions can be evaluated using the TVA methodology. Being aware of conditioned variations in EHR is of the utmost importance for the reliable reuse of routine hospitalization data.F.J.P.B, C.S., J.M.G.G. and J.A.C. were funded Universitat Politecnica de Valencia, project "ANALISIS DE LA CALIDAD Y VARIABILIDAD DE DATOS MEDICOS". www.upv.es. J.M.G.G.is also partially supported by: Ministerio de Economia y Competitividad of Spain through MTS4up project (National Plan for Scientific and Technical Research and Innovation 2013-2016, No. DPI2016-80054-R); and European Commission projects H2020-SC1-2016-CNECT Project (No. 727560) and H2020-SC1-BHC-2018-2020 (No. 825750). The funders did not play any role in the study design, data collection and analysis, decision to publish, nor preparation of the manuscript.Perez-Benito, FJ.; Sáez Silvestre, C.; Conejero, JA.; Tortajada, S.; Valdivieso, B.; Garcia-Gomez, JM. (2019). Temporal variability analysis reveals biases in electronic health records due to hospital process reengineering interventions over seven years. PLoS ONE. 14(8):1-19. https://doi.org/10.1371/journal.pone.0220369S119148Aguilar-Savén, R. S. (2004). Business process modelling: Review and framework. International Journal of Production Economics, 90(2), 129-149. doi:10.1016/s0925-5273(03)00102-6Poulymenopoulou, M. (2003). Journal of Medical Systems, 27(4), 325-335. doi:10.1023/a:1023701219563Dadam P, Reichert M, Kuhn K. Clinical Workflows -The Killer Application for Process-oriented Information Systems? Proceedings of the 4th International Conference on Business Information Systems. London: Springer London; 2000. pp. 36–59. doi: https://doi.org/10.1007/978-1-4471-0761-3Lenz, R., & Reichert, M. (2007). IT support for healthcare processes – premises, challenges, perspectives. Data & Knowledge Engineering, 61(1), 39-58. doi:10.1016/j.datak.2006.04.007Rebuge, Á., & Ferreira, D. R. (2012). Business process analysis in healthcare environments: A methodology based on process mining. Information Systems, 37(2), 99-116. doi:10.1016/j.is.2011.01.003Amour EAEH, Ghannouchi SA. Applying Data Mining Techniques to Discover KPIs Relationships in Business Process Context. 2017 18th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT). IEEE; 2017. pp. 230–237. doi: https://doi.org/10.1109/PDCAT.2017.00045Chou, Y.-C., Chen, B.-Y., Tang, Y.-Y., Qiu, Z.-J., Wu, M.-F., Wang, S.-C., … Chuang, W.-C. (2010). Prescription-Filling Process Reengineering of an Outpatient Pharmacy. Journal of Medical Systems, 36(2), 893-902. doi:10.1007/s10916-010-9553-5Leu, J.-D., & Huang, Y.-T. (2009). An Application of Business Process Method to the Clinical Efficiency of Hospital. Journal of Medical Systems, 35(3), 409-421. doi:10.1007/s10916-009-9376-4Gand K. Investigating on Requirements for Business Model Representations: The Case of Information Technology in Healthcare. 2017 IEEE 19th Conference on Business Informatics (CBI). IEEE; 2017. pp. 471–480. doi: https://doi.org/10.1109/CBI.2017.36Ferreira, G. S. A., Silva, U. R., Costa, A. L., & Pádua, S. I. D. de D. (2018). The promotion of BPM and lean in the health sector: main results. Business Process Management Journal, 24(2), 400-424. doi:10.1108/bpmj-06-2016-0115Abdulrahman Jabour RM. Cancer Reporting: Timeliness Analysis and Process. 2016; Available: https://search.proquest.com/openview/4ecf737c5ef6d2d503e948df8031fe54/1?pq-origsite=gscholar&amp;cbl=18750&amp;diss=yHewitt M, Simone J V. Enhancing Data Systems to Improve the Quality of Cancer Care [Internet]. National Academy Press; 2000. Available: http://www.nap.edu/catalog/9970.htmlWeiskopf, N. G., & Weng, C. (2013). Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association, 20(1), 144-151. doi:10.1136/amiajnl-2011-000681Saez C, Robles M, Garcia-Gomez JM. Comparative study of probability distribution distances to define a metric for the stability of multi-source biomedical research data. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. IEEE; 2013. pp. 3226–3229. doi: https://doi.org/10.1109/EMBC.2013.6610228Sáez, C., Rodrigues, P. P., Gama, J., Robles, M., & García-Gómez, J. M. (2014). Probabilistic change detection and visualization methods for the assessment of temporal stability in biomedical data quality. Data Mining and Knowledge Discovery, 29(4), 950-975. doi:10.1007/s10618-014-0378-6Sáez, C., Zurriaga, O., Pérez-Panadés, J., Melchor, I., Robles, M., & García-Gómez, J. M. (2016). Applying probabilistic temporal and multisite data quality control methods to a public health mortality registry in Spain: a systematic approach to quality control of repositories. Journal of the American Medical Informatics Association, 23(6), 1085-1095. doi:10.1093/jamia/ocw010International Ethical Guidelines for Epidemiological Studies [Internet]. Geneva: Council for International Organizations of Medical Sciences (CIOMS) in collaboration with the World Health Organization; 2009. Available: https://cioms.ch/wp-content/uploads/2017/01/International_Ethical_Guidelines_LR.pdfResearch Ethics Committee of the Universitari i Politècnic La Fe Hospital [Internet]. Available: https://www.iislafe.es/en/research/ethics-committees/Charlson, M. E., Pompei, P., Ales, K. L., & MacKenzie, C. R. (1987). A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. Journal of Chronic Diseases, 40(5), 373-383. doi:10.1016/0021-9681(87)90171-8Schneeweiss, S., Wang, P. S., Avorn, J., & Glynn, R. J. (2003). Improved Comorbidity Adjustment for Predicting Mortality in Medicare Populations. Health Services Research, 38(4), 1103-1120. doi:10.1111/1475-6773.00165Quan, H., Sundararajan, V., Halfon, P., Fong, A., Burnand, B., Luthi, J.-C., … Ghali, W. A. (2005). Coding Algorithms for Defining Comorbidities in ICD-9-CM and ICD-10 Administrative Data. Medical Care, 43(11), 1130-1139. doi:10.1097/01.mlr.0000182534.19832.83Sáez Silvestre C. Probabilistic methods for multi-source and temporal biomedical data quality assessment [Internet]. Thesis. Universitat Politècnica de València. 2016. doi: https://doi.org/10.4995/Thesis/10251/62188Amari S, Nagaoka H. Methods of Information Geometry [Internet]. Amer. Math. Soc. and Oxford Univ. Press. American Mathematical Society; 2000. Available: https://books.google.es/books?hl=es&amp;lr=&amp;id=vc2FWSo7wLUC&amp;oi=fnd&amp;pg=PR7&amp;dq=Methods+of+Information+geometry&amp;ots=4HmyCCY4PX&amp;sig=2-dpCuwMQvEC1iREjxdfIX0yEls#v=onepage&amp;q=MethodsofInformationgeometry&amp;f=falseCsiszár, I., & Shields, P. C. (2004). Information Theory and Statistics: A Tutorial. Foundations and Trends™ in Communications and Information Theory, 1(4), 417-528. doi:10.1561/0100000004Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1), 145-151. doi:10.1109/18.61115M.Cover T. Elements Of Information Theory Notes [Internet]. 2006. Available: http://books.google.fr/books?id=VWq5GG6ycxMC&printsec=frontcover&dq=intitle:Elements+of+Information+Theory&hl=&cd=1&source=gbs_api%5Cnpapers2://publication/uuid/BAF426F8-5A4F-44A4-8333-FA8187160D9BBrandes, U., & Pich, C. (s. f.). Eigensolver Methods for Progressive Multidimensional Scaling of Large Data. Lecture Notes in Computer Science, 42-53. doi:10.1007/978-3-540-70904-6_6Liaw, S. T., Rahimi, A., Ray, P., Taggart, J., Dennis, S., de Lusignan, S., … Talaei-Khoei, A. (2013). Towards an ontology for data quality in integrated chronic disease management: A realist review of the literature. International Journal of Medical Informatics, 82(1), 10-24. doi:10.1016/j.ijmedinf.2012.10.001Arts, D. G. T. (2002). Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework. Journal of the American Medical Informatics Association, 9(6), 600-611. doi:10.1197/jamia.m1087Bray, F., & Parkin, D. M. (2009). Evaluation of data quality in the cancer registry: Principles and methods. Part I: Comparability, validity and timeliness. European Journal of Cancer, 45(5), 747-755. doi:10.1016/j.ejca.2008.11.032Parkin, D. M., & Bray, F. (2009). Evaluation of data quality in the cancer registry: Principles and methods Part II. Completeness. European Journal of Cancer, 45(5), 756-764. doi:10.1016/j.ejca.2008.11.033Fernandez-Llatas, C., Ibanez-Sanchez, G., Celda, A., Mandingorra, J., Aparici-Tortajada, L., Martinez-Millana, A., … Traver, V. (2019). Analyzing Medical Emergency Processes with Process Mining: The Stroke Case. Lecture Notes in Business Information Processing, 214-225. doi:10.1007/978-3-030-11641-5_17Fernandez-Llatas, C., Lizondo, A., Monton, E., Benedi, J.-M., & Traver, V. (2015). Process Mining Methodology for Health Process Tracking Using Real-Time Indoor Location Systems. Sensors, 15(12), 29821-29840. doi:10.3390/s151229769Van der Aalst, W., Weijters, T., & Maruster, L. (2004). Workflow mining: discovering process models from event logs. IEEE Transactions on Knowledge and Data Engineering, 16(9), 1128-1142. doi:10.1109/tkde.2004.47Weijters AJMM, Van Der Aalst WMP, Alves De Medeiros AK. Process Mining with the HeuristicsMiner Algorithm [Internet]. Available: https://pdfs.semanticscholar.org/1cc3/d62e27365b8d7ed6ce93b41c193d0559d086.pdfShim, S. J., & Kumar, A. (2010). Simulation for emergency care process reengineering in hospitals. Business Process Management Journal, 16(5), 795-805. doi:10.1108/14637151011076476Svolba, G., & Bauer, P. (1999). Statistical Quality Control in Clinical Trials. Controlled Clinical Trials, 20(6), 519-530. doi:10.1016/s0197-2456(99)00029-xKahn, M. G., Raebel, M. A., Glanz, J. M., Riedlinger, K., & Steiner, J. F. (2012). A Pragmatic Framework for Single-site and Multisite Data Quality Assessment in Electronic Health Record-based Clinical Research. Medical Care, 50, S21-S29. doi:10.1097/mlr.0b013e318257dd67Batini, C., Cappiello, C., Francalanci, C., & Maurino, A. (2009). Methodologies for data quality assessment and improvement. ACM Computing Surveys, 41(3), 1-52. doi:10.1145/1541880.1541883Heinrich, B., Klier, M., & Kaiser, M. (2009). A Procedure to Develop Metrics for Currency and its Application in CRM. Journal of Data and Information Quality, 1(1), 1-28. doi:10.1145/1515693.1515697Sirgo, G., Esteban, F., Gómez, J., Moreno, G., Rodríguez, A., Blanch, L., … Bodí, M. (2018). Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: The importance of data quality assessment. International Journal of Medical Informatics, 112, 166-172. doi:10.1016/j.ijmedinf.2018.02.007Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the Dimensionality of Data with Neural Networks. Science, 313(5786), 504-507. doi:10.1126/science.1127647Kohn LT, Corrigan JM. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America. 2000. p. 287. National Academies Press