Did changing primary care delivery models change performance? A population based study using health administrative data

<p>Abstract</p> <p>Background</p> <p>Primary care reform in Ontario, Canada started with the introduction of new enrollment models, the two largest of which are Family Health Networks (FHNs), a capitation-based model, and Family Health Groups (FHGs), a blended fee-for-service model. The purpose of this study was to evaluate differences in performance between FHNs and FHGs and to compare performance before and after physicians joined these new primary care groups.</p> <p>Methods</p> <p>This study used Ontario administrative claims data to compare performance measures in FHGs and FHNs. The study population included physicians who belonged to a FHN or FHG for at least two years. Patients were included in the analyses if they enrolled with a physician in the two years after the physician joined a FHN or FHG, and also if they saw the physician in a two year period prior to the physician joining a FHN or FHG. Performance was derived from the administrative data, and included measures of preventive screening for cancer (breast, cervical, colorectal) and chronic disease management (diabetes, heart failure, asthma).</p> <p>Results</p> <p>Performance measures did not vary consistently between models. In some cases, performance approached current benchmarks (Pap smears, mammograms). In other cases it was improving in relation to previous measures (colorectal cancer screening). There were no changes in screening for cervical cancer or breast cancer after joining either a FHN or FHG. Colorectal cancer screening increased in both FHNs and FHGs. After enrolling in either a FHG or a FHN, prescribing performance measures for diabetes care improved. However, annual eye examinations decreased for younger people with diabetes after joining a FHG or FHN. There were no changes in performance measures for heart failure management or asthma care after enrolling in either a FHG or FHN.</p> <p>Conclusions</p> <p>Some improvements in preventive screening and diabetes management which were seen amongst people after they enrolled may be attributed to incentive payments offered to physicians within FHGs and FHNs. However, these primary care delivery models need to be compared with other delivery models and fee for service practices in order to describe more specifically what aspects of model delivery and incentives affect care.</p

Using Biomedical Text as Data and Representation Learning for Identifying Patients with an Osteoarthritis Phenotype in the Electronic Medical Record

Introduction Electronic medical records (EMRs) are increasingly used in health services research. Accurate/efficient identification of a target population with a specific disease phenotype is a necessary precursor to studying the health of these individuals. Objectives and Approach We explored the use of biomedical text as inputs to supervised phenotype identification algorithms. We employed a two-stage classification approach to map the discrete, sparse high-dimensional biomedical text data to a dense low dimensional vector space using methods from unsupervised machine learning. Next we used these learned vectors as inputs to supervised machine learning algorithms for phenotype identification. We were able to demonstrate the applicability of the approach to identifying patients with an osteoarthritis (OA) phenotype using primary care data from the Electronic Medical Record Administrative data Linked Database (EMRALD) held at ICES. Results EMRALD contains approximately 20Gb of biomedical text data on approximately 500,000 patients. The unit of analysis for this study is the patient. We were interested in identifying OA patients using solely text data as features. Labelled outcome information wass available from a random sample of 7,500 patients. We divided patients into training (N=6000), validation (N=750) and test (N=750) cohorts. We learned low dimensional representations of the input text data on the entire EMRALD corpus (N=500,000). We used learned numeric vectors as inputs to supervised machine learning models for OA classification (N=6,000 training set patients). We compared models in terms of accuracy, sensitivity, specificity, PPV and NPV. The best learned models achieved approximately 90\% sensitivity and 80\% specificity. Classification accuracy varied as a function of learned inputs. Conclusion/Implications We developed an approach to phenotype identification using solely biomedical text as an input. Preliminary results suggest our two-stage ML approach has improved operating characteristics compared to existing clinically derived decision rules for OA classification. Future work will explore the generalizability of this methodology to other disease phenotypes

Learning Unsupervised Representations from Biomedical Text

Introduction Healthcare settings are becoming increasingly technological. Interactions/events involving healthcare providers and the patients they service are captured as digital text. Healthcare organizations are amassing increasingly large/complex collections of biomedical text data. Researchers and policy makers are beginning to explore these text data holdings for structure, patterns, and meaning. Objectives and Approach EMRALD is a primary care electronic medical record (EMR) database, comprised of over 40 family medicine clinics, nearly 400 primary care physicians and over 500,000 patients. EMRALD includes full-chart extractions, including all clinical narrative information/data in a variety of fields. The input data (raw text strings) are discrete, sparse and high dimensional. We assessed scalable statistical models for high dimensional discrete data, including fitting, assessing and exploring models from three broad statistical areas: i) matrix factorization/decomposition models ii) probabilistic topic models and iii) word-vector embedding models. Results EMRALD is comprised of 12 text data streams. EMRALD text data is structured into 84 million clinical notes (3.5 billion word/language tokens) and is approximately 18Gb in storage size. We employ a “text as data” pipeline, i) mapping raw strings to sequences of word/language tokens, ii) mapping token sequences to numeric arrays, and finally iii) using numeric arrays as inputs to statistical models. Fitted topic models yield useful thematic summaries of the EMRALD corpora. Topics discovered reflect core responsibilities of primary care physicians (e.g. women’s health, pain management, nutrition/diet, etc.). Fitted vector embedding models capture structure of discourse/syntax. Related words are mapped to similar locations of vector spaces. Analogical reasoning is possible in the embedding space. Conclusion/Implications “Text as data” requires an understanding of statistical models for discrete, sparse, high dimensional data. We fit a variety of unsupervised statistical models to biomedical text data. Preliminary results suggest that the learned low dimensional representations of the biomedical text data are effective at uncovering meaningful patterns/structure

An administrative data validation study of the accuracy of algorithms for identifying rheumatoid arthritis: the influence of the reference standard on algorithm performance

BACKGROUND: We have previously validated administrative data algorithms to identify patients with rheumatoid arthritis (RA) using rheumatology clinic records as the reference standard. Here we reassessed the accuracy of the algorithms using primary care records as the reference standard. METHODS: We performed a retrospective chart abstraction study using a random sample of 7500 adult patients under the care of 83 family physicians contributing to the Electronic Medical Record Administrative data Linked Database (EMRALD) in Ontario, Canada. Using physician-reported diagnoses as the reference standard, we computed and compared the sensitivity, specificity, and predictive values for over 100 administrative data algorithms for RA case ascertainment. RESULTS: We identified 69 patients with RA for a lifetime RA prevalence of 0.9%. All algorithms had excellent specificity (>97%). However, sensitivity varied (75-90%) among physician billing algorithms. Despite the low prevalence of RA, most algorithms had adequate positive predictive value (PPV; 51-83%). The algorithm of “[1 hospitalization RA diagnosis code] or [3 physician RA diagnosis codes with ≥1 by a specialist over 2 years]” had a sensitivity of 78% (95% CI 69–88), specificity of 100% (95% CI 100–100), PPV of 78% (95% CI 69–88) and NPV of 100% (95% CI 100–100). CONCLUSIONS: Administrative data algorithms for detecting RA patients achieved a high degree of accuracy amongst the general population. However, results varied slightly from our previous report, which can be attributed to differences in the reference standards with respect to disease prevalence, spectrum of disease, and type of comparator group

How do we enhance linked administrative data based chronic disease surveillance in Canada? Results of an environmental scan.

Introduction The Canadian Chronic Disease Surveillance System (CCDSS) is a collaboration of provincial and territorial surveillance systems which generates estimates of chronic diseases using linked population-level administrative health databases and standard case definitions. We conducted an environmental scan of administrative data validation studies and identified opportunities for CCDSS case definition enhancement. Objectives and Approach The purpose of this project is to develop a methodology for and conduct an environmental scan, identifying opportunities for enhancing the CCDSS. This multifaceted approach consists of the following elements: 1) key informant interviews and stakeholder consultations to identify new and existing priority conditions for updating/validating within the CCDSS, and new areas of conceptual and methodological relevance for administrative data disease surveillance, 2) a systematic literature review of PubMed, Ovid and Embase from 2013-2017 using MeSH terms and a librarian peer-reviewed search strategy, and 3) a review of the grey literature. Results Key stakeholders identified the following priorities for validation work and/or case definition enhancement: diabetes, mood and anxiety disorders, schizophrenia, obesity, hypertension, chronic obstructive pulmonary disease, osteoarthritis, stroke, early-onset dementia, rheumatoid arthritis and gout. Scientific and grey literature reviews of validation work for these conditions examined the following concepts/methods: 1) evaluating validity of disease-specific case definitions over time, and in different ages, sub-populations and settings, 2) defining incidence versus prevalence using linked administrative data, 3) determining opportunities and constraints of using linked administrative data to conduct surveillance on diseases that are chronic versus episodic in nature and defining active versus lifetime prevalence, and 4) assessing the feasibility of using new sources of data for linkage to enhance case definition validity. Conclusion/Implications Utilization of linked administrative databases for chronic disease surveillance has expanded across many jurisdictions since the inception of the CCDSS. As disease estimates generated in this manner are increasingly being relied upon by policy makers working to enhance public health, the methodological opportunities and constraints identified here require consideration

Using a data entry clerk to improve data quality in primary care electronic medical records: a pilot study

Background The quality of electronic medical record (EMR) data is known to be problematic; research on improving these data is needed. Objective The primary objective was to explore the impact of using a data entry clerk to improve data quality in primary care EMRs. The secondary objective was to evaluate the feasibility of implementing this intervention. Methods We used a before and after design for this pilot study. The participants were 13 community based family physicians and four allied health professionals in Toronto, Canada. Using queries programmed by a data manager, a data clerk was tasked with re-entering EMR information as coded or structured data for chronic obstructive pulmonary disease (COPD), smoking, specialist designations and interprofessional encounter headers. We measured data quality before and three to six months after the intervention. We evaluated feasibility by measuring acceptability to clinicians and workload for the clerk. Results After the intervention, coded COPD entries increased by 38% (P = 0.0001, 95% CI 23 to 51%); identifiable data on smoking categories increased by 27% (P = 0.0001, 95% CI 26 to 29%); referrals with specialist designations increased by 20% (P = 0.0001, 95% CI 16 to 22%); and identifiable interprofessional headers increased by 10% (P = 0.45, 95 CI _3 to 23%). Overall, the interventionwas rated as being at least moderately useful and moderately usable. The data entry clerk spent 127 hours restructuring data for 11 729 patients. Conclusions Utilising a data manager for queries and a data clerk to re-enter data led to improvements in EMR data quality. Clinicians found this approach to be acceptable

Tracking family medicine graduates. Where do they go, what services do they provide and whom do they see?

<p>Abstract</p> <p>Background</p> <p>There are continued concerns over an adequate supply of family physicians (FPs) practicing in Canada. While most resource planning has focused on intake into postgraduate education, less information is available on what postgraduate medical training yields. We therefore undertook a study of Family Medicine (FM) graduates from the University of Toronto (U of T) to determine the type of information for physician resource planning that may come from tracking FM graduates using health administrative data. This study compared three cohorts of FM graduates over a 10 year period of time and it also compared FM graduates to all Ontario practicing FPs in 2005/06. The objectives for tracking the three cohorts of FM graduates were to: 1) describe where FM graduates practice in the province 2) examine the impact of a policy introduced to influence the distribution of new FM graduates in the province 3) describe the services provided by FM graduates and 4) compare workload measures. The objectives for the comparison of FM graduates to all practicing FPs in 2005/06 were to: 1) describe the patient population served by FM graduates, 2) compare workload of FM graduates to all practicing FPs.</p> <p>Methods</p> <p>The study cohort consisted of all U of T FM postgraduate trainees who started and completed their training between 1993 and 2003. This study was a descriptive record linkage study whereby postgraduate information for FM graduates was linked to provincial health administrative data. Comprehensiveness of care indicators and workload measures based on administrative data where determined for the study cohort.</p> <p>Results</p> <p>From 1993 to 2003 there were 857 University of Toronto FM graduates. While the majority of U of T FM graduates practice in Toronto or the surrounding Greater Toronto Area, there are FM graduates from U of T practicing in every region in Ontario, Canada. The proportion of FM graduates undertaking further emergency training had doubled from 3.6% to 7.8%. From 1993 to 2003, a higher proportion of the most recent FM graduates did hospital visits, emergency room care and a lower proportion undertook home visits. Male FM graduates appear to have had higher workloads compared with female FM graduates, though the difference between them was decreasing over time. A 1997 policy initiative to discount fees paid to new FPs practicing in areas deemed over supplied did result in a decrease in the proportion of FM graduates practicing in metropolitan areas.</p> <p>Conclusions</p> <p>We were able to profile the practices of FM graduates using existing and routinely collected population-based health administrative data. Further work tracking FM graduates could be helpful for physician resource forecasting and in examining the impact of policies on family medicine practice.</p