Average difference between the indicators computed in pairs of sources.

<p>On the left (local governance scenario): comparison between MR and IAD. On the right (central governance scenario): comparison between IAD and “best estimate”. For each indicator the p-value of the significance of the difference is shown.</p

Comparison of compliance measured by IAD, by MR or by either of the two data sources, on the whole population.

<p>Difference in the value of indicators between the patients that IAD correctly identified as having the disease and patients not detected by IAD (ND), and between the patients that IAD correctly identified as having the disease and patients that IAD only classified as having the disease (FD). Difference was computed using EITHER for compliance, and adjusting per age, gender and LHU. Standards are listed in decreasing order of Cohen’s kappa.</p

Monitoring compliance with standards of care for chronic diseases using healthcare administrative databases in Italy: Strengths and limitations

<div><p>Background</p><p>A recent comprehensive report on healthcare quality in Italy published by the Organization of Economic Co-operation and Development (OECD) recommended that regular monitoring of quality of primary care by means of compliance with standards of care for chronic diseases is performed. A previous ecological study demonstrated that compliance with standards of care could be reliably estimated on regional level using administrative databases. This study compares estimates based on administrative data with estimates based on GP records for the same persons, to understand whether ecological fallacy played a role in the results of the previous study.</p><p>Methods</p><p>We compared estimates of compliance with diagnostic and therapeutic standards of care for type 2 diabetes (T2DM), hypertension and ischaemic heart disease (IHD) from administrative data (IAD) with estimates from medical records (MR) for the same persons registered with 24 GP’s in 2012. Data were linked at an individual level.</p><p>Results</p><p>32,688 persons entered the study, 12,673 having at least one of the three diseases according to at least one data source. Patients not detected by IAD were many, for all three conditions: adding MR increased the number of cases of T2DM, hypertension, and IHD by +40%, +42%, and +104%, respectively. IAD had imperfect sensitivity in detecting population compliance with therapies (adding MR increased the estimate, from +11.5% for statins to +14.7% for antithrombotics), and, more substantially, with diagnostic recommendations (adding MR increased the estimate, from +23.7% in glycated hemoglobin tests, to +50.5% in electrocardiogram). Patients not detected by IAD were less compliant with respect to those that IAD correctly identified (from -4.8 percentage points in proportion of IHD patients compliant with a yearly glycated hemoglobin test, to -40.1 points in the proportion of T2DM patients compliant with the same recommendation). IAD overestimated indicators of compliance with therapeutic standards (significant differences ranged from 3.3. to 3.6 percentage points) and underestimated indicators of compliance with diagnostic standards (significant differences ranged from -2.3 to -14.1 percentage points).</p><p>Conclusion</p><p>IAD overestimated the percentage of patients compliant with therapeutic standards by less than 6 percentage points, and underestimated the percentage of patients compliant with diagnostic standards by a maximum of 14 percentage points. Therefore, both discussions at local level between GP's and local health unit managers and discussions at central level between national and regional policy makers can be informed by indicators of compliance estimated by IAD, which, based on those results, have the ability of signalling critical or excellent clusters. However, this study found that estimates are partly flawed, because a high number of patients with chronic diseases are not detected by IAD, patients detected are not representative of the whole population of patients, and some categories of diagnostic tests are markedly underrecorded in IAD (up to 50% in the case of electrocardiograms). Those results call to caution when interpreting IAD estimates. Audits based on medical records, on the local level, and an interpretation taking into account information external to IAD, on the central level, are needed to assess a more comprehensive compliance with standards.</p></div

Scenarios of quality governance where the results from this study can be used.

<p>IAD: administrative databases, MR: primary care medical records, EITHER: either among IAD or MR.</p

Standards of care, with levels and grades of recommendation.

<p>SID: Italian Diabetes Society. ESC/EASD: European Society of Cardiology and European Association for the Study of Diabetes. ESH/ESC: European Society of Hypertension and European Society of Cardiology. ACC/AHA: American Cardiology Association and American Heart Association. A symbol ^{<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0188377#t001fn001" target="_blank">*</a>} means that the recommendation only applies when the condition is at a high level of severity. Diagnostic tests are recommended once per year, except HbA1c for T2DM which is recommended twice a year.</p

Patients detected by IAD or by MR, for each disease.

<p>“Patients that IAD only classified as having the disease” were those detected by IAD, but not by MR. "Patients that IAD correctly identified as having the disease" were identified by both IAD and MR. “Patients not detected by IAD" were identified by MR, but not by IAD.</p

Scatter plots comparing age-and- gender standardised measures of compliance with standards of care, in the two governance scenarios.

<p>In the Local governance scenario the 24 clusters of patients of the same GP are measured by IAD on the Y-axis and MR on the X-axis. In the Central governance scenario the 5 clusters of patients in the same LHU are measured by IAD on the Y-axis and best estimate (proportion of patients detected by MR with are compliant according to EITHER) on the X-axis.</p

Indicators of compliance with standards of care during a year of follow-up.

<p><b>ATC</b>: Anatomical Therapeutic Chemical classification system for drugs. ICSO: Italian coding system for outpatient services.</p

Study population.

<p>Description of the GPs in the five (A, B, C, D, E) pairs of samples (HS, measured by means of clinical data, and VALORE, measured by means of healthcare administrative data). Description of the general population they have in charge (general population), of diabetic patients (Diabetes), of patients with ischaemic heart disease (IHD) and of patients with heart failure (HF). N GP: number of GPs in the study. N populations: number of inhabitants registered with the GPs in the study. N patients: total number of patients with the chronic disease registered with GPs in the study. N registered per GP: average number of persons in charge to each GP, with test for difference in means within each pair. Female: percentage of women in the population, with test for difference in means within each pair. Age band: age distribution of the population, with chi square test within each pair.</p

Box-plots of the distribution of indicators of quality of care for diabetes (2 indicators), IHD (3 indicators) and HF (3 indicators) in 5 pairs of samples of GPs.

<p>Each pair contains the distribution obtained from the VALORE data (dark gray) and the distribution obtained from HSD data (light gray). For each indicator the pair of samples are ordered according to the median value in the VALORE sample.</p