A simulation study of sample size for multilevel logistic regression models

<p>Abstract</p> <p>Background</p> <p>Many studies conducted in health and social sciences collect individual level data as outcome measures. Usually, such data have a hierarchical structure, with patients clustered within physicians, and physicians clustered within practices. Large survey data, including national surveys, have a hierarchical or clustered structure; respondents are naturally clustered in geographical units (e.g., health regions) and may be grouped into smaller units. Outcomes of interest in many fields not only reflect continuous measures, but also binary outcomes such as depression, presence or absence of a disease, and self-reported general health. In the framework of multilevel studies an important problem is calculating an adequate sample size that generates unbiased and accurate estimates.</p> <p>Methods</p> <p>In this paper simulation studies are used to assess the effect of varying sample size at both the individual and group level on the accuracy of the estimates of the parameters and variance components of multilevel logistic regression models. In addition, the influence of prevalence of the outcome and the intra-class correlation coefficient (ICC) is examined.</p> <p>Results</p> <p>The results show that the estimates of the fixed effect parameters are unbiased for 100 groups with group size of 50 or higher. The estimates of the variance covariance components are slightly biased even with 100 groups and group size of 50. The biases for both fixed and random effects are severe for group size of 5. The standard errors for fixed effect parameters are unbiased while for variance covariance components are underestimated. Results suggest that low prevalent events require larger sample sizes with at least a minimum of 100 groups and 50 individuals per group.</p> <p>Conclusion</p> <p>We recommend using a minimum group size of 50 with at least 50 groups to produce valid estimates for multi-level logistic regression models. Group size should be adjusted under conditions where the prevalence of events is low such that the expected number of events in each group should be greater than one.</p

Effectiveness of aerobic exercise for adults living with HIV: systematic review and meta-analysis using the Cochrane Collaboration protocol

Background: People with HIV are living longer with the health-related consequences of HIV, multi-morbidity, and aging. Exercise is a key strategy that may improve or sustain health for people living with HIV. Our aim was to examine the safety and effectiveness of aerobic exercise interventions on immunological, virological, cardiorespiratory, strength, weight, body composition, and psychological outcomes in adults living with HIV. Methods: We conducted a systematic review using the Cochrane Collaboration protocol. We searched databases up to April 2013. We included randomized controlled trials comparing aerobic exercise with no exercise or another intervention performed at least three times per week for at least four weeks among adults living with HIV. Two reviewers independently determined study eligibility. Data were extracted from studies that met inclusion criteria using standardized forms. We assessed risk of bias using the Cochrane Collaboration’s tool for assessing risk of bias. Outcomes were analyzed as continuous and meta-analyses conducted using random effects models with Review Manager (RevMan) computer software. Results: Twenty-four studies met inclusion criteria (n = 936 participants at study completion); the majority of participants were men (73 %) and the majority were taking antiretroviral therapy (19/24 included studies). The exercise intervention included aerobic exercise alone (11 studies) or a combination of aerobic and resistive exercise (13 studies) ranging from 5 to 52 weeks. Fifty-eight meta-analyses were performed. Main results indicated statistically significant improvements in selected outcomes of cardiorespiratory status (maximum oxygen consumption, exercise time), strength (chest press, knee flexion), body composition (lean body mass, percent body fat, leg muscle area), depression symptoms, and quality of life (SF-36 questionnaire) among exercisers compared with non-exercisers. No significant differences in change in CD4 count and viral load were found. Conclusions: Performing aerobic exercise or a combination of aerobic and resistive exercise at least three times per week for at least five weeks is safe and can lead to improvements in cardiorespiratory fitness, strength, body composition and quality of life for adults with HIV. Aerobic exercise is safe and beneficial for adults living with HIV who are medically stable

Did changing primary care delivery models change performance? A population based study using health administrative data

<p>Abstract</p> <p>Background</p> <p>Primary care reform in Ontario, Canada started with the introduction of new enrollment models, the two largest of which are Family Health Networks (FHNs), a capitation-based model, and Family Health Groups (FHGs), a blended fee-for-service model. The purpose of this study was to evaluate differences in performance between FHNs and FHGs and to compare performance before and after physicians joined these new primary care groups.</p> <p>Methods</p> <p>This study used Ontario administrative claims data to compare performance measures in FHGs and FHNs. The study population included physicians who belonged to a FHN or FHG for at least two years. Patients were included in the analyses if they enrolled with a physician in the two years after the physician joined a FHN or FHG, and also if they saw the physician in a two year period prior to the physician joining a FHN or FHG. Performance was derived from the administrative data, and included measures of preventive screening for cancer (breast, cervical, colorectal) and chronic disease management (diabetes, heart failure, asthma).</p> <p>Results</p> <p>Performance measures did not vary consistently between models. In some cases, performance approached current benchmarks (Pap smears, mammograms). In other cases it was improving in relation to previous measures (colorectal cancer screening). There were no changes in screening for cervical cancer or breast cancer after joining either a FHN or FHG. Colorectal cancer screening increased in both FHNs and FHGs. After enrolling in either a FHG or a FHN, prescribing performance measures for diabetes care improved. However, annual eye examinations decreased for younger people with diabetes after joining a FHG or FHN. There were no changes in performance measures for heart failure management or asthma care after enrolling in either a FHG or FHN.</p> <p>Conclusions</p> <p>Some improvements in preventive screening and diabetes management which were seen amongst people after they enrolled may be attributed to incentive payments offered to physicians within FHGs and FHNs. However, these primary care delivery models need to be compared with other delivery models and fee for service practices in order to describe more specifically what aspects of model delivery and incentives affect care.</p

Primary care bonus payments and patient-reported access in urban Ontario: a cross-sectional study

BACKGROUND: Rurality strongly correlates with higher pay-for-performance access bonuses, despite higher emergency department use and fewer primary care services than in urban settings. We sought to evaluate the relation between patient-reported access to primary care and access bonus payments in urban settings. METHODS: We conducted a cross-sectional, secondary data analysis using Ontario survey and health administrative data from 2013 to 2017. We used administrative data to calculate annual access bonuses for eligible urban family physicians. We linked this payment data to adult (≥ 16 yr) patient data from the Health Care Experiences Survey to examine the relation between access bonus achievement (in quintiles of the proportion of bonus achieved, from lowest [Q1, reference category] to highest [Q5]) and 4 patient-reported access outcomes. The average survey response rate to the patient survey during the study period was 51%. We stratified urban geography into large, medium and small settings. In a multilevel regression model, we adjusted for patient-, physician- and practice-level covariates. We tested linear trends, adjusted for clustering, for each outcome. RESULTS: We linked 18 893 respondents to 3940 physicians in 414 bonus-eligible practices. Physicians in small urban settings earned the highest proportion of their maximum potential access bonuses. Access bonus achievement was positively associated with telephone access (Q2 odds ratio [OR] 1.18, 95% confidence interval [CI] 0.98-1.42; Q3 OR 1.34, 95% CI 1.10-1.63; Q4 OR 1.46, 95% CI 1.19-1.79; Q5 OR 1.87, 95% CI 1.50-2.33), after hours access (Q2 OR 1.26, 95% CI 1.09-1.47; Q3 OR 1.46, 95% CI 1.23-1.74; Q4 OR 1.77, 95% CI 1.46-2.15; Q5 OR 1.88, 95% CI 1.52-2.32), wait time for care (Q2 OR 1.01, 95% CI 0.85-1.20; Q3 OR 1.17, 95% CI 0.97-1.41; Q4 OR 1.27, 95% CI 1.05-1.55; Q5 OR 1.63, 95% CI 1.32-2.00) and timeliness (Q2 OR 1.29, 95% CI 0.98-1.69; Q3 OR 1.29, 95% CI 0.94-1.77; Q4 OR 1.58, 95% CI 1.16-2.13; Q5 OR 1.98, 95% CI 1.38-2.82). When stratified by geography, we observed several of these associations in large urban settings, but not in small urban settings. Trend tests were statistically significant for all 4 outcomes. INTERPRETATION: Although the access bonus correlated with access in larger urban settings, it did not in smaller settings, aligning with previous research questioning its utility in smaller geographies. The access bonus may benefit from a redesign that considers geography and patient experience

Is there an impact of public smoking bans on self-reported smoking status and exposure to secondhand smoke?

<p>Abstract</p> <p>Background</p> <p>Implementation of smoke free policies has potentially substantial effects on health by reducing secondhand smoke exposure. However little is known about whether the introduction of anti-smoking legislation translates into decreased secondhand smoke exposure. We examined whether smoking bans impact rates of secondhand smoke exposure in public places and rates of complete workplace smoking restriction.</p> <p>Methods</p> <p>Canadian Community Health Survey was used to obtain secondhand smoking exposure rates in 15 Ontario municipalities. Data analysis included descriptive summaries and 95% confidence intervals were calculated and compared across groups</p> <p>Results</p> <p>Across all studied municipalities, secondhand smoke exposure in public places decreased by 4.7% and workplace exposure decreased by 2.3% between the 2003 and 2005 survey years. The only jurisdiction to implement a full ban from no previous ban was also the only setting that experienced significant decreases in both individual exposure to secondhand smoke in a public place (-17.3%, 95% CI -22.8, -11.8) and workplace exposure (-18.1%, 95% CI -24.9, -11.3). Exposures in vehicles and homes declined in almost all settings over time.</p> <p>Conclusions</p> <p>Implementation of a full smoking ban was associated with the largest decreases in secondhand smoke exposure while partial bans and changes in existing bans had inconsistent effects. In addition to decreasing exposure in public places as would be expected from legislation, bans may have additional benefits by decreasing rates of current smokers and decreasing exposures to secondhand smoke in private settings.</p

Primary care and health inequality : Difference-in-difference study comparing England and Ontario

BACKGROUND: It is not known whether equity-oriented primary care investment that seeks to scale up the delivery of effective care in disadvantaged communities can reduce health inequality within high-income settings that have pre-existing universal primary care systems. We provide some non-randomised controlled evidence by comparing health inequality trends between two similar jurisdictions-one of which implemented equity-oriented primary care investment in the mid-to-late 2000s as part of a cross-government strategy for reducing health inequality (England), and one which invested in primary care without any explicit equity objective (Ontario, Canada). METHODS: We analysed whole-population data on 32,482 neighbourhoods (with mean population size of approximately 1,500 people) in England, and 18,961 neighbourhoods (with mean population size of approximately 700 people) in Ontario. We examined trends in mortality amenable to healthcare by decile groups of neighbourhood deprivation within each jurisdiction. We used linear models to estimate absolute and relative gaps in amenable mortality between most and least deprived groups, considering the gradient between these extremes, and evaluated difference-in-difference comparisons between the two jurisdictions. RESULTS: Inequality trends were comparable in both jurisdictions from 2004-6 but diverged from 2007-11. Compared with Ontario, the absolute gap in amenable mortality in England fell between 2004-6 and 2007-11 by 19.8 per 100,000 population (95% CI: 4.8 to 34.9); and the relative gap in amenable mortality fell by 10 percentage points (95% CI: 1 to 19). The biggest divergence occurred in the most deprived decile group of neighbourhoods. DISCUSSION: In comparison to Ontario, England succeeded in reducing absolute socioeconomic gaps in mortality amenable to healthcare from 2007 to 2011, and preventing them from growing in relative terms. Equity-oriented primary care reform in England in the mid-to-late 2000s may have helped to reduce socioeconomic inequality in health, though other explanations for this divergence are possible and further research is needed on the specific causal mechanisms

Modeling factors influencing the demand for emergency department services in ontario: a comparison of methods

<p>Abstract</p> <p>Background</p> <p>Emergency departments are medical treatment facilities, designed to provide episodic care to patients suffering from acute injuries and illnesses as well as patients who are experiencing sporadic flare-ups of underlying chronic medical conditions which require immediate attention. Supply and demand for emergency department services varies across geographic regions and time. Some persons do not rely on the service at all whereas; others use the service on repeated occasions. Issues regarding increased wait times for services and crowding illustrate the need to investigate which factors are associated with increased frequency of emergency department utilization. The evidence from this study can help inform policy makers on the appropriate mix of supply and demand targeted health care policies necessary to ensure that patients receive appropriate health care delivery in an efficient and cost-effective manner. The purpose of this report is to assess those factors resulting in increased demand for emergency department services in Ontario. We assess how utilization rates vary according to the severity of patient presentation in the emergency department. We are specifically interested in the impact that access to primary care physicians has on the demand for emergency department services. Additionally, we wish to investigate these trends using a series of novel regression models for count outcomes which have yet to be employed in the domain of emergency medical research.</p> <p>Methods</p> <p>Data regarding the frequency of emergency department visits for the respondents of Canadian Community Health Survey (CCHS) during our study interval (2003-2005) are obtained from the National Ambulatory Care Reporting System (NACRS). Patients' emergency department utilizations were linked with information from the Canadian Community Health Survey (CCHS) which provides individual level medical, socio-demographic, psychological and behavioral information for investigating predictors of increased emergency department utilization. Six different multiple regression models for count data were fitted to assess the influence of predictors on demand for emergency department services, including: Poisson, Negative Binomial, Zero-Inflated Poisson, Zero-Inflated Negative Binomial, Hurdle Poisson, and Hurdle Negative Binomial. Comparison of competing models was assessed by the Vuong test statistic.</p> <p>Results</p> <p>The CCHS cycle 2.1 respondents were a roughly equal mix of males (50.4%) and females (49.6%). The majority (86.2%) were young-middle aged adults between the ages of 20-64, living in predominantly urban environments (85.9%), with mid-high household incomes (92.2%) and well-educated, receiving at least a high-school diploma (84.1%). Many participants reported no chronic disease (51.9%), fell into a small number (0-5) of ambulatory diagnostic groups (62.3%), and perceived their health status as good/excellent (88.1%); however, were projected to have high Resource Utilization Band levels of health resource utilization (68.2%). These factors were largely stable for CCHS cycle 3.1 respondents. Factors influencing demand for emergency department services varied according to the severity of triage scores at initial presentation. For example, although a non-significant predictor of the odds of emergency department utilization in high severity cases, access to a primary care physician was a statistically significant predictor of the likelihood of emergency department utilization (OR: 0.69; 95% CI OR: 0.63-0.75) and the rate of emergency department utilization (RR: 0.57; 95% CI RR: 0.50-0.66) in low severity cases.</p> <p>Conclusion</p> <p>Using a theoretically appropriate hurdle negative binomial regression model this unique study illustrates that access to a primary care physician is an important predictor of both the odds and rate of emergency department utilization in Ontario. Restructuring primary care services, with aims of increasing access to undersupplied populations may result in decreased emergency department utilization rates by approximately 43% for low severity triage level cases.</p

Tracking family medicine graduates. Where do they go, what services do they provide and whom do they see?

<p>Abstract</p> <p>Background</p> <p>There are continued concerns over an adequate supply of family physicians (FPs) practicing in Canada. While most resource planning has focused on intake into postgraduate education, less information is available on what postgraduate medical training yields. We therefore undertook a study of Family Medicine (FM) graduates from the University of Toronto (U of T) to determine the type of information for physician resource planning that may come from tracking FM graduates using health administrative data. This study compared three cohorts of FM graduates over a 10 year period of time and it also compared FM graduates to all Ontario practicing FPs in 2005/06. The objectives for tracking the three cohorts of FM graduates were to: 1) describe where FM graduates practice in the province 2) examine the impact of a policy introduced to influence the distribution of new FM graduates in the province 3) describe the services provided by FM graduates and 4) compare workload measures. The objectives for the comparison of FM graduates to all practicing FPs in 2005/06 were to: 1) describe the patient population served by FM graduates, 2) compare workload of FM graduates to all practicing FPs.</p> <p>Methods</p> <p>The study cohort consisted of all U of T FM postgraduate trainees who started and completed their training between 1993 and 2003. This study was a descriptive record linkage study whereby postgraduate information for FM graduates was linked to provincial health administrative data. Comprehensiveness of care indicators and workload measures based on administrative data where determined for the study cohort.</p> <p>Results</p> <p>From 1993 to 2003 there were 857 University of Toronto FM graduates. While the majority of U of T FM graduates practice in Toronto or the surrounding Greater Toronto Area, there are FM graduates from U of T practicing in every region in Ontario, Canada. The proportion of FM graduates undertaking further emergency training had doubled from 3.6% to 7.8%. From 1993 to 2003, a higher proportion of the most recent FM graduates did hospital visits, emergency room care and a lower proportion undertook home visits. Male FM graduates appear to have had higher workloads compared with female FM graduates, though the difference between them was decreasing over time. A 1997 policy initiative to discount fees paid to new FPs practicing in areas deemed over supplied did result in a decrease in the proportion of FM graduates practicing in metropolitan areas.</p> <p>Conclusions</p> <p>We were able to profile the practices of FM graduates using existing and routinely collected population-based health administrative data. Further work tracking FM graduates could be helpful for physician resource forecasting and in examining the impact of policies on family medicine practice.</p

Validation of Case-Finding Algorithms Derived from Administrative Data for Identifying Adults Living with Human Immunodeficiency Virus Infection

OBJECTIVE: We sought to validate a case-finding algorithm for human immunodeficiency virus (HIV) infection using administrative health databases in Ontario, Canada. METHODS: We constructed 48 case-finding algorithms using combinations of physician billing claims, hospital and emergency room separations and prescription drug claims. We determined the test characteristics of each algorithm over various time frames for identifying HIV infection, using data abstracted from the charts of 2,040 randomly selected patients receiving care at two medical practices in Toronto, Ontario as the reference standard. RESULTS: With the exception of algorithms using only a single physician claim, the specificity of all algorithms exceeded 99%. An algorithm consisting of three physician claims over a three year period had a sensitivity and specificity of 96.2% (95% CI 95.2%-97.9%) and 99.6% (95% CI 99.1%-99.8%), respectively. Application of the algorithm to the province of Ontario identified 12,179 HIV-infected patients in care for the period spanning April 1, 2007 to March 31, 2009. CONCLUSIONS: Case-finding algorithms generated from administrative data can accurately identify adults living with HIV. A relatively simple "3 claims in 3 years" definition can be used for assembling a population-based cohort and facilitating future research examining trends in health service use and outcomes among HIV-infected adults in Ontario