Performance of statistical models to predict mental health and substance abuse cost

BACKGROUND: Providers use risk-adjustment systems to help manage healthcare costs. Typically, ordinary least squares (OLS) models on either untransformed or log-transformed cost are used. We examine the predictive ability of several statistical models, demonstrate how model choice depends on the goal for the predictive model, and examine whether building models on samples of the data affects model choice. METHODS: Our sample consisted of 525,620 Veterans Health Administration patients with mental health (MH) or substance abuse (SA) diagnoses who incurred costs during fiscal year 1999. We tested two models on a transformation of cost: a Log Normal model and a Square-root Normal model, and three generalized linear models on untransformed cost, defined by distributional assumption and link function: Normal with identity link (OLS); Gamma with log link; and Gamma with square-root link. Risk-adjusters included age, sex, and 12 MH/SA categories. To determine the best model among the entire dataset, predictive ability was evaluated using root mean square error (RMSE), mean absolute prediction error (MAPE), and predictive ratios of predicted to observed cost (PR) among deciles of predicted cost, by comparing point estimates and 95% bias-corrected bootstrap confidence intervals. To study the effect of analyzing a random sample of the population on model choice, we re-computed these statistics using random samples beginning with 5,000 patients and ending with the entire sample. RESULTS: The Square-root Normal model had the lowest estimates of the RMSE and MAPE, with bootstrap confidence intervals that were always lower than those for the other models. The Gamma with square-root link was best as measured by the PRs. The choice of best model could vary if smaller samples were used and the Gamma with square-root link model had convergence problems with small samples. CONCLUSION: Models with square-root transformation or link fit the data best. This function (whether used as transformation or as a link) seems to help deal with the high comorbidity of this population by introducing a form of interaction. The Gamma distribution helps with the long tail of the distribution. However, the Normal distribution is suitable if the correct transformation of the outcome is used

Variation in antibiotic treatment for diabetic patients with serious foot infections: A retrospective observational study

<p>Abstract</p> <p>Background</p> <p>Diabetic foot infections are common, serious, and diverse. There is uncertainty about optimal antibiotic treatment, and probably substantial variation in practice. Our aim was to document whether this is the case: A finding that would raise questions about the comparative cost-effectiveness of different regimens and also open the possibility of examining costs and outcomes to determine which should be preferred.</p> <p>Methods</p> <p>We used the Veterans Health Administration (VA) Diabetes Epidemiology Cohorts (DEpiC) database to conduct a retrospective observational study of hospitalized patients with diabetic foot infections. DEpiC contains computerized VA and Medicare patient-level data for VA patients with diabetes since 1998, including demographics, ICD-9-CM diagnostic codes, antibiotics prescribed, and VA facility. We identified all patients with ICD-9-CM codes for cellulitis/abscess of the foot and then sub-grouped them according to whether they had cellulitis/abscess plus codes for gangrene, osteomyelitis, skin ulcer, or none of these. For each facility, we determined: 1) The proportion of patients treated with an antibiotic and the initial route of administration; 2) The first antibiotic regimen prescribed for each patient, defined as treatment with the same antibiotic, or combination of antibiotics, for at least 5 continuous days; and 3) The antibacterial spectrum of the first regimen.</p> <p>Results</p> <p>We identified 3,792 patients with cellulitis/abscess of the foot either alone (16.4%), or with ulcer (32.6%), osteomyelitis (19.0%) or gangrene (32.0%). Antibiotics were prescribed for 98.9%. At least 5 continuous days of treatment with an unchanged regimen of one or more antibiotics was prescribed for 59.3%. The means and (ranges) across facilities of the three most common regimens were: 16.4%, (22.8%); 15.7%, (36.1%); and 10.8%, (50.5%). The range of variation across facilities proved substantially greater than that across the different categories of foot infection. We found similar variation in the spectrum of the antibiotic regimen.</p> <p>Conclusions</p> <p>The large variations in regimen appear to reflect differences in facility practice styles rather than case mix. It is unlikely that all regimens are equally cost-effective. Our methods make possible evaluation of many regimens across many facilities, and can be applied in further studies to determine which antibiotic regimens should be preferred.</p

Medical profiling: improving standards and risk adjustments using hierarchical models. J Health Econ

Abstract The conclusions from a profile analysis to identify performance extremes can be affected substantially by the standards and statistical methods used and by the adequacy of risk adjustment. Medically meaningful standards are proposed to replace common statistical standards. Hierarchical regression methods can handle several levels of random variation, make risk adjustments for the providers&apos; case-mix differences, and address the proposed standards. These methods determine probabilities needed to make meaningful profiles of medical units based on standards set by all appropriate parties. Published by Elsevier Science B.V

Suicide risk in Veterans Health Administration patients with mental health diagnoses initiating lithium or valproate: a historical prospective cohort study

BACKGROUND: Lithium has been reported in some, but not all, studies to be associated with reduced risks of suicide death or suicidal behavior. The objective of this nonrandomized cohort study was to examine whether lithium was associated with reduced risk of suicide death in comparison to the commonly-used alternative treatment, valproate. METHODS: A propensity score-matched cohort study was conducted of Veterans Health Administration patients (n=21,194/treatment) initiating lithium or valproate from 1999-2008. RESULTS: Matching produced lithium and valproate treatment groups that were highly similar in all 934 propensity score covariates, including indicators of recent suicidal behavior, but recent suicidal ideation was not able to be included. In the few individuals with recently diagnosed suicidal ideation, a significant imbalance existed with suicidal ideation more prevalent at baseline among individuals initiating lithium than valproate (odds ratio (OR) 1.30, 95% CI 1.09, 1.54; p=0.003). No significant differences in suicide death were observed over 0-365 days in A) the primary intent-to-treat analysis (lithium/valproate conditional odds ratio (cOR) 1.22, 95% CI 0.82, 1.81; p=0.32); B) during receipt of initial lithium or valproate treatment (cOR 0.86, 95% CI 0.46, 1.61; p=0.63); or C) after such treatment had been discontinued/modified (OR 1.51, 95% CI 0.91, 2.50; p=0.11). Significantly increased risks of suicide death were observed after the discontinuation/modification of lithium, compared to valproate, treatment over the first 180 days (OR 2.72, 95% CI 1.21, 6.11; p=0.015). CONCLUSIONS: In this somewhat distinct sample (a predominantly male Veteran sample with a broad range of psychiatric diagnoses), no significant differences in associations with suicide death were observed between lithium and valproate treatment over 365 days. The only significant difference was observed over 0-180 days: an increased risk of suicide death, among individuals discontinuing or modifying lithium, compared to valproate, treatment. This difference could reflect risks either related to lithium discontinuation or higher baseline risks among lithium recipients (i.e., confounding) that became more evident when treatment stopped. Our findings therefore support educating patients and providers about possible suicide-related risks of discontinuing lithium even shortly after treatment initiation, and the close monitoring of patients after lithium discontinuation, if feasible. If our findings include residual confounding biasing against lithium, however, as suggested by the differences observed in diagnosed suicidal ideation, then the degree of beneficial reduction in suicide death risk associated with active lithium treatment would be underestimated. Further research is urgently needed, given the lack of interventions against suicide and the uncertainties concerning the degree to which lithium may reduce suicide risk during active treatment, increase risk upon discontinuation, or both