State Plan Index: A Tool for Assessing the Quality of State Public Health Plans

Introduction The State Plan Index is an evaluation instrument that uses a Likert scale to assess 60 indicators of the quality of state public health plans. The State Plan Index was needed to enable evaluation of plans that were developed using a variety of public health planning models. Methods Federal, state, and academic partners participated in developing and testing the instrument. The authors conducted a literature review, interviews with experts, and several rounds of formative evaluation to assess item inclusion, coverage, weighting, organization of items, and content validity. In two rounds of field testing, public health practitioners at the federal and state levels rated 10 state public health plans for obesity prevention. Results Field-test raters took an average of two hours to rate a plan and indicated that the State Plan Index was “easy to use,” “comprehensive,” and “fair.” Mean Cronbach a for components of the State Plan Index was 0.88 (median 0.93). Component scores among the 10 plans rated ranged from 0.2 to 4.8, indicating that raters made distinctions in quality among the components and the plans they rated. Correlations between component scores and overall scores were statistically significant (P < .001), except for one component. Conclusion Public health professionals at the federal and state levels found the State Plan Index to be a useful tool for evaluating public health plans that were developed by states using various planning approaches. After the field tests, state staff reported adapting the State Plan Index for use as a planning tool, an evaluation tool for local plans, and a self-assessment tool for drafts of state plans. In addition, the State Plan Index can be revised easily for use in other chronic disease areas

Using the State Plan Index to Evaluate the Quality of State Plans to Prevent Obesity and Other Chronic Diseases

Introduction Implicit in public health planning models is the assumption that good public health plans lead to good programs, and good programs lead to desired health outcomes. Despite considerable resources that are devoted to developing plans, public health agencies and organizations have lacked a tool for evaluating the finished product of their planning efforts — the written plan itself — as an important indicator of progress. To address the need for an instrument to assess the quality of state plans designed to prevent and control chronic diseases, we created and tested the State Plan Index and used it to evaluate the quality of nine state plans aimed at preventing and reducing obesity. Methods The State Plan Index was developed under the auspices of the Centers for Disease Control and Prevention (CDC) in collaboration with public health experts in federal, state, and academic settings. The State Plan Index included 55 items related to plan quality arranged into nine components. Each item was rated on a Likert scale from 0 to 5, with 5 being the highest rating. Each plan also received a separate overall plan quality score using the same scale. Each state plan was evaluated by four or five raters using the State Plan Index. For each plan, the 55 items were averaged to calculate an item average score, and a subscore was calculated for each State Plan Index component. Finally, five states also self-rated their own plans (self score). Results The mean item average score for all plans was 2.4 out of 5.0. The range of item average scores was 1.0 to 3.0. The component of the State Plan Index with the highest mean component score (3.3) was Presentation of Epidemiologic Data on Disease Burden. The components with the lowest component scores were Resources for Plan Implementation (0.7); Integration of Obesity Efforts with Other Chronic Disease Efforts (1.7); and Program Evaluation (2.0). Plan quality was rated higher when based on the single overall plan quality score assigned by raters. In addition, self scores were consistently and substantially higher than rater-assigned scores. Conclusion Evaluation of plans early in the life of programs can be used to strengthen existing programs and to guide programs newly engaged in chronic disease prevention planning. The CDC has used the State Plan Index evaluation results to guide technical assistance, plan training sessions, and enhance communication with state staff about plan content, quality, and public health approach. Some state program directors self-evaluated their obesity draft plan and used the evaluation results to strengthen their planning process and to guide plan revisions. Other states have adapted the State Plan Index as a framework for new planning efforts to prevent obesity as well as other chronic diseases