Identification of complex health interventions suitable for evaluation: development and validation of the 8-step scoping framework

Background: There is extensive literature on the methodology of evaluation research and the development and evaluation of complex interventions but little guidance on the formative stages before evaluation and how to work with partner organizations that wish to have their provision evaluated. It is important to be able to identify suitable projects for evaluation from a range of provision and describe the steps required, often with academic institutions working in partnership with external organizations, in order to set up an evaluation. However, research evaluating programs or interventions rarely discusses these stages. Objective: This study aimed to extend work on evaluability assessment and pre-evaluation planning by proposing an 8-Step Scoping Framework to enable the appraisal of multiple programs in order to identify interventions suitable for evaluation. We aimed to add to the literature on evaluability assessment and more recent evaluation guidance by describing the processes involved in working with partner organizations. Methods: This paper documents the steps required to identify multiple complex interventions suitable for process and outcome evaluation. The steps were developed using an iterative approach by working alongside staff in a local government organization, to build an evidence base to demonstrate which interventions improve children’s outcomes. The process of identifying suitable programs for evaluation, thereby establishing the pre-evaluation steps, was tested using all Flying Start provision. Results: The 8-Step Scoping Framework was described using the example of the local government organization Flying Start to illustrate how each step contributes to finding projects suitable for process and outcome evaluation: (1) formulating overarching key questions that encompass all programs offered by an organization, (2) gaining an in-depth understanding of the work and provision of an organization and engaging staff, (3) completing a data template per project/program offered, (4) assessing the robustness/validity of data across all programs, (5) deciding on projects suitable for evaluation and those requiring additional data, (6) negotiating with chosen project leads, both within and outside the organization, (7) developing individual project evaluation protocols, and (8) applying for ethical approval from the university and partner organization. Conclusions: This paper describes the processes involved in identifying suitable projects for evaluation. It adds to the existing literature on the assessment of specific programs suitable for evaluation and guidance for conducting evaluations by establishing the formative steps required to identify suitable programs from a range of provision. This scoping framework particularly relates to academic partners and organizations tasked with delivering evidence-based services designed to meet local needs. The steps identified have been described in the context of early years provision but can be applied to a range of community-based evaluations, or more generally, to cases where an academic partner is working with external stakeholders to identify projects suitable for academic evaluation

Carotid Artery Stenting Using a Novel Self-Expanding Braided Nickel–Titanium Stent: Feasibility and Safety Porcine Trial

We studied the deliverability and safety of a braided, self-expanding, closed-cell nickel–titanium (NiTi) stent (E-volution, Jotec GmbH, Hechingen, Germany) especially designed for the endovascular treatment of carotid artery bifurcation stenosis with special regard to in-stent stenosis and thrombosis compared with a laser-cut reference nitinol stent in a porcine model of percutaneous vascular interventions. We aimed to assess histopathologic response in minipig carotid and subclavian arteries. Eight minipigs received a total of 42 stents: 14 reference stents and 28 E-volution stents. Eleven of the E-volution stents were additionally coated with heparin. Control angiography was obtained immediately before and after vascular intervention as well as 4 weeks after the procedure. Primary endpoints were 28 days of angiographic analyses as well as histomorphometric analysis, including injury score, inflammation score, luminal diameter, vessel diameter, maximal neointimal thickness, and area of in-stent stenosis. Secondary end points were procedural success, 28-day mortality, and stent thrombosis. All stents could be delivered successfully without procedural complications, morbidity, or mortality during our observation time. As confirmed by histology, no in-stent thrombosis was observed. Compared with common carotid arteries, subclavian arteries are significantly more vulnerable to developing in-stent stenosis caused by neointima proliferation (p < 0.05). Compared with the use of 1 single stent/artery, serial application of two stents leads to a more excessive but not significantly different neointimal proliferation (p > 0.05). The E-volution stent, especially when heparin coated, is in line with the comparison to the laser-cut reference stent displaying similar results of angiographic, histologic, and histomorphometric analyses (p > 0.05). Compared with the reference laser-cut stent, the self-expanding nitinol stent (E-volution) with its advanced braiding technology is feasible and safe. In our opinion, the high radial resistive force and the advanced braided design with tight stent-strut interstices may be beneficial in terms of plaque stabilization. Further studies are necessary and warranted

Consumer satisfaction with primary care provider choice and associated trust

BACKGROUND: Development of managed care, characterized by limited provider choice, is believed to undermine trust. Provider choice has been identified as strongly associated with physician trust. Stakeholders in a competitive healthcare market have competing agendas related to choice. The purpose of this study is to analyze variables associated with consumer's satisfaction that they have enough choice when selecting their primary care provider (PCP), and to analyze the importance of these variables on provider trust. METHODS: A 1999 randomized national cross-sectional telephone survey conducted of United States residential households, who had a telephone, had seen a medical professional at least twice in the past two years, and aged ≥ 20 years was selected for secondary data analyses. Among 1,117 households interviewed, 564 were selected as the final sample. Subjects responded to a core set of questions related to provider trust, and a subset of questions related to trust in the insurer. A previously developed conceptual framework was adopted. Linear and logistic regressions were performed based on this framework. RESULTS: Results affirmed 'satisfaction with amount of PCP choice' was significantly (p < .001) associated with provider trust. 'PCP's care being extremely effective' was strongly associated with 'satisfaction with amount of PCP choice' and 'provider trust'. Having sought a second opinion(s) was associated with lower trust. 'Spoke to the PCP outside the medical office,' 'satisfaction with the insurer' and 'insurer charges less if PCP within network' were all variables associated with 'satisfaction with amount of PCP choice' (all p < .05). CONCLUSION: This study confirmed the association of 'satisfaction with amount of PCP choice' with provider trust. Results affirmed 'enough PCP choice' was a strong predictor of provider trust. 'Second opinion on PCP' may indicate distrust in the provider. Data such as 'trust in providers in general' and 'the role of provider performance information' in choice, though import in PCP choice, were not available for analysis and should be explored in future studies. Results have implications for rethinking the relationships among consumer choice, consumer behaviors in making trade-offs in PCP choice, and the role of healthcare experiences in 'satisfaction with amount of PCP choice' or 'provider trust.

Tuberculosis care: an evaluability study

OBJECTIVE: to verify whether the tuberculosis control program (TCP) is evaluable and to examine the feasibility of building an evaluation model in apriority municipality for the control of tuberculosis.METHOD: this evaluability study was conducted in a municipality in northeastern Brazil. For data collection, documental analysis and interviews with key informants were performed. For indicator validation, the nominal group technique was adopted.RESULTS: the details of TCP were described, and both the logical model and the classification framework for indicators were developed and agreed up on, with the goal of characterizing the structural elements of the program, defining the structure and process indicators, and formulating the evaluation questions.CONCLUSION: TCP is evaluable. Based on logical operational analysis, it was possible to evaluate the adequacy of the program goals for the control of tuberculosis. Therefore, the performance of a summative evaluation is recommended, with a focus on the analysis of the effects of tuberculosis control interventions on decreasing morbidity and mortality