Evaluation of complex integrated care programmes: the approach in North West London

Background: Several local attempts to introduce integrated care in the English National Health Service have been tried, with limited success. The Northwest London Integrated Care Pilot attempts to improve the quality of care of the elderly and people with diabetes by providing a novel integration process across primary, secondary and social care organisations. It involves predictive risk modelling, care planning, multidisciplinary management of complex cases and an information technology tool to support information sharing. This paper sets out the evaluation approach adopted to measure its effect. Study design: We present a mixed methods evaluation methodology. It includes a quantitative approach measuring changes in service utilization, costs, clinical outcomes and quality of care using routine primary and secondary data sources. It also contains a qualitative component, involving observations, interviews and focus groups with patients and professionals, to understand participant experiences and to understand the pilot within the national policy context. Theory and discussion: This study considers the complexity of evaluating a large, multi-organisational intervention in a changing healthcare economy. We locate the evaluation within the theory of evaluation of complex interventions. We present the specific challenges faced by evaluating an intervention of this sort, and the responses made to mitigate against them. Conclusions: We hope this broad, dynamic and responsive evaluation will allow us to clarify the contribution of the pilot, and provide a potential model for evaluation of other similar interventions. Because of the priority given to the integrated agenda by governments internationally, the need to develop and improve strong evaluation methodologies remains strikingly important

Linking factual and procedural knowledge in solving science problems: A case study in a thermodynamics course

Well-specified problems of the type presented boxed in the introduction to this article are extremely common in science courses. Unfortunately, this does not mean that students find them easy to solve, even when a teacher provides model answers to problems which differ only marginally (in the teacher's eyes) from those put before the students. The central difficulty with such courses is that they do not embody instructional principles that reflect students' need for “direction” in problem solving. In this article, we describe how the necessary heuristics and strategic knowledge were built into the remake of a conventional thermodynamics course. In contrast to mainstream American work on learning problem solving we chose to direct our curriculum reconstruction using the Gal'perin theory of stage-by-stage formation of mental actions and Landa's description of the “through” systematization of knowledge. As indicated by both, we first developed an integrated system of instructional objectives: a programme of actions and methods (PAM) to solve problems in thermodynamics. Then the plan of instruction was designed. This plan indicates which instructional procedures and materials should be used to realize the instructional functions, derived from the learning theory. The evaluation design contained two control and three experimental courses. In discussing our main findings, we consider the generalizability of the procedures we followed in constructing the PAM and the instructional plan

Terms of Reference: Evaluability Assessments of the Regional Integrated Initiatives, CGIAR

The CGIAR Independent Advisory and Evaluation Services (IAES) 2022‒24 multi-year workplan provides for the evaluability assessments of Regional Integrated Initiatives (RIIs) in 2023. These ToRs respond to the mandate of the CGIAR IAES Evaluation Function to execute the evaluability study; and sets out the purpose, objectives and a process to follow, in accordance with the CGIAR Evaluability Assessment Guidelines. The primary purpose of the Evaluability Assessment is to ensure evaluability and preparedness for an evaluation, aligned to different placing in the project cycle. The goal of the assessment is to support the RIIs in preparing for successful evaluations, which can involve assessing the adequacy of the ToC, establishing the interests of the stakeholders, and determining whether available data would make it possible to formulate a judgment given the initiative timeframe and resources. Guided by IAES budget consideration, the RAFS Action Area director, and subsequently RII coordinator (appointed July 2023), the following criteria were applied: (1) status of the Initiative in the project cycle and funding levels; (2) explicit interest and/or availability from the RIIs; and (3) geographical representation, with potential for deeper learning at a regional level. The selected RIIs are: Fragility to Resilience in Central and West Asia and North Africa (CWANA); Asian Mega-Deltas (AMD); Transforming Agrifood Systems In South Asia (TAFFSA); West and Central African Food Systems Transformations (WCA). The EA approach will follow utilization-focused and collaborative, as well as focus on accountability and learning. This approach will be mixed methods of data collection and analysis: (1) documents and data reviews: a desk review of secondary data sources: documents and related resources; and (2) primary data collection: key informant interviews and focus group discussions. Please contact [email protected] for related inquiries

Laptops for teachers: An evaluation of the TELA scheme in schools (Years 4 to 6)

The purpose of this evaluation was to investigate the impacts of the Laptops for Teachers Scheme (referred to from here as the TELA scheme) on Years 4 to 6 teachers’ work over a period of three years (2004-2006) and to record emerging changes in laptop use. The investigation focused on the Ministry of Education expectation (Ministry of Education, 2004) that teacher access to a laptop for their individual professional use would lead to gains in confidence and expertise in the use of ICTs, to efficiencies in administration, would contribute to teacher collaboration and support the preparation of high quality lesson resources. It was also anticipated that teacher would use their laptop in the classroom for teaching and learning

Development and Pilot Study of Group-Based Dietary Self-Management Program for Community Dwellers with Hypertension

Background: In most countries worldwide, hypertension is considered as an important problem. Moreover, an increasing trend in the prevalence and incidence has been reported in most countries. This increasing trend requires an innovative approach to improve the lifestyle modification of hypertensive sufferers including their dietary behaviors. Objective: This developmental research aims to develop a program for improving the dietary behaviors of community dwellers with hypertension. Method: The process of this program development includes a literature review related to the self-management programs for hypertension, and dietary behavior outcomes, expert validation, and pilot testing. Result: The setting, strategies, duration, and outcome measurement from the literature review were taken into consideration to develop the new program. The newly developed group-based self-management program consists of: 1) the sharing and reflecting of individual current dietary behavior, 2) group educational session, 3) individual comparison of behavior and reflection of obstacles, 4) individual goal setting, and 5) follow up. In the educational session, the DASH eating plan is used as the reference as it is commonly used in studies about diet for hypertension

Research gaps in scale up of family planning and reproductive health programming

The Evidence Project, in collaboration with the Evidence to Action Project, the Health Policy Project, and MEASURE Evaluation convened a meeting to discuss research gaps related to scale-up. The meeting was held at the request of USAID’s Office of Population and Reproductive Health (OPRH) to help guide their research strategy. The meeting had four objectives: discuss scale-up experiences and the role of research and monitoring for strengthening program performance at scale; identify research gaps related to scale-up; prioritize the scale-up research gaps for OPRH; identify next steps for addressing these priority research gaps. This meeting generated a rich discussion about the need to better document the scale-up process, particularly the vertical (institutionalization) aspect of scale-up. Key questions revolved around determining capacity building for scale, identifying and disseminating the facilitating factors, working with key health-system elements as defined by WHO, shortening the scale-up timeframe, and balancing sustainability with equity. The results of this meeting will be used to inform the OPRH Research Strategy and the work of implementing partners