Principles in Patterns (PiP) : Project Evaluation Synthesis

Evaluation activity found the technology-supported approach to curriculum design and approval developed by PiP to demonstrate high levels of user acceptance, promote improvements to the quality of curriculum designs, render more transparent and efficient aspects of the curriculum approval and quality monitoring process, demonstrate process efficacy and resolve a number of chronic information management difficulties which pervaded the previous state. The creation of a central repository of curriculum designs as the basis for their management as "knowledge assets", thus facilitating re-use and sharing of designs and exposure of tacit curriculum design practice, was also found to be highly advantageous. However, further process improvements remain possible and evidence of system resistance was found in some stakeholder groups. Recommendations arising from the findings and conclusions include the need to improve data collection surrounding the curriculum approval process so that the process and human impact of C-CAP can be monitored and observed. Strategies for improving C-CAP acceptance among the "late majority", the need for C-CAP best practice guidance, and suggested protocols on the knowledge management of curriculum designs are proposed. Opportunities for further process improvements in institutional curriculum approval, including a re-engineering of post-faculty approval processes, are also recommended

Principles in Patterns (PiP) : Piloting of C-CAP - Evaluation of Impact and Implications for System and Process Development

The Principles in Patterns (PiP) project is leading a programme of innovation and development work intended to explore and develop new technology-supported approaches to curriculum design, approval and review. It is anticipated that such technology-supported approaches can improve the efficacy of curriculum approval processes at higher education (HE) institutions, thereby improving curriculum responsiveness and enabling improved and rapid review mechanisms which may produce enhancements to pedagogy. Curriculum design in HE is a key "teachable moment" and often remains one of the few occasions when academics will plan and structure their intended teaching. Technology-supported curriculum design therefore presents an opportunity for improving academic quality, pedagogy and learning impact. Approaches that are innovative in their use of technology offer the promise of an interactive curriculum design process within which the designer is offered system assistance to better adhere to pedagogical best practice, is exposed to novel and high impact learning designs from which to draw inspiration, and benefits from system support to detect common design issues, many of which can delay curriculum approval and distract academic quality teams from monitoring substantive academic issues. This strand of the PiP evaluation (WP7:38) attempts to understand the impact of the PiP Class and Course Approval Pilot (C-CAP) system within specific stakeholder groups and seeks to understand the extent to which C-CAP is considered to support process improvements. As process improvements and changes were studied in a largely quantitative capacity during a previous but related evaluative strand, this strand includes the gathering of additional qualitative data to better understand and verify the business process improvements and change effected by C-CAP. This report therefore summarises the outcome of C-CAP piloting within a University faculty, presents the methodology used for evaluation, and the associated analysis and discussion. More generally this report constitutes an additional evaluative contribution towards a wider understanding of technology-supported approaches to curriculum design and approval in HE institutions and their potential in improving process transparency, efficiency and effectiveness

An exploratory trial implementing a community-based child oral health promotion intervention for Australian families from refugee and migrant backgrounds: a protocol paper for Teeth Tales

Introduction: Inequalities are evident in early childhood caries rates with the socially disadvantaged experiencing greater burden of disease. This study builds on formative qualitative research, conducted in the Moreland/Hume local government areas of Melbourne, Victoria 2006–2009, in response to community concerns for oral health of children from refugee and migrant backgrounds. Development of the community-based intervention described here extends the partnership approach to cogeneration of contemporary evidence with continued and meaningful involvement of investigators, community, cultural and government partners. This trial aims to establish a model for child oral health promotion for culturally diverse communities in Australia.<p></p> Methods and analysis: This is an exploratory trial implementing a community-based child oral health promotion intervention for Australian families from refugee and migrant backgrounds. Families from an Iraqi, Lebanese or Pakistani background with children aged 1–4 years, residing in metropolitan Melbourne, were invited to participate in the trial by peer educators from their respective communities using snowball and purposive sampling techniques. Target sample size was 600. Moreland, a culturally diverse, inner-urban metropolitan area of Melbourne, was chosen as the intervention site. The intervention comprised peer educator led community oral health education sessions and reorienting of dental health and family services through cultural Competency Organisational Review (CORe).<p></p> Ethics and dissemination: Ethics approval for this trial was granted by the University of Melbourne Human Research Ethics Committee and the Department of Education and Early Childhood Development Research Committee. Study progress and output will be disseminated via periodic newsletters, peer-reviewed research papers, reports, community seminars and at National and International conferences.<p></p&gt

Evaluation of the 20,000 days campaign

The aim of Counties Manuaku District Health Board (CMDHB) 20,000 Days Campaign was to give back to the community 20,000 healthy and well days to avoid predicted growth in hospital bed days. After tracking the difference between projected demand and actual use, at the end of the Campaign on 1st July 2013, CMDHB reported that 23,060 bed days were given back to the people of Counties Manukau. This evaluation report explains how using the Institute of Healthcare Improvement Breakthrough Series the Campaign was run with the expectation that small immediate changes to practical problems (in this case the work of 13 Collaborative teams), will accumulate into large effects (a reduction of 20,000 bed days against predicted bed days use by July 2013). The evaluation found the Campaign did save bed days, though attributing causality was always going to be difficult, and overall the Campaign was very successful in keeping the energy and motivation of participants

Using Theory of Change to design and evaluate complex mental health interventions in low and middle income countries: the case of PRIME

Background: Many health interventions are complex and consist of several interacting components (Medical Research Council 2008). These components include multiple causal strands, outcomes and levels of governance and may result in unexpected outcomes and non-linear change (Glouberman and Zimmerman 2002). As such they present challenges to the design and evaluation of complex health interventions. Although broad theoretical guidance has been developed by the UK Medical Research Council (MRC) (Craig, Dieppe et al. 2008), it contains little practical guidance and has been criticised for not including theory driven approaches to evaluation such as Theory of Change (ToC) (Anderson 2008). De Silva, Breuer et al. (2014) have proposed that ToC may complement the MRC guidance on complex health interventions. Methods: This study explores how ToC can strengthen the design and evaluation of complex health interventions using the example of The Programme for Improving Mental Health Care (PRIME). PRIME is a research programme which aimed to develop, implement and evaluate the integration of mental health into primary health in districts or sub-districts in Ethiopia, India, Nepal, South Africa and Uganda. A ToC approach was used in addition to other approaches to design and evaluate these complex mental health interventions. Firstly, I conducted a systematic review to determine the extent to which ToC has been used to design and evaluate public health interventions. Secondly, I compared the process of developing the ToC between all five PRIME countries and reflected on the value of ToC workshops using a framework analysis of workshop documentation and interviews with facilitators. Thirdly, I explored the development of the ToCs within the programme as a whole and the implications for the development of the intervention and the choice of evaluation methods. Fourthly, I presented a ToC for the integration of mental health care in low and middle income countries. Fifthly, I demonstrated how ToC can be used as a framework for a qualitative comparative analysis of process and outcome data using longitudinal data from 10 PRIME implementation facilities in Nepal. Lastly, I provide a set of 10 key lessons learned from PRIME in the application of ToC to complex mental health interventions. Results I found that the ToC approach has been used for the design and evaluation of public health interventions since the 1990s. However, there is a lack of clear description of the use of ToC in the literature on public health interventions and inconsistency in how it is used. In applying the ToC approach to PRIME, I found that facilitators reported that ToC workshops were a valuable way to develop ToCs and that different stakeholders at the workshop contribute different types of information to the ToC process. Hierarchies within the health system are an important consideration for ToC workshops as power dynamics are likely to influence the functioning of the group. In addition, I found that the development of a cross country ToC can result in a programme theory which is relevant for complex multilevel intervention in different contexts. This ToC can provide a framework to map contextually relevant interventions and can be used to complement other intervention development approaches. The ToC can also be used to ensure indicators for all the short-, medium- and long-term outcomes are identified. However, combining process and outcome data analysis using the ToC is not straightforward. Qualitative Comparative Analysis (QCA) can be used to analyse process and outcome data in a single analysis in health services research. Conclusion: ToC can be used to strengthen the design and evaluation of complex health interventions and can be used to complement the MRC guidance in the design and evaluation of complex health interventions

Sustaining Continuous Improvement in Public Sector Services Through Double Loop Learning

Abstract Purpose: Public Service Organisations (PSOs) are facing continuing funding challenges and increased pressure to maintain and improve service delivery with fewer resources. One response, with the promise of improving efficiency rather than cutting services, has been to implement Continuous Improvement (CI) but success has been sporadic and unpredictable. The well documented and pervasive CI methodologies in PSOs, including Lean and Six Sigma, have general agreement across practitioners and scholars alike, thus the reasons behind their potted success must lie elsewhere, in the culture or the environment perhaps? This work explores the wider contextual issues of CI implementation with the aim of providing guidelines to give a greater confidence of successful implementation. Study design/methodology/approach: A structured literature review provided the initial conceptual framework that was further developed through a series of in-depth, semi-structured interviews carried out with industry experts. The framework illustrates the phenomenon of organisational learning in relation to CI in UK PSOs. Findings: The research shows that emphasis should be placed on addressing logic and mindsets at an individual and organisational level in order to re-focus CI efforts and achieve sustainable process improvement. Particular attention should be placed on the role of leaders. Originality/value: This research takes a unique approach to CI in the UK PSO context, providing insights into the achievement of sustainable CI and a theoretical framework for addressing PSO logic. It establishes a theoretical foundation for the evaluation of organisational learning in relation to sustainable CI in UK PSOs. It also makes practical recommendations to support PSO to reveal, evaluate and address organisational principles through interactive workshops, which are currently being piloted in a UK health sector organisation. Research should continue to focus on the critical role of organisational learning and governing variables in relation to addressing PSO logic for sustained CI. Key words: Public Service Organisation (PSO), Continuous Improvement (CI), organisational learning, sustainment Paper Type: Research pape

Ambiguity and Conflict in Policy Implementation: The Case of the New Care Models (Vanguard) Programme in England

Policy driven change is challenging, with a significant gap between theory and practice. A key tension in enacting such change is achieving a balance between bottom-up development of local, context-specific approaches, and top-down, centrally determined policy solutions and their mutual sequencing. Ideal type models of the policy-making process envisage a rational ordered approach, driven by evidence and accompanied by ongoing evaluation of outcomes (Parsons, 1995, p77); however, the reality is far more complex. We examine the implementation and early operation of the New Care Models (NCM) Vanguard programme in England, using Matland’s (1995) ambiguity-conflict model, to explore the aims and expectations of the programme. We consider the relationship between top-down and bottom-up approaches to policy development and draw attention to the pressures coming from what was initially perceived as a permissive policy approach of encouraging experimentation, whilst also requiring rapid learning, scale and spread. We suggest that future programmes for large-scale policy implementation initiatives could be crafted differently to take account of the environment of implementation and render ambitions more realistic. Rather than aiming to create a set of definite products and templates, it may be that a set of principles for design and implementation should be developed and spread