Supporting strategy : a survey of UK OR/MS practitioners

This paper reports the results of an on-line survey conducted with practitioner members of the UK Operational Research (OR) Society. The purpose of the survey was to explore the current practice of supporting strategy in terms of activities supported and tools used. The results of the survey are compared to those of previous surveys to explore developments in, inter-alia, the use of management/strategy tools and „soft‟ Operational Research / Management Science (OR/MS) tools. The survey results demonstrate that OR practitioners actively support strategy within their organisations. Whilst a wide variety of tools, drawn from the OR/MS and management / strategy fields are used to support strategy within organisations, the findings suggest that soft OR/MS tools are not regularly used. The findings also demonstrate that tools are combined to support strategy from both within and across the OR/MS and management / strategy fields. The paper ends by identifying a number of areas for further research

Evaluating Social Innovation

The philanthropic sector has been experimenting with innovative grantmaking in the hopes of triggering significant and sustainable change. FSG's latest research report, collaboratively written with the Center for Evaluation Innovation, challenges grantmakers to explore the use of Developmental Evaluation when evaluating complex, dynamic, and emergent initiatives

Evaluating Participatory Modeling: Developing a Framework for Cross-case Analysis

Participatory modeling is increasingly recognised as an effective way to assist collective decision-making processes in the domain of natural resource management. This paper introduces a framework for evaluating projects that have adopted a participatory modeling approach. This framework – known as the ‘Protocol of Canberra’ – was developed through a collaboration between French and Australian researchers engaged in participatory modeling and evaluation research. The framework seeks to assess the extent to which different participatory modeling practices reinforce or divert from the theoretical assumptions they are built upon. The paper discusses the application of the framework in three case-studies, two from Australia and one from the Pacific island of the Republic of Kiribati. The paper concludes with some comments for future use of the framework in a range of participatory modeling contexts, including fostering consideration of why and how different methodological approaches are used to achieve project aims and to build a collective vision amongst diverse stakeholders.participation, modeling, evaluation, complex systems science

A systems approach to evaluate One Health initiatives

Challenges calling for integrated approaches to health, such as the One Health (OH) approach, typically arise from the intertwined spheres of humans, animals, and ecosystems constituting their environment. Initiatives addressing such wicked problems commonly consist of complex structures and dynamics. As a result of the EU COST Action (TD 1404) “Network for Evaluation of One Health” (NEOH), we propose an evaluation framework anchored in systems theory to address the intrinsic complexity of OH initiatives and regard them as subsystems of the context within which they operate. Typically, they intend to influence a system with a view to improve human, animal, and environmental health. The NEOH evaluation framework consists of four overarching elements, namely: (1) the definition of the initiative and its context, (2) the description of the theory of change with an assessment of expected and unexpected outcomes, (3) the process evaluation of operational and supporting infrastructures (the “OH-ness”), and (4) an assessment of the association(s) between the process evaluation and the outcomes produced. It relies on a mixed methods approach by combining a descriptive and qualitative assessment with a semi-quantitative scoring for the evaluation of the degree and structural balance of “OH-ness” (summarised in an OH-index and OH-ratio, respectively) and conventional metrics for different outcomes in a multi-criteria-decision-analysis. Here, we focus on the methodology for Elements (1) and (3) including ready-to-use Microsoft Excel spreadsheets for the assessment of the “OH-ness”. We also provide an overview of Element (2), and refer to the NEOH handbook for further details, also regarding Element (4) (http://neoh.onehealthglobal.net). The presented approach helps researchers, practitioners, and evaluators to conceptualise and conduct evaluations of integrated approaches to health and facilitates comparison and learning across different OH activities thereby facilitating decisions on resource allocation. The application of the framework has been described in eight case studies in the same Frontiers research topic and provides first data on OH-index and OH-ratio, which is an important step towards their validation and the creation of a dataset for future benchmarking, and to demonstrate under which circumstances OH initiatives provide added value compared to disciplinary or conventional health initiatives

“Pilot implementation of an interdisciplinary course on climate solutions”

A pilot implementation of an experimental interdisciplinary course on climate solutions was undertaken at San Jose´ State University in the fall semester of 2008. The course, co-taught by seven faculty members from six colleges, was approved for a general education requirement and was open to upperclass students campus-wide. A course with such a breadth of topics and range of student backgrounds was the first of its kind here. The lessons learned from the pilot effort were assessed from student, faculty, and administrative perspectives. The educational benefits to students from the interdisciplinary format were found to be substantial, in addition to faculty development. However, challenges associated with team-teaching were also encountered and must be overcome for the long-term viability of the course. The experimental course was approved as a permanent course starting in the fall semester of 2009 based on the pilot effort, and plays a role in the College of Engineering’s recent initiatives in sustainability in addition to campus-wide general educatio

What Should EDA Fund? Developing a Model for Pre-Assessment of Economic Development Investments

This paper describes the completion of a “comprehensive study of regionalism” that was conducted by a joint team of economists and economic development specialists for the Economic Development Administration (EDA). The project consisted of two main activities: an examination of the factors associated with economic development success and the creation of a practical interactive tool for EDA project assessment and comparison. Findings from surveys, interviews, and project case studies are discussed in terms of their support for a positive relationship between successful economic development efforts and factors such as leadership and private investment. Also, the authors discuss the creation of a quantitative assessment model utilizing well-known approaches such as economic impact multipliers and cluster theory. The primary contribution of this work to the existing body of EDA-focused research and evaluation literature is introducing a means of using standardized scores, also known as z-scores, to compare and assess economic development projects across both industries and regions.regional economic development, economic impact, assessment model, EDA, z-score

Supporting Defect Causal Analysis in Practice with Cross-Company Data on Causes of Requirements Engineering Problems

[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-company data on causes of requirements engineering problems from 74 Brazilian organizations and built a Bayesian network. Our DCA approach uses the diagnostic inference of the Bayesian network to support DCA sessions. We evaluated our approach by applying a model for technology transfer to industry and conducted three consecutive evaluations: (i) in academia, (ii) with industry representatives of the Fraunhofer Project Center at UFBA, and (iii) in an industrial case study at the Brazilian National Development Bank (BNDES). [Results] We received positive feedback in all three evaluations and the cross-company data was considered helpful for determining main causes. [Conclusions] Our results strengthen our confidence in that supporting DCA with cross-company data is promising and should be further investigated.Comment: 10 pages, 8 figures, accepted for the 39th International Conference on Software Engineering (ICSE'17

Changing Stakeholder Needs and Changing Evaluator Roles: The Central Valley Partnership of the James Irvine Foundation

This case study describes the evolution of the evaluator's role as the program evolved and developed, and as the needs of the client and intended users changed over time. The initiative aimed to assist immigrants in California's Central Valley. The case illustrates important tensions among accountability, learning and capacity building purposes of evaluation