An Exploratory Study on Public Sector Program Evaluation Practices and Culture in Barbados, Belize, Guyana and Saint Vincent and the Grenadines: Where Are We? Where Do We Need To Go?

Background: Little is known about the status of program evaluation culture and practice in the English Speaking Commonwealth Caribbean (ESCC). This study examined the extent of evaluation culture and practice in four small Caribbean nations: Barbados, Belize, Guyana, and Saint Vincent and the Grenadines.   Purpose: The objective of this study was to learn about public sector program evaluation practices and culture in these four countries.   Setting: Public sector agencies in the four countries.   Intervention: NA   Research Design:  Cross-sectional survey   Data Collection and Analysis: A total of eighty-eight public sector officials participated in a cross-sectional survey study.   Findings: The research highlighted that while program evaluation is being practiced in the public sector in all four countries, the extent of practice varied among the countries. One noticeable factor that affected program evaluation practices was the financial costs associated with program evaluations. This problem was particularly evident in Guyana where program evaluations were conducted primarily to satisfy funding requirements. However, despite the variation in practices, all four countries recognized that program evaluation is a valuable and important activity. Solutions proposed by the study participants to build a more vibrant public-sector program evaluation culture included making program evaluation culture a part of organizational culture, building organizational monitoring and evaluation (M&E) capacity, and making program evaluation a regular mandatory public sector activity

A Practical Framework and Model for Promoting Cost-Inclusive Evaluations

Background: Cost studies are an important component of any serious professional evaluation. Regardless of whether an evaluation is conducted in the public or private sector, decision-makers want answers to two fundamental questions: (1) Is the program, project, or investment worthwhile? and (2) Can funds be used in a better way? The answers to both questions rely on the use of cost analysis techniques. However, according to the literature, most evaluations do not include any type of cost study (see Christie & Fleischer, 2010) and when studies are included, very few are of high quality and rigor (see Madsen, Eddleston, Hansen, & Konradsen, 2017). This may in part be as a result of the relatively limited emphasis placed on this important topic in the evaluation-specific literature, coupled with the reality that many evaluators lack expertise to conduct cost analysis studies. Purpose: Given the limited use of cost studies in professional evaluations to assess actual program merit/worth, this paper presents a practical framework/model to help evaluators understand fundamental issues that must be considered when thinking about some form of cost-inclusive evaluation. Setting: Not Applicable. Intervention: Not Applicable. Research Design:  External desk research was used to ascertain the extent to which cost analysis is discussed in the evaluation literature and the reasons highlighted for underuse. This review provided the foundation for developing the practical framework/model presented in this paper for promoting cost-inclusive evaluation. Data Collection and Analysis: Not Applicable. Findings: Cost analysis, which can considerably enrich professional evaluations, is currently quite underused in the evaluation profession. Notable progress has been made over the last 15 years with government agencies around the globe taking the lead to promote cost analysis. However,  most independent evaluators are still failing to measure, report, and analyze costs because many do not understand the fundamentals of cost studies. This paper promotes cost-inclusive evaluation by providing evaluators with a framework/model to help them understand important issues in cost analysis.                   Keywords: cost-analysis, costs, cost studies, benefits, evaluation, framework, measurement, model, valuation

Evaluation in Our New Normal Environment: Navigating the Challenges with Data Collection

Background: Data collection is a critical component of all evaluations. However, it often presents a number of challenges under the best of circumstances. For instance, the evaluation budget and time frame both have implications for the quality and type of data that is collected. Additionally, adherence to high quality international ethical best practices is necessary when collecting data for any purpose, methodological rigor is important for ensuring the credibility of the evaluation, improving access to important documents and stakeholders, as well as decreasing excessive evaluation anxiety on the part of critical stakeholders, when possible, is vital. These challenges have now been considerably exacerbated by the COVID-19 global health pandemic which has changed our world in fundamental ways. In what is now considered as our new normal environment, evaluators will need to make profound changes to the manner in which they plan and undertake data collection. Objectives: This paper examines the many and varied challenges that will be encountered with data collection in our new normal environment. This new normal has had an impact on evaluation practices in all countries, developed and developing, and has significantly amplified existing challenges in countries with limited evaluation culture, budgets, technological coverage, access, and connectivity. It makes an important contribution to the literature since data collection has historically and traditionally been conducted using primarily face-to-face field work and through the freedom of movement of people to undertake this task. Setting: Not applicable. Intervention: Not applicable. Research Design: Desk review was utilized for the preparation of this paper. Findings: Evaluators need to be extremely flexible, innovative, and amendable to different approaches to data collections as our new normal environment will likely be with us for a while. This pandemic has thrown everyone a very painful curveball and introduced significant new work-related challenges for a myriad of work types and work environments. Innovation and the willingness to learn new methods have become an important necessity to help with learning, accountability, transparency. The COVID-19 pandemic has highlighted the plight of the most vulnerable and evidence-based data is the only means to assist this group. Evaluators must rise to the challenge, devise new ways to collect data that is credible and useful, and continue to promote the importance and benefits of the field of evaluation. As such, evaluators have an important role to play in the global economic recovery efforts. Keywords: budgets, challenges; COVID-19; data collection; evaluation; evaluators; new normal environment; time fram

Enhancing the Strategic Management Process Through the Use of Professional Evaluation Methods and the Logic of Evaluation

Background: The evaluation discipline has the potential to enhance other disciplines by integrating evaluation specific methodologies and logic into processes within other disciplines and improving/strengthening the manner in which evaluation of processes, programs and policies are carried out. Purpose: This paper will highlight the evaluative nature of one of the most popular strategic management models (SMM) in corporate America, namely, Fred David’s (2013) SMM, and examine how professional evaluation logic and methodology can be used to improve David's SMM. Setting: NA Intervention: NA Research Design: The paper will make a comparative analysis between Michael Scriven's Key Evaluation Checklist (KEC) and David's SMM and highlight similarities and differences between the two models. Data Collection and Analysis: NA Findings: The paper will conclude by providing several suggestions to enhance and strengthen David's SMM to make it more robust, valuable, and useful in the business environment

Reconstructing irreducible links in temporal networks: which tool to choose depends on the network size

n/