Evaluating the Chile Solidario program: results using the Chile Solidario panel and the administrative databases

This paper presents the results of a three-year evaluation performed on the first cohorts of Chile Solidario, the most important anti-poverty program in Chile. The paper presents a description of the program, emphasizing the mechanism by which people were admitted into the program. We then propose evaluation strategies and discuss their validity. The final evaluation is conducted using a Matching estimator, and we discuss the principles surrounding the potential for this to be a valid evaluation method. The initial results using the Chile Solidario Panel suggest that the program had positive effects on psychosocial welfare and on take-up of subsidies and social programs. However, it is not possible to obtain reliable results due to data shortcomings, particularly the lack of baseline data. In order to solve the problem, we generated a database using six years of administrative data, including around 1,000,000 family records per year. A method for overcoming the treatment substitution problem is discussed and implemented. Results are much more robust than those of the Chile Solidario Panel and show small, but clearly positive effects for several variables, especially the number of workers in the family, the percentage of workers in the famly and the employment of the head of the family.Matching, Extreme poverty, Impact evaluation, Treatment substitution, Administrative data.

A Framework for Open Policy Analysis

The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While several social science disciplines are undergoing a “credibility revolution” focused on openness and replication, policy analysis has yet to systematically embrace transparency and reproducibility. We argue that policy analysis should adopt the open research practices increasingly espoused in related disciplines to advance the credibility of evidence-based policymaking. We first discuss the importance of evidence-based policy in an era of increasing disagreement about facts, analysis, and expertise. We present a novel framework for “open” policy analysis (OPA) and how to achieve it, focusing on examples of recent policy analyses that have incorporated open research practices such as transparent reporting, open data, and code sharing. We conclude with recommendations on how key stakeholders in evidence-based policy can make OPA the norm and thus safeguard trust in using empirical evidence to inform important public policy decisions

Author Correction: A consensus-based transparency checklist.

An amendment to this paper has been published and can be accessed via a link at the top of the paper

A consensus-based transparency checklist

We present a consensus-based checklist to improve and document the transparency of research reports in social and behavioural research. An accompanying online application allows users to complete the form and generate a report that they can submit with their manuscript or post to a public repository

Evaluating the Chile Solidario program: results using the Chile Solidario panel and the administrative databases

This paper presents the results of a three-year evaluation performed on the first cohorts of Chile Solidario, the most important anti-poverty program in Chile. The paper presents a description of the program, emphasizing the mechanism by which people were admitted into the program. We then propose evaluation strategies and discuss their validity. The final evaluation is conducted using a Matching estimator, and we discuss the principles surrounding the potential for this to be a valid evaluation method. The initial results using the Chile Solidario Panel suggest that the program had positive effects on psychosocial welfare and on take-up of subsidies and social programs. However, it is not possible to obtain reliable results due to data shortcomings, particularly the lack of baseline data. In order to solve the problem, we generated a database using six years of administrative data, including around 1,000,000 family records per year. A method for overcoming the treatment substitution problem is discussed and implemented. Results are much more robust than those of the Chile Solidario Panel and show small, but clearly positive effects for several variables, especially the number of workers in the family, the percentage of workers in the family and the employment of the head of the familyEste artículo presenta los resultados de una evaluación de tres años realizada en las primeras cohortes del Programa Chile Solidario, el programa de reducción de la pobreza más importante de Chile. El artículo presenta una descripción del programa, haciendo hincapié en el mecanismo por el cual las personas fueron admitidas en el programa. A continuación, propone estrategias de evaluación y discute su validez. La evaluación final se realiza mediante un estimador de �Matching� (pareo), y se discuten los principios en base a los cuales se trataría de un método de evaluación válida. Los resultados iniciales usando el Panel de Chile Solidario sugieren que el programa ha tenido efectos positivos sobre el bienestar psicosocial y en la adopción de subsidios y programas sociales. Sin embargo, no es posible obtener resultados fiables debido a las deficiencias de datos, en particular la falta de una línea de base. Con el fin de resolver el problema, hemos generado una base de datos utilizando seis años de datos administrativos, incluyendo alrededor de 1.000.000 de registros de familias por año. Un método para superar el problema de sustitución de tratamiento es discutido y aplicado. Los resultados son mucho más robustos que los del Panel de Chile Solidario y muestran pequeños, pero evidentes efectos positivos de varias variables, especialmente relativos al número de trabajadores en el hogar, el porcentaje de trabajadores en el hogar y el empleo del jefe de hoga

Evaluating the Chile Solidario program: results using the Chile Solidario panel and the administrative databases

This paper presents the results of a three-year evaluation performed on the first cohorts of Chile Solidario, the most important anti-poverty program in Chile. The paper presents a description of the program, emphasizing the mechanism by which people were admitted into the program. We then propose evaluation strategies and discuss their validity. The final evaluation is conducted using a Matching estimator, and we discuss the principles surrounding the potential for this to be a valid evaluation method. The initial results using the Chile Solidario Panel suggest that the program had positive effects on psychosocial welfare and on take-up of subsidies and social programs. However, it is not possible to obtain reliable results due to data shortcomings, particularly the lack of baseline data. In order to solve the problem, we generated a database using six years of administrative data, including around 1,000,000 family records per year. A method for overcoming the treatment substitution problem is discussed and implemented. Results are much more robust than those of the Chile Solidario Panel and show small, but clearly positive effects for several variables, especially the number of workers in the family, the percentage of workers in the family and the employment of the head of the family

A framework for open policy analysis

The evidence-based policy movement promotes the use of empirical evidence to inform policy decision-making. While several social science disciplines are undergoing a 'credibility revolution' focused on openness and replication, policy analysis has yet to systematically embrace transparency and reproducibility. We argue that policy analysis should adopt the open research practices increasingly espoused in related disciplines to advance the credibility of evidence-based policy making. We first discuss the importance of evidence-based policy in an era of increasing disagreement about facts, analysis, and expertise. We present a novel framework for 'open' policy analysis (OPA) and how to achieve it, focusing on examples of recent policy analyses that have incorporated open research practices such as transparent reporting, open data, and code sharing. We conclude with recommendations on how key stakeholders in evidence-based policy can make OPA the norm and thus safeguard trust in using empirical evidence to inform important public policy decisions

Evaluando el programa Chile Solidario: resultados usando el panel Chile Solidario y datos administrativos

This paper presents the results of a three-year evaluation performed on the first cohorts of Chile Solidario, the most important anti-poverty program in Chile. The paper presents a description of the program, emphasizing the mechanism by which people were admitted into the program. We then propose evaluation strategies and discuss their validity. The final evaluation is conducted using a Matching estimator, and we discuss the principles surrounding the potential for this to be a valid evaluation method. The initial results using the Chile Solidario Panel suggest that the program had positive effects on psychosocial welfare and on take-up of subsidies and social programs. However, it is not possible to obtain reliable results due to data shortcomings, particularly the lack of baseline data. In order to solve the problem, we generated a database using six years of administrative data, including around 1,000,000 family records per year. A method for overcoming the treatment substitution problem is discussed and implemented. Results are much more robust than those of the Chile Solidario Panel and show small, but clearly positive effects for several variables, especially the number of workers in the family, the percentage of workers in the family and the employment of the head of the family.Este artículo presenta los resultados de una evaluación de tres años realizada en las primeras cohortes del Programa Chile Solidario, el programa de reducción de la pobreza más importante de Chile. El artículo presenta una descripción del programa, haciendo hincapié en el mecanismo por el cual las personas fueron admitidas en el programa. A continuación, propone estrategias de evaluación y discute su validez. La evaluación final se realiza mediante un estimador de “Matching” (pareo), y se discuten los principios en base a los cuales se trataría de un método de evaluación válida. Los resultados iniciales usando el Panel de Chile Solidario sugieren que el programa ha tenido efectos positivos sobre el bienestar psicosocial y en la adopción de subsidios y programas sociales. Sin embargo, no es posible obtener resultados fiables debido a las deficiencias de datos, en particular la falta de una línea de base. Con el fin de resolver el problema, hemos generado una base de datos utilizando seis años de datos administrativos, incluyendo alrededor de 1.000.000 de registros de familias por año. Un método para superar el problema de sustitución de tratamiento es discutido y aplicado. Los resultados son mucho más robustos que los del Panel de Chile Solidario y muestran pequeños, pero evidentes efectos positivos de varias variables, especialmente relativos al número de trabajadores en el hogar, el porcentaje de trabajadores en el hogar y el empleo del jefe de hogar