Multidimensional evaluation of a program for early-career researcher in Brazil: the young investigator in emerging centers program

São Paulo Research Foundation, FAPESP, has implemented an award program as a way to develop new research centers and to provide more opportunity to recent doctors, or productive doctors without regular job position, the "Young Investigator in Emerging Centers" (YIEC). This paper intends to present main results from an external evaluation of this program, covering a period from 1996 to 2006. Results found in this evaluation evidence that YIEC has reached some of targets, mainly concerning the research activity in private institutions. This program was a very important instrument to create the basic infrastructure for research activities in this type of HI opening new opportunities to recent doctors. The creation and improvement of research groups out of the excellence axe also should be highlighted, since it is one of the main goals. In general, the impact in institutions could be considered expressive, since that even new graduation courses were created, and the impact in post-graduation programs was expressive, improving and creating new disciplines and courses. © 2011 IEEE.São Paulo Research Foundation, FAPESP, has implemented an award program as a way to develop new research centers and to provide more opportunity to recent doctors, or productive doctors without regular job position, the “Young Investigator in Emerging Censem informaçãosem informação(2010) Doutores 2010: Estudos Da Demografia Da Base Tecnico-cientifica Brasileira - Brasilia [Doctors 2010: Demographic Studies for the Brazilian Techno-scientifc Basis], , www.cgee.org.br/atividades/redirect.php?idProduto=6401, CGEE DF: Available at: hnpPian, C.A., Meneghini, R., Assessment of scientific programs: A necessary procedure for Brazilian scientific policy - The Young Investigator program of the Silo Paulo Research Foundation (2007) Anais Da Academia Rasileira de Ciências, 79 (3), pp. 543-557Georghiou, L., Roessner, D., Evaluating technology programs: Tools and methods (2000) Research Policy, 29, pp. 657-678Salles-Filho, S., Avilla, A.F., Sepulveda, J.E., Colugnati, F.A.B., Multidimensional assessment of technology and innovation programs: The impact evaluation of INCAGRO-Peru (2010) Research Evaluation, 19 (5), pp. 361-372. , 2010Salles-Filho, S.L.M., Bonacelli, M.B.M., Carneiro, A.M., Castro, P.F.D., Santos, F.O.A., Evaluation of ST &I programs: A methodological approach to the Brazilian small business program and some comparisons with the SBIR program (2011) Research Evaluation, 20 (2), pp. 157-169Greenacre, M.J., Interpreting Multiple Correspondence Analysis (1991) Appl. Stochastic Models Data Anal, 7 (2), pp. 195-210Carney, J., Smith, W., Parsad, A., Johnston, K., Millsap, M., (2008) Evaluation of the Faculty Early Career Development (CAREER) Program, , Abt Associates, Inc, Bethesda, MDBöhmer, S., Von Ins, M., Different - Not just by label: Research-oriented academic careers in Germany (2009) Research Evaluation, 18 (3), pp. 177-184Atlanta Conference on Science and Innovation Policy - SMARTec

Dietary Reference Intakes: Application Of Tables In Nutritional Studies [dietary Reference Intakes: Aplicabilidade Das Tabelas Em Estudos Nutricionais]

Diet planning and food intake evaluation are professional activities classically performed by comparing mean values of ingestion against reference values of nutrient and energy intakes, for either groups or individuals. Without considering certain intrinsic limitations of the reference values, such task may render equivocal conclusions with regard to nutritional adequacy. The nutritional recommendations, prepared as a joint effort between the US Institute of Medicine and Health Canada published since 1997 and known as Dietary Reference Intakes, offer a new model by establishing indices of nutrient intake and refining the concept of risk associated with diet evaluation. Errors of both intra and interindividual nature, due to the variability of an individual's feeding pattern, the uneven distribution of requirements within a population, respectively, and the small number of days of observation, are factors that together can have a large impact on the confidence of the analysis. For these reasons, only the judicious selection of the reference values, published in the form of tables from 1997 through 2005, should enable the user to reach the proper assessment. This paper had the objective of highlighting various critical characteristics of application and compiling the values in order to facilitate their use by both professionals and students of the area.196741760Hegsted, D.M., Establishment of nutritional requirements in man. Bordens Rev (1959) Nutr Res, 20 (2), pp. 13-22(1997) Dietary reference intakes for calcium, phosphorus, magnesium, vitamin D, and fluoride, , Institute of Medicine, Washington DC, National Academy Press;Food and Agriculture Organization of the United Nations. Human energy requirements. Rome, 2004. Report of a Joint FAO/WHO/UNU Expert Consultation Food and Nutrition Technical report series 1. [cited 2006 Mar 1]. Available from: http://www.fao.org/documents/show_.htmHauber, U., Neuhauser-Berthold, M., Historical development of dietary recommendations (1996) Z Ernährungswiss, 35 (2), pp. 157-162(1998) Dietary reference intakes: A risk assessment model for establishing upper intake levels for nutrients, , Institute of Medicine, Washington DC, National Academy Press;Food and Agriculture Organization of the United Nations. Human vitamin and mineral requirements. Rome2002. Report of a joint FAO/WHO expert consultation Bangkok(2003) Dietary reference intakes: Applications in dietary planning, , Institute of Medicine, Washington DC, National Academy Press;(2000) Dietary reference intakes: Applications in dietary assessment, , Institute of Medicine, Washington DC, National Academy Press;(2000) Dietary reference intakes for vitamin C, vitamin E, selenium, and carotenoids, , Institute of Medicine, Washington DC, National Academy Press;Nusser, S.M., Carriquiry, A.L., Dood, K.W., Fuller, W.A., A semiparametric transformation approach to estimating usual daily intake distribuitions (1996) J Am Stat Assoc, 91 (436), pp. 1440-1449(1998) Dietary reference intakes for thiamin, riboflavin, niacin, vitamin B6, folate, vitamin B12, pantothenic acid, biotin, and choline, , Institute of Medicine, Washington DC, National Academy Press;(2002) Dietary reference intakes for vitamin A, vitamin K, arsenic, boron, chromium, copper, iodine, iron, manganese, molybdenum, nickel, silicon, vanadium, and zinc, , Institute of Medicine, Washington DC, National Academy Press;Trumbo, P., Yates, A.A., Schlicker, S., Poos, M., Dietary reference intakes: Vitamin A, vitamin K, arsenic, boron, chromium, copper, iodine, iron, manganese, molybdenum, nickel, silicon, vanadium, and zinc (2001) J Am Diet Assoc, 101 (3), pp. 294-301Slater, B., Marchioni, D.L., Fisberg, R.M., Estimando a prevalência da ingestão inadequada de nutrientes (2004) Rev Saúde Pública, 38 (4), pp. 599-605Carriquiry, A.L., Assessing the prevalence of nutrient inadequacy (1999) Public Health Nutr, 2 (1), pp. 23-33(1986) Nutrient adequacy: Assessment using food consumption surveys, , National Research Council, Washington DC, National Academy Press;Amaya-Farfán, J., Domene, S.M.A., Padovani, R.M., DRIs, Síntese comentada das novas propostas sobre recomendações nutricionais para antioxidantes (2001) Rev Nutr, 14 (1), pp. 71-78Marchioni, D.M.L., Slater, B., Fisberg, R.M., Aplicação das dietary reference intakes na avaliação da ingestão de nutrientes para indivíduos (2004) Rev Nutr, 17 (2), pp. 207-216(2005) Dietary reference intakes for energy, carbohydrate, fiber, fat, fatty acids, cholesterol, protein, and amino acids, , Institute of Medicine, Washington DC, National Academy Press;(2004) Dietary reference intakes for water, potassium, sodium, chloride, and sulfate, , Institute of Medicine, Washington DC, National Academy Press;(2006) Dietary reference intakesthe essential guide to nutriente requirements, , Institute of Medicine, Washington DC, National Academy Press

Retinol Blood Levels In High School Students Of São Paulo, Brazil [retinol Sérico De Adolescentes De Uma Escola Da Cidade De São Paulo]

Objective: The purpose of the study was to examine plasma retinol levels in 218 adolescents of both sexes, ages 10 to 19 years old. All the subjects attended a private high school in the city of São Paulo. Methods: In order to evaluate the students' nutritional condition, the researchers performed anthropometrical measurements - weight and height - in addition to investigating the subjects' diet history. The retinol level was measured through the spectrophotometer technique, considering as inadequate values of concentration under <1.05μmol/L (30μg/dL). Results: The results indicate that the average intake, according to the age range, was higher than recommended, even though presenting high deviation patterns for both genders. The percentage of teenagers with plasma concentrations of vitamin A under the adequate level was 30% for both genders, and neither the analysis of the correlation between Body Mass Index and plasma concentrations of retinol, nor the analysis of the correlation between the density of vitamin A intake and plasma concentration of retinol, were significant. Conclusion: The research findings point out to the need for further investigation of vitamin A concentrations during adolescence, in order to check more accurately whether the low plasma concentration of this nutrient should be considered a risk factor, or the result of the accelerated absorption which takes place in adolescence to fulfill the metabolic requirements characteristic of this period of human growth.173291299West Jr., K.P., Dietary vitamin A deficiency: Effects on growth, infection, and mortality (1991) Food Nutr Bull, 13, pp. 119-131Evaion-Brior, D., Porquet, D., Therrond, P., Paulsen, A.F., Grenede, M.O., François, L., Vitamin A and nocturnal growth hormone secretion in short children (1994) Lancet, 343, pp. 87-88Herberth, B., Spyckerelle, Y., Deschamps, J.P., Determinants of plasma retinol b-carotenoid and a-tocopherol during adolescence (1991) Am J Clin Nutr, 54, pp. 884-889Brabin, L., Brabin, B., The cost successful adolescent growth and development in girls in relation to iron and vitamin A status (1992) Am J Clin Nutr, 55 (5), pp. 955-958(2000) Dietary Assessment, , Washington, DC: The Nacional Academy PressGama, C.M., (1999) Consumo Alimentar e Estado Nutricional de Adolescentes Matriculados na Rede de Ensino Particular e Pública do Bairro de Vila Mariana da Cidade de São Paulo, São Paulo, , [tese]. São Paulo: Pós-Graduação em Nutrição, Universidade Federal de São Paulo, Escola Paulista de MedicinaBatista Filho, M., Alimentação, nutrição e saúde (1999) Epidemiologia e Saúde. 5.Ed., pp. 353-374. , Rouquayrol MZ, Almeida Filho N. São Paulo: MEDSIRoncada, M.J., Wilson, D., Mazzilli, R.N., Gandra, Y.R., (1981) Hipovitaminose A em Comunidades do Estado de São Paulo, Brasil, 15, pp. 338-349. , Rev Saude PublicaWilson, D., Silva Nery, M.E., Hypovitaminosis A in Rio Grande do Sul, Brazil. Preliminary study (1983) Int J Vitam Res Suppl, 24, pp. 35-44Flores, H., Araújo, C.R., Liver levels of retinol in unselect necropsy specimens: A prevalence survey of vitamin A deficiency in Recife, Brazil (1984) Am J Clin Nutr, 40 (1), pp. 146-152Thompson, F.E., Byers, T., Dietary assessment resource manual (1994) J Nutr, 124 (SUPPL. 11), pp. 22455-23015Anção, M.S., Cuppari, L., Tudisco, E.S., Draibe, A.S., Sigulen, D.M., (1993) Sistema de Apoio à Decisão em Nutrição, Versão 2.5, , Centra de Informática em Saúde da Universidade Federal de São Paulo, Escola Paulista de Medicina. São Paulo(1996) Estudo Nacional de Despesa Familiar (ENDEF): Tabela de Composição de Alimentas. 4.Ed., , Secretaria de Planejamento da República. Rio de Janeiro: DEDIT/CDDIPinheiro, A.B., Lacerda, E.M.A., Benzercry, E., Gomes, M.C.S., Costa, V.M., (1996) Tabela para Avaliação de Consumo Alimentar em Medidas Caseiras. 3.Ed., , Rio de Janeiro: Produção IndependenteHansen, R.G., Wyse, B.W., Expression of nutrient allowances per 1000 kilocalories (1980) J Am Diet Assoc, 76 (3), pp. 233-227Must, A., Dallal, G.E., Dietz, W.H., Reference data for obesity: 85th and 95th percentiles of body mass index (wt/ht2) - A correction (1991) Am J Clin Nutr, 54, p. 773(1995) Physical Status: The Use and Interpretation of Anthropometry, , Geneva: World Health Organization. Technical Report Series n.854Bessey, O., Lowry, O.H., Brock, M.J., Lopez, J.A., The determination of vitamin A and carotene in small quantities of blood serum (1946) J Biol Chem, 11, pp. 177-188Araujo, C.R.C., Flores, H., Improved spectrophotometric vitamin A assay (1978) Letters Clin Chem, 24 (2), p. 386Flores, H., Importance of the early diagnosis of vitamin A deficiency at the epidemiological level (1983) Int J Vitam Res, 24, pp. 23-24Siegel, S., Castellan Jr., N.J., (1988) Nonparametrics Statistics, , New York: McGraw-HillDietary reference intakes (2003) Application in Dietary Planning, , Washington, DC: The National Academic Press: 2003. 237pUnderwood, B.A., Arthur, A., The contribuition of vitamin A to public health (1996) FASEB J, 10, pp. 1040-1048Sommer, A., (1995) Vitamin A Deficiency and Its Consequences: A Fiel Guide to Detection and Control - Epidemiology, pp. 1-73. , Geneva: World Health OrganizationSouza Queiroz, S., Sato, K., Torres, M.A.A., Detection of the prevalence of hipovitaminose A in children under 2 years of age enrolled in Basic Health care units cities of State of São Paulo (1998) 16th Proceedings of the Congress of Nutrition, p. 292. , 1998, Montréal. Montréal. (PT 659)Neuhouser, M.L., Rock, C.L., Eldrige, A.L., Kristal, A.R., Patterson, R.E., Cooper, D.A., Serum concentration of retinol, a-tocopherol and the carotenoids are influenced by diet, race and obesity in a sample of healthy adolescents (2001) J Nutr, 131, pp. 2184-2191Lewis, C.J., Mc Dowell, M.A., Sempos, C.T., Lewis, K.C., Yetley, E.A., Relationship between age and serum vitamin A in children age 4-11 (1990) Am J Clin Nutr, 52 (2), pp. 353-360Olson, J.A., Atwater Lecture. The irresistible fascination of carotenoids and vitamin A (1993) Am J Clin Nutr, 57 (6), pp. 833-839Lee, R.D., Nieman, D.C., (1995) Nutritional Assessment - Anthropometric, Biochemical, Clinical, Dietary, , St Louis: MOSB

Sampling Plan For The Brazilian Taco Poject

The TACO project (Brazilian Table of Food Composition), sponsored by the Brazilian Ministry of Health and executed by NEPA-UNICAMP (Center for Studies and Research on Foods-State University of Campinas), is generating new data on the most consumed foods in Brazil, based on a national sampling plan and analyses carried out by laboratories with demonstrated laboratory capability in nutrient analysis. Key foods have been chosen according to a national multicentric survey of food consumption, starting with 200 food items. The sampling plan covers nine cities in the five official Brazilian geopolitical regions (North, Northeast, South, Southeast and Centralwest), corresponding to approximately 16.8 million inhabitants out of a total Brazilian population of about 170 million. Samples of principal brands (maximum of five for each food) are collected from supermarkets/hypermarkets where 84-85% of total food purchases are made by the Brazilian population. Two units of each principal brand of each product are taken at each sampling site. The total units for each food from all regions are mixed and packed in cans, and three final composites of 100-200 g for each food are sent for analyses in approved laboratories. © 2002 Elsevier Science Ltd. All rights reserved.154499505Cohran, W.G., Sampling Techniques (1977), John Wiley & Sons, New YorkDomene, S.M.A., Lima, D.M., Padovani, R.M., Lajolo, F.M., Taddei, J.A.A.C., Galeazzi, M.A.M., TACO project-Brazilian food composition database (1999), Paper presented at the Third International Food Data Conference, RomeFranco, G.V.E., (1982) Nutrição: Texto Básico e Tabela de Composição Quimica dos Alimentos, , 6th edn. Atheneu, São PauloGaleazzi, M.A.M., Domene, S.M.A., Sichieri, R., Estudo multicêntrico sobre consumo alimentar (1997) Caderno de Debates, pp. 1-62. , (Eds.) special volumeGaleazzi, M.A.M., Domene, S.M.A., Zabotto, C., Inquérito de consumo alimentar familiar de Belém (1998), Relatório Ténico para o Centro Colaborador da Região Norte, Ministério da SaúdeGaleazzi, M.A.M., Falconi, P.J., (1998), Inquérito de consumo alimentar familiar (INCAF) de Brasília. Relatório Técnico para Secretaria de Saúde do DF(1977), IBGE. Estudo Nacional de Despesa Familiar-ENDEF: Tabela de Composição de Alimentos. Instituto Brasileiro de Geografia e Estatística, Rio de Janeiro(2000), http://www.ibge.gov.br, IBGE. Instituto Brasileiro de Geografia e Estatística. Retrieved June 3, 2002 from the World Wide Web: Censo DemográficoPehrsson, P.R., Haytowitz, D.B., Holden, J.M., Perry, C.R., Beckler, D.G., USDA's national food and nutrient analysis program: Food sampling (2000) J. Food Comp. Anal, 13, pp. 379-389Rodriguez-Amaya, D.B., Colugnati, F.A.B., Padovani, R.M., Lima, D.M., Galeazzi, M.A.M., Evaluation of the laboratory capability in Brazil (2001), Paper presented at the Fourth International Food Data Base Conference, BratislavaSouza, R.A.M., de Silva, R.O.P., Mandelli, C.S., Tasco, A.M.P., Comercialização hortícola: Análise de alguns setores do Mercado varejista de São Paulo (1998) Inform. Econ, 28, pp. 7-23(2001) Tabela Brasileira de Composição de Alimentos, , http://www.fcf.usp.br/tabe, USP. Universidade de São Paulo. Retrieved on June 4, 2002 from the World Wide We

A Propensity Score Approach In The Impact Evaluation On Scientific Production In Brazilian Biodiversity Research: The Biota Program

Evaluation has become a regular practice in the management of science, technology and innovation (ST&I) programs. Several methods have been developed to identify the results and impacts of programs of this kind. Most evaluations that adopt such an approach conclude that the interventions concerned, in this case ST&I programs, had a positive impact compared with the baseline, but do not control for any effects that might have improved the indicators even in the absence of intervention, such as improvements in the socio-economic context. The quasi-experimental approach therefore arises as an appropriate way to identify the real contributions of a given intervention. This paper describes and discusses the utilization of propensity score (PS) in quasi-experiments as a methodology to evaluate the impact on scientific production of research programs, presenting a case study of the BIOTA Program run by FAPESP, the State of São Paulo Research Foundation (Brazil). Fundamentals of quasi-experiments and causal inference are presented, stressing the need to control for biases due to lack of randomization, also a brief introduction to the PS estimation and weighting technique used to correct for observed bias. The application of the PS methodology is compared to the traditional multivariate analysis usually employed.101185107Abadie, A., Semiparametric difference-in-differences estimators (2005) The Review of Economic Studies, 72 (1), pp. 1-19Azoulay, P., Zivin, J.S.G., Manso, G., Incentives and creativity: Evidence from the academic life sciences (2011) RAND Journal of Economics, 42 (3), pp. 527-554Böhmer, S., von Ins, M., Different—Not just by label: Research-oriented academic careers in Germany (2009) Research Evaluation, 18 (3), pp. 177-184Carney, J., Smith, W., Parsad, A., Johnston, K., Millsap, M., (2008) Evaluation of the Faculty Early Career Development (CAREER) Program, , Abt Associates Inc, Bethesda, MD:Chen, S., Mu, R., Ravallion, M., Are there lasting impacts of aid to poor areas? Evidence for rural China. Policy Research Working Paper 4084 (2008) World Bank, , Washington: DCCrump, R.K., Hotz, J., Imbens, W., Mitinik, O.A., Dealing with limited overlap in estimation of average treatment effects (2009) Biometrika, 96 (1), pp. 187-199D’Agostino, R.B., Jr., Rubin, D.B., Estimating and using propensity scores with partially missing data (2000) Journal of the American Statistical Association, 95, pp. 749-759Ferraro, P.J., Counterfactual thinking and impact evaluation in environmental policy (2009) New Directions for Evaluation, 122 (Special Issue: Environmental Program and Policy Evaluation: Addressing Methodological Challenges)Ferraro, P.J., Pattanayak, S.K., Money for nothing? A call for empirical evaluation of biodiversity conservation investments (2006) PLoS Biology, 4 (4), p. e105Friedman, J.H., Greedy function approximation: A gradient boosting machine (2001) Annals of Statistics, 29 (5), pp. 1189-1232Frondel, M., Schmidt, C.M., Evaluating environmental programs: The perspective of modern evaluation research (2005) Ecological Economics, 55 (4), pp. 515-526Hastie, T., Tibshirani, R., Friedman, J., (2001) The elements of statistical learning, , Springer, New York:Hirano, K., Imbens, G.W., Ridder, G., Efficient estimation of average treatment effects using the estimated propensity score (2003) Econometrica, 71, pp. 1161-1189Hornbostel, S., Böhmer, S., Klingsporn, B., Neufeld, J., von Ins, M., Funding of young scientist and scientific excellence (2009) Scientometrics, 79 (1), pp. 171-190Hoshino, T., A Bayesian propensity score adjustment for latent variable modeling and MCMC algorithm (2008) Computational Statistics & Data Analysis, 52 (3), pp. 1413-1429Ihaka, R., Gentleman, R., R: A language for data analysis and graphics (1996) Journal of Computational and Graphical Statistics, 5 (3), pp. 299-314Inglesi-Lotz, R., Pouris, A., Scientometric impact assessment of a research policy instrument: The case of rating researchers on scientific outputs in South Africa (2011) Scientometrics, 88 (3), pp. 747-760Jacob, B.A., Lefgren, L., The impact of NIH postdoctoral training grants on scientific productivity (2011) Research Policy, 40, pp. 864-874Jalan, J., Ravallion, M., Estimating the benefit incidence of an antipoverty program by propensity-score matching (2003) Journal of Business and Economic Statistics, 21 (1), pp. 19-30Joly, C.A., Rodrigues, R.R., Metzger, J.P., Haddad, C.F.B., Verdade, L.M., Oliveira, M.C., Bolazani, V.S., Biodiversity conservation research, training, and policy in São Paulo (2010) Science, 328, pp. 1358-1359Khandker, S.R., Koolwal, G.B., Samad, H.A., (2009) Handbook on impact evaluation quantitative methods and practices, , World Bank, Washington, DC:Liu, X., Zhang, L., Hong, S., Global biodiversity research during 1900–2009: A bibliometric analysis (2011) Biodiversity Conservation, 20 (807), pp. 807-826Lynch, L., Gray, W., Geoghegan, J., Are farmland preservation program easement restrictions capitalized into farmland prices? What can a propensity score matching analysis tell us? (2007) Applied Economic Perspectives and Policy, 29 (3), pp. 502-509McCarey, D., Ridgeway, G., Morral, A., Propensity score estimation with boosted regression for evaluating adolescent substance abuse treatment (2004) Psychological Methods, 9 (4), pp. 403-425Mena-Chalco, J.P., Cesar, R.M., Jr., scriptLattes: An open-source knowledge extraction system from the Lattes platform (2009) Journal of the Brazilian Computer Society, 15 (4), pp. 31-39Mutz, R., Daniel, H.-D., The generalized propensity score methodology for estimating unbiased journal impact factors (2012) Scientometrics, 92 (2), pp. 377-390Nelder, J., Wedderburn, R., Generalized linear models (1972) Journal of the Royal Statistical Society. Series A (General), 135 (3), pp. 370-384Ravallion, M., Evaluating anti-poverty programs (2008) Handbook of development economics, pp. 3787-3846. , Schultz TP, Strauss J, (eds), North-Holland, Amsterdam:Ridgeway, G., The state of boosting (1999) Computing Science and Statistics, 31, pp. 172-181http://cran.r-project.org/doc/packages/gbm.pdf, Ridgeway, G. (2005). GBM 1.5 package manualRidgeway, G., Assessing the effect of race bias in post-traffic stop outcomes using propensity scores (2006) Journal of Quantitative Criminology, 22 (1), pp. 1-29Ridgeway, G., McCaffrey, D., Morral, A., (2006) twang, , http://cran.r-project.org/web/packages/twang/index.html, Toolkit for weighting and analysis of nonequivalent groups, Software for using matching methods in R:Robins, J.M., Rotnitzky, A., Comment on “inference for semiparametric models: Some questions and an answer” by P. Bickel and J. Kwon (2001) Statistica Sinica, 11, pp. 920-936Rodrigues, R.R., Bonomi, V.L.R., (2008) Diretrizes para conservação e restauração da biodiversidade no estado de São Paulo, , (eds), SMA-SP & FAPESP, São Paulo:Rosenbaum, P., Rubin, D., The central role of the propensity score in observational studies for causal effects (1983) Biometrika, 70 (1), pp. 41-55Rubin, D., Estimating causal effects of treatments in randomized and nonrandomized studies (1974) Journal of Educational Psychology, 66 (5), pp. 688-701Rubin, D., Using propensity scores to help design observational studies: Application to the tobacco litigation (2001) Health Services and Outcomes Research Methodology, 2, pp. 169-188Salles-Filho, S., Avila, F.D., Sepulveda, J., Colugnati, F.A.B., Multidimensional assessment of technology and innovation programs: The impact evaluation of INCAGRO-Peru (2010) Research Evaluation, 19 (5), pp. 361-372Salles-Filho, S., Bonacelli, M.B.M., Zackiewicz, M., Castro, P.F.D., Bin, A., (2007). Development and application of a methodology for evaluating S,T&I Programs: The decomposition method. In Workshop Internacional sobre Inovações Metodológicas na Avaliação de Impacto da pesquisa Agropecuária, 2008, Brasília, XII Seminário Latino-Iberoamericano de Gestión Tecnológica - ALTEC 2007 (2007) Buenos Aires, 1, pp. 2-6Salles-Filho, S., Castro, P. F. D., Bonacelli, M. B. M., Zeitoum, C., & Sá, F. P. (2010b). Indicators for evaluation of results and impacts of research program in conservation and sustainable use of biodiversity, the case of BIOTA/FAPESP. In International conference: Getting post 2010 biodiversity targets right, Bragança PaulistaStuart, E.A., Matching methods for causal inference: A review and a look forward (2010) Statistical Science, 25 (1), pp. 1-21White, H., A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity (1980) Econometrica, 48, pp. 817-83