The purpose of this manuscript is to present an on-going experience of impact evaluation applied to a recently created S&T funding agency. The Institute Serrapilheira (ISP) is a private nonprofit organization created to fund new and challenging themes of research. Serrapilheira brings innovation to the Brazilian scenario of science funding in at least three directions. First, ISP is the first family-philanthropically funded organization totally dedicated to support science in Brazil. Secondly, as a private institution, it can provide researchers with greater flexibility in terms of resource allocation comparing to public funding agencies. The 2018 estimated budget is US$ 4,5 million. Thirdly, in its first call for proposals launched in mid-2017, ISP asked young researchers to present proposals out of the “Normal Science”. The definition of the impact evaluation model of the first call of Serrapilheira was based on both the institutional model proposed by the organization – and its main purposes – as well as on the particular goals of the first call. As a first step, these objectives were discussed with Serrapilheira’s staff. The second step consisted of detecting evaluation hypotheses, themes and indicators based on the discussion of the model’s objectives. As a result, six evaluation themes were defined – one oriented to characterizing the grantees both in terms of diversity and academic training and the other five oriented to measure impacts: (i) professional trajectory, (ii) scientific and technology production, (iii) night science, (iv) insertion and prominence and (v) research culture. Traditional and widely accepted indicators of scientific and technological impact were used, even for dialogue with communities and scientific institutions and terms of comparison. Nevertheless, the main challenge was to identify indicators that allow the measurement of variables that directly or indirectly approach the particularities of the research-funding model. As a first remark, we understand that the particularities of the new research-funding model proposed by ISP asks for a new evaluation framework, more oriented to alternative indicators and to a broad perspective about outputs, outcomes and impacts in a more diverse and multifaceted research system. In this current hands-on impact-study a methodology is being built and simultaneously applied to a concrete case. This experience may contribute to the field of impact evaluation to the extent it proposes not only alternative indicators to be mixed with traditional ones, but also because it has a longitudinal perspective of following grantees since the inception (with the advantage of starting with a baseline) till two years after projects end, performing a 5-year long evaluation. Another possible contribution refers to the understanding about the ways funding agencies are evaluating the outcomes and impacts of their initiatives
