Friedmann, M.

CGSpace

S e p t e m b e r  2 0 1 7RTB Quality of Research for Development (QR4D): current thinking and future perspectives MICHAEL FRIEDMANN •ISC MEETING, DAR ES SALAAM, TANZANIAOutline• Summary of ISPC deliberations on framework for QR4D• QR4D in RTB Research Program• DiscussionISPC Framework for QR4DQR4D-ISPC status• ISPC to facilitate System-wide agreement on the nature and assessment of quality of science• broadened to Quality of Research for Development (QR4D)• CGIAR and donors concerned with both quality of programs and likelihood of delivery.ISPC to develop a frame of reference for QR4D across the CGIAR that gives guidance at different levels but without burdening the Centers and CRPs; and, to use that toolQR4D viewed as an integrated whole of four key elements: relevance, credibility, legitimacy and effectiveness (Belcher et al., 2016)Belcher, B.M., et al. (2016). Defining and assessing research quality in a transdisciplinary context. Res. Eval. 25, 1-17.QR4D- Relevance“importance, significance and usefulness of the research objectives, processes and findings to the problem context and to society, and CGIAR’s comparative advantage to address the problems”Consideration Indicator (RTB) Challengestrategic stakeholder engagementworkshop reports, survey data and analysis need to go beyond counting and measure quality of engagementimpact pathways (IPs) ProposalUpdates in POWB and Annual Reportsneed to follow progression along IPoriginal, socially relevant research aligned to national and regional priorities and CGIAR SRF and SDGsRTB proposal and annual reports;  ex-ante study reports and publicationswhite papers, discussion papersnational and regional priorities lack detailQR4D- Credibility“refers to the quality of science and implies that the research findings are robust and sources of knowledge are dependable”Consideration Indicator (RTB) Challengeimportance of good scientific practice such as peer reviewpeer-reviewed publications: ISI-journals, books, book chapters need to go beyond counting and establish doable guidelines for rankingclear demonstration of: adequacy of the data; methods used to procure the data; data clearly presented with logical interpretation of findingsDatasets strategies to evaluate adequacy of data-beyond  publicationquality of breeding pipelines (methods used, breeding populations etc.)Breeding Program Assessment Tool- BPAT; center evaluation of breeding programsaddress context;extend to which partners;QR4D- Legitimacy“the research process is fair and ethical and perceived as such”Consideration Indicator (RTB) Challengeethical and fair representation of all involved; genuine recognition of partners’ contributionsethics boards-reports and assessments; partners included in publications;letters of partner support  counting partners included  might not capture all involvedconsideration of interests and perspectives of intended users transparency/lack of conflict of interestethics boards-reports and assessments; surveys of partner perceptions of collaborations need to incorporate into research plans and allocate resourcesrecognition of responsibilities that go with public fundingcontracts with partners,open access guidelinesResponsibilities set; feasibility of monitoring and enforcing with partnersQR4D- Effectiveness“research generates knowledge, products and services that stimulate actions that address the problem and contribute to solutions and innovations”Consideration Indicator (RTB) Challengedynamic theories of change with assumptions for how change happensPublications (ranking & citations), training materials, datasets & tools, genetic material; reports (impact assessments); survey data and analysis go beyond counting and measure usage and adoption; need allocation of resources and time; ranking non-ISI publicationsnegative unintended consequenceshow to evaluate negative consequences; indicators?effective communication communication products changes in policy (government documents, media releases)measure usage of communication materials;measure changes in behaviorleadership & capacity developmentnumber of trainings and trainees; follow up on trainees resource allocation, cannot capture all typesQR4D in RTB Research ProgramRelevance: how is RTB doing?• portfolio• flagships/clusters with connections between discovery research and product delivery• impact pathway aligned to SDGs and CGIAR SRFs• well aligned with priority setting study• Phase II Proposal scores among best• dedicated cluster for priority setting (CC5.1) with consultative process across RTB flagships for  impact study selection• Emphasis on end-user preferences and demand-driven breeding• Linkages to other CRPs and PlatformsD.Dufour CIRADCredibility: how is RTB doing?• enhanced communication strategies for program-stronger culture for Science Quality• linkages to Big Data Platform• Ensure all CGIAR data is FAIR (Findable, Accessible, Interoperable, and Reusable)• Generate data sharing platforms and protocols• Facilitate discoverability of data• Breeding Community of Practice Cluster (BCoP)• More effective breeding programs (implement BPAT, Stagegate pipeline management and outreach to partner breedersBCoP meeting, June 2017Legitimacy: how is RTB doing?• RBM pilots-stakeholders involved in improved Theories of Change for selected clusters• MEL system implemented-allowing better tracking of partner involvement• RTB annual meeting togetherwith local partnersEffectiveness: how is RTB doing?• product portfolio addresses improved variety development, seed systems, value chains and major biotic/abiotic constraints with extensive network of local partners• enhanced communication strategies for program• promotion of scaling strategies for RTB products (Scaling Innovation Fund)• impact studies important part of portfolio• increase in ISI publications and trainees (short term)M.Friedmann/ RTB0501001502012 2013 2014 2015 2016ISI pubs0200004000060000800002012 2013 2014 2015 2016TraineesQR4D Indicators & StrategiesCategory Indicator Who Oversight PeriodicityRelevance Phase II Proposal RTB ISPC 6 yrAnnual Report, POWB RTB SO AnnualProgram workplans Centers BoTs Annualex-ante and foresight study reportsCenters BoTs AnnualCredibility ISI publications and IF Centers RTB Mgmt AnnualH-score senior staff Centers RTB Mgmt 3 or 6 yr"relevant" non-ISI pubs Centers RTB Mgmt Annual"quality" datasets Centers RTB Mgmt Annual"quality" of research pipeline managementCenters BoTs & RTB MgmtAnnualO. Ortiz, DDG Research, CIPQR4D Indicators & StrategiesCategory Indicator Who Oversight PeriodicityLegitimacy ethics committees Centers RTB Mgmt 6 yrpolicies in place Centers RTB Mgmt 6 yr# open access publications Centers SO & RTB MgmtAnnual?# publications with partner coauthorsCenters RTB Mgmt AnnualEffectiveness outcome indicators RTB SO & RTB MgmtAnnual or longer?impact studies RTB SO & RTB MgmtAnnualTools & technologies releasedCenters RTB mgmt Annual or longer?usage stats on communication productsRTB SO & RTB MgmtAnnual# of CapDev activities Centers RTB Mgmt Annual# of trainees Centers RTB Mgmt Annualmid-term review of CRP RTB IEA 3 or 4 yrO. Ortiz, DDG Research, CIPQR4D concluding thoughts• Considering annual peer review of RTB Flagship programs (1 or 2 per year, at RTB Annual meeting), suggested by IEA• Issue of selecting and using indicators- they should:• be measurable (as simple as possible)• be evaluated fairly in a comparable scale• be balanced between their effectiveness and ease of measuringProgram must continue to dedicate (and increase) resources to do follow-up studies, or have these be part of projects to generate evidence of impactTHANK   YOU

RTB quality of research for development (QR4D): current thinking and future perspectives

https://cgspace.cgiar.org/bitstream/handle/10568/88105/1.4_Friedmann_QR4D_17_09_04.pdf?sequence=1&isAllowed=y

RTB quality of research for development (QR4D): current thinking and future perspectives

Abstract

Similar works

Full text

Available Versions

CGSpace