The IDS Accountable Grant is a large and complex four-year grant working across seven Policy Themes, 20 sub-themes, 68 activity domains and over 300 Level 2 Outputs. 

The scale and diversity of activity under the grant, the dynamic and unpredictable policy environment in which it operates, the number of people and partners involved, together with the imperative to deliver within tight deadlines, highlights the need for a simple yet robust system for tracking planned and completed outputs.

This paper reports on the Monitoring and Evaluation of Year 1 of the Accountable Grant, including progress to date, the development of CRM and future developments.DFI

Befani, Barbara

Munslow, Tamlyn

Sumberg, James

English

IDS OpenDocs

IDSEVIDENCE REPORTNo 6
Policy Anticipation, Response and Evaluation
Monitoring and Evaluation Report: Year One
Tamlyn Munslow, Barbara Befani and James Sumberg
June 2013
MONITORING AND EVALUATION REPORT: YEAR ONE
Tamlyn Munslow (Research Officer), Barbara Befani (Research Fellow) and James Sumberg 
(Research Fellow and Grant Director)
June 2013
This is an Open Access publication distributed under the terms of the Creative Commons Attribution License, which permits unrestricted 
use, distribution, and reproduction in any medium, provided the original author and source are clearly credited.
First published by the Institute of Development Studies in June 2013
© Institute of Development Studies 2013
IDS is a charitable company limited by guarantee and registered in England (No. 877338).
The IDS programme on Strengthening Evidence-based Policy works across seven key themes. Each theme 
works with partner institutions to co-construct policy-relevant knowledge and engage in policy-influencing 
processes. This material has been developed under the Policy Anticipation, Response and Evaluation theme.
This development of this material has been led by the Institute of Development Studies who holds the 
copyright.  
The material has been funded by UK aid from the UK Government, however the views expressed do not  
necessarily reflect the UK Government’s official policies.
AG Level 2 Output ID: 347
1 
 
Contents 
1 Introduction and Background 2 
 
2 Progress to date 3 
2.1 CRM 3 
 
3 Future developments 6 
3.1 Sentinel outputs 6 
3.2   Accountability and learning 6 
 
Figures 
Figure 1             Level 2 Outputs in CRM Year 1                                                                      3 
Figure 2             Information held on Level 2 Output                                                                4 
 
Tables 
Table 1              Level 2 Outputs completed Year 1                                                                 2 
Table 2              Countries in which the Level 2 Outputs are active                                         4 
Table 3              Example Gantt Chart, Theme 1, Year 2                                                         5 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2 
 
1 Introduction and Background 
 
The IDS AG is a large and complex four-year grant, involving seven Outputs (also called 
Policy Themes or Themes); 20 Sub-themes; 68 Activity Domains; and over 300 Level 2 
Outputs (or ‘policy products’). Level 2 Outputs include papers, reviews, reports, briefs, 
events, guidelines and other tools. Sixty-eight Level 2 Outputs were completed in Year 1 
(Table 1). 
 
Table 1  Level 2 Outputs completed Year 1  
 
 
A diverse team of staff work across the institute to ensure the efficient production of these 
outputs. Working within the seven Policy Themes, these include theme convenors, sub-
theme convenors, research fellows, knowledge services staff, communications staff and 
administrative support staff.  
 
The scale and diversity of activity under the grant, the dynamic and unpredictable policy 
environment in which it operates, the number of people and partners involved, together with 
the imperative to deliver within tight deadlines, highlights the need for a simple yet robust 
system for tracking planned and completed outputs. This system must provide the framework 
for analysing the level of achievement against a logical framework that focuses upon 
availability (count available), accessibility (number of downloads) and perceived quality 
(opinion survey) of the Level 2 Outputs.   
 
 
 
 
 
 
 
 
 
Theme 1 Theme 2 Theme 3 Theme 4 Theme 5 Theme 6 Theme 7 Total 
Level 2 Output Type
Agenda 1 1
Brief 1 6 1 1 6 3 18
Bulletin 0
Database 1 1
Event 4 4
Fellowship 4 4
Grand Total 0
Guide 0
Guidelines 1 1
Paper 2 6 8
Recommendations 0
Report 2 4 1 1 3 6 7 24
Review 2 1 1 1 5
Strategy 1 1
Tool 0
Toolkit 0
Typology 0
Total 6 11 5 3 20 7 16 68
3 
 
2 Progress to date  
2.1 CRM 
 
In recent years IDS has invested heavily in Microsoft Dynamics CRM software. This software 
integrates contracts, contacts and process management functions. It also enables links to the 
institute’s budgeting and financial management software. While CRM has all the functionality 
required for large-scale programme management, previous to the AG it had not been used in 
this way at IDS. 
 
Therefore, after initially exploring other alternatives, a decision was taken to build the AG 
M&E system around CRM, and since early September 2012 we have worked closely with 
David Beaven (Head of IDS Computer and Technical Services) and Harvey Coates (Finance 
Department) to do this. The remainder of this report outlines progress to date and future 
developments. Others involved in this work include Tamlyn Munslow (Research Officer, 25 
per cent of time spent on M&E), Barbara Befani (Impact Fellow), Fran Seballos (Partnerships 
Officer), Robin Coleman (Communications), Nason Bimbe (Library/OpenDocs) and James 
Sumberg (Grant Director). 
 
To date, the basic structure of the CRM AG database has been created, populated with 
information about all Level 2 Outputs and cross checked to reflect completed Year 1 activity 
and planned Year 2 activities (Figure 1). Each Level 2 Output has a unique ID number 
(Figure 2). Opportunities for further development of the system exist and we are currently 
exploring these. 
 
  
Figure 1  Level 2 Outputs in CRM Year 1  
 
4 
 
Figure 2  Information held on Level 2 Output 
 
CRM can be used to generate queries as well as access and export information for further 
analysis (e.g. Table 2). 
 
Table 2  Countries in which the Level 2 Outputs are active  
 
 
 
Country Theme 1 Theme 2 Theme 3 Theme 4 Theme 5 Theme 6 Total Region
Egypt 1 1 tbc 1 3 N Africa
Ethiopia 1 1 Horn of Africa
Kenya 1 1 1 3 E Africa
Uganda 1 1 2 E Africa
Tanzania 1 1 E Africa
Malawi 1 1 S Africa
Zambia 1 1 S Africa
Mozambique 1 1 S Africa
South Africa 1 1 1 3 S Africa
Ghana 1 1 W Africa
Nigeria 1 1 W Africa
Sierra Leone 1 1 W Africa
India 1 1 1 1 1 5 Asia
Bangladesh 1 1 2 Asia
Nepal 1 1 2 Asia
Indonesia 1 1 Asia
Cambodia 1 1 Asia
Philippines 1 1 Asia
China 1 1 Asia
Russia 1 1 CIS
Brazil 1 1 S America
Yemen 1 1 Middle East
10 4 7 0 7 7 35
5 
 
CRM has been used to produce Gantt charts for each theme showing the delivery of Level 2 
Outputs over the grant period (Table 4). ‘Planned start date’ reflects the date in which Level 2 
Outputs become ‘live’, and ‘Planned end date’ reflects the submission of the final output for 
publication. The Year 2 planned end dates have been staggered so as to facilitate 
dissemination and maximise the potential impact of the Level 2 Outputs.  
 
Table 3  Example Gantt chart Theme 1, Year 2  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Level 2 
output ID Output Name Level 2 Output type
Planned 
start date
Planned 
End date Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13
1 Mapping paper 1 (Ghana) Paper 01/07/2012 31/03/2013 1 1 1 1 1 1 1 1 1
2 Policy guidelines 1 (Ghana) Guidelines 01/07/2012 30/06/2013 1 1 1 1 1 1 1 1 1 1 1
3 Mapping paper 2 (Nigeria) Paper 01/07/2012 30/09/2013 1 1 1 1 1 1 1 1 1 1 1
4 Policy guidelines 2 (Nigeria) Guidelines 01/07/2012 31/12/2013 1 1 1 1 1 1 1 1 1 1 1
5 Mapping paper 3 Paper 01/04/2013 31/03/2014 1 1
6 Policy guidelines 3 Guidelines 01/04/2014 30/06/2014
7 Business case study 1 Report 01/04/2013 30/12/2013 1 1
8 Business case brief 1 InFocus Policy Brief 01/12/2013 30/12/2013
9 Business case study 2 Report 01/04/2013 30/12/2013 1 1
10 Business case brief 2 InFocus Policy Brief 01/12/2013 30/12/2013
11 Business case study 3 Report 01/04/2014 30/09/2014
12 Business case brief 3 InFocus Policy Brief 01/09/2014 30/09/2014
13 Business case study 4 Report 01/04/2014 30/12/2014
14 Business case brief 4 InFocus Policy Brief 01/12/2014 30/12/2014
15 Tool for value chain ranking Tool 01/04/2014 30/09/2014
16 In-depth case assessment Report 01/04/2014 31/12/2015
17 Synthesis report of policy guidelines Report 01/04/2015 31/12/2015
18 Learning event Report 01/04/2015 30/09/2015
6 
 
3 Future developments  
3.1 Sentinel outputs 
 
An important part of the M&E function is to generate information and understanding 
concerning the quality and impact of the 300+ Level 2 Outputs. To help in this regard we 
have developed the notion of Sentinel Outputs, which refers to the relatively limited 
selection of what are the key or critical outputs per theme and per year. The logic is that by 
monitoring and understanding how these sentinel outputs impact on policy processes and 
outcomes, we will have a better picture of grant impact overall than if we were to try to 
monitor all outputs.  
 
Theme Convenors will work with Knowledge Services (KS) and Communications staff to 
identify Sentinel Outputs. For Year 2 we are re-organising how Knowledge Services and 
Communications inputs are made and managed. For KS, the idea is to concentrate efforts on 
the Sentinel Outputs. In addition to both greater clarity about roles and tighter coordination 
with Communications, it is important that KS are involved in thinking about potential uptake 
pathways from the very beginning.  
 
Each output should have a ‘dissemination plan’. This will be particularly important for 
Sentinel Outputs, and we anticipate that the new arrangements with KS and 
Communications will allow significant progress along these lines. One idea is that we 
developed is a kind of generic dissemination plan for each type of Level 2 Output (e.g. 
Evidence Report, IDS Policy Briefing, Rapid Response Briefing, etc). Such discussions will 
need to anticipate a protocol for gathering, storing, updating and analysing download data on 
AG sentinel outputs as well as to consider how these will be stored in the CRM database 
(Figure 3 and Table 4).  
 
From an M&E perspective, the focus is upon generating, capturing and analysing feedback 
from external users with regards to these Sentinel Outputs. The M&E system, together with 
the notion of Sentinel Outputs, can be used to generate concrete, substantiated findings on 
the Level 2 Outputs as well as offer narrative explanations as to the role they have played in 
policy processes.  
 
3.2 Accountability and learning  
 
CRM assists efficient reporting, production of timeframes and monitoring of Level 2 Outputs. 
In this sense it enhances accountability to the donor. Additionally, CRM has the potential to 
increase accountability to Level 2 Output users, since indicators are geared towards 
generating feedback from the end user. As we progress, lessons will be shared as much 
about this two-directional approach as reporting on Level 2 Outputs.  
 
 
 
Brighton BN1 9RE 
T +44 (0)1273 606261 
F +44 (0)1273 621202 
E ids@ids.ac.uk
www.ids.ac.uk
IDS_Master Logo


Monitoring and Evaluation Report: Year One

https://opendocs.ids.ac.uk/opendocs/bitstream/20.500.12413/2672/1/ER6%20Final%20Online.pdf

Monitoring and Evaluation Report: Year One

Abstract

Similar works

Full text

Available Versions

IDS OpenDocs