A guide to monitoring and evaluating policy influence
Harry Jones, ODI (2011)
https://odi.org/en/publications/a-guide-to-monitoring-and-evaluating-policy-influence/
Five key dimensions of possible policy impact:
1⃣ attitudinal change
2⃣ discursive commitments
3⃣ procedural change
4⃣ policy content
5⃣ behaviour change
The production and use of monitoring & evaluation (M&E) information during and after an intervention is generally seen as
🔷a central plank in systems for reporting and accountability
🔷in demonstrating performance
🔷for learning from experience and improving future work
Challenges:
🔴conceptual and technical (difficulty in establishing causality, ‘attribution problem’)
🔴nature of policy influencing work (Policy changes tend to occur over long timeframes)
🔴 practical problems that constrain the production and use of knowledge
Theory of Change (Toc): model of how the policy influencing activities are envisaged to result in the desired changes in policy or in people’s lives
🔷 Improving projects, sense-making
🔷 Accountability &reporting (dimensions & intermediate outcomes to measure project influence)
Developing Theory of Change (Toc)
🛠️ Causal chain (➡️➡️➡️)
🛠️ Dimensions of influence (help create the conditions for policy change)
🛠️ Actor-centred theories
ToC
1⃣Start with a picture of what drives change in the ‘target'
* Large leaps
* Coalition
* Policy Windows
* Messaging & frameworks
* Power politics
* Grassroots
2⃣Link into this the way(s) that the project aims to influence the target.
What to measure?
Evidence & advice
Research activities lead to outputs (tangible goods and services, e.g. briefs, events)➡️
‘uptake’, direct responses to the research (such as using it or quoting it)➡️
Outcomes or impact; changes in behaviour and in people’s lives
Evaluating uptake and use
* Uptake logs: this is simply a log (perhaps an email
inbox or database) where comments, anecdotes
and examples of ‘uptake’ or influence are recorded.
* New areas for citation analysis
* User surveys
"Understanding the link between the information presented in the media and the effect on the target audience is an extremely difficult area"
Lobbying: crucial to shape the course of policy
there is little literature on M&E for this interaction
strong incentives against the sharing of good practice in this area
‘This is not rocket science’, quite straightforward tools
🔷Collecting info is time-consuming; ensure muliple uses
🔷Develop Theory of Change as early as possible
🔷Collect relevant data opportunistically or at periodic
intervals throughout the policy influencing work
Oorspronkelijk getweet door Wilte Zijlstra (@wilte) op 24 maart 2021.
wilte 