The effectiveness of measuring influence (GSDRC Helpdesk Research Report 911)
Reviews evaluations projects that seek to influence policy or opinion using advocacy, lobbying, negotiation and knowledge uptake
Abstract
Query
Review a selection of evaluations of projects that sought to influence policy or opinion using advocacy, lobbying, negotiation and/or knowledge uptake. Note if they were able to evaluate the projects effectively, whether problems arose in conducting the evaluations, and what lessons can be learnt.
Key findings
- There are four main types of influencing (evidence and advice, advocacy and campaigning, lobbying and negotiating, and soft power) which each require a different method of evaluation. Evaluators therefore need to use multiple methods to critically analyse and triangulate findings.
- A large body of research states that the tools available for measuring influence do not allow for rigor, nor do they allow for replicable findings. There is difficulty in comparing findings between organisations.
- A preference for quantitative metrics tends to lead to a focus on activities and outputs and to attempts to quantify qualitative information – methods which, it is argued, do not robustly document impact (Coe & Majot, 2013).
- Some experts argue that the present tools are adequate but that there is a failure of applying the methods rigorously.
- Organisations that have found a process to measure their influencing work tend to focus their monitoring and learning on success stories, having a clear theory of change and finding anecdotal evidence to assess contribution.
- There are few examples of evaluations that reflect on the effectiveness of the review.
Citation
Tsui, J. The effectiveness of measuring influence (GSDRC Helpdesk Research Report 911). Governance and Social Development Resource Centre, University of Birmingham, Birmingham, UK (2013) 10 pp.
Links