It is extraordinary how much resistance there is within development agencies to rigorous evaluation of development interventions.
I almost never disagree with Owen, but I think he’s wrong here. There’s nothing extraordinary about resistance to evaluations within development agencies. The explanations are all prosaic:
- Although Econ 101 tells us that sunk costs should be discounted, it can be surprisingly hard to do this.
- When something’s going poorly it’s very tempting to push on just a little further in the hope that things get better. (Call this the ‘how we got really lost in the forest’ theory.)
- Not all development agency staff understand evaluations.
- Most development agency staff are overworked. Changing a project takes time. Even simply getting to the bottom of the ramifications of a negative evaluation takes time (remember the key question is not just did it work? but also if not why not?)
- Evaluations are not wholly reliable. Even the ‘gold standard’ of RCTs are fallible. And most interventions can’t be studied with RCTs – other types of evaluation are required. And other types of evaluation are often less reliable. Moreover, any evaluation of any form is only as good as the contractor you get to do it. None of this means evaluations should be discounted. Of course not. However, the fact that they aren’t guaranteed conduits of the truth is an issue. An issue of its own accord and also because it contributes to point 6 below.
- A combination of points 3 and 5 – not all development agency staff trust evaluations.
- The discontinuation of any aid project/programme save for the very worst will usually involve harming someone in need. If your evaluation shows that 90% of money to the Ministry of Health is lost to corruption there’s still 10% remaining helping people. And it’s hard to stop something when you know people will be hurt because of it, even if you also know that more people can be helped if money is spent elsewhere.
- Some aid is given for political reasons. Add an evaluation to this and you get <42 point flaming capital letters> politically sensitive results</42 point flaming capital letters>
To be perfectly clear here: these are explanations not excuses. I think a crucial component of success in development work is development agencies embracing good evaluations and embracing the learning and change that comes from them. And I’ve listed the points above simply because those who wish to see this happen need to think actively about overcoming them.
I’ve also listed the points in increasing order (as I see it) of intractability. For what it’s worth, point 8 really deserves a category of its own. Also, worth noting that: not all aid is politically motivated, even from the worst offending donor countries; from the best donor countries next to no aid is. Also, my belief is that politico-strategic aid is less of an issue globally than it used to be. To the extent that it remains an issue, it’s one that needs to be taking up with politicians and the voting public. There’s no use blaming aid agencies.