Via Development Impact, Tim Hartford on a huge community development programme, and a equally gigantic attempt to evaluate it:
Tuungane (“Let’s Unite”), a programme funded by the UK’s Department for International Development to the tune of £90m. The first £30m, Tuungane I, funded classrooms, clinics and other investments in 1,250 villages with a total population of almost 1.8 million people.
If the project is ambitious, what’s really fascinating is that Dfid has tried to evaluate Tuungane I rigorously, using a randomised controlled trial. Villages were enrolled in Tuungane through a public lottery. With 1.8 million people in the treatment group and a large control group, such an evaluation would be challenging to conduct in a rich country. In the DRC, where enumerators risk death and must wade through water up to their necks to reach the villages they are surveying, it’s mind-blowing.
Tuungane is a community-driven development project, or CDD. CDD is fashionable in development circles these days. The idea is that communities receive cash from donor agencies on condition that they come up with appropriate institutions for deciding how to spend the money. CDD projects will often insist on standards of transparency, democracy, or gender balance. The hope is that not only will the money be spent well, but that it may also catalyse accountable local institutions.
But does CDD deliver on this? The Tuungane I evaluation has now been completed by a team from Columbia University. The evaluation was based on a second project, Rapid, which was separate from Tuungane. Rapid handed out cash with few strings attached to Tuungane and non-Tuungane villages. The evaluators observed what happened: was the money embezzled, or well spent, and how were the decisions reached? This was the acid test of whether Tuungane had helped to promote effective village institutions.
The good news is that the Tuungane project was well executed and the money typically arrived where it was supposed to. In the DRC this is no small achievement.
Yet the results fall short of the CDD hype. The evaluators found that the Rapid cash grants were reasonably well spent whether or not a village had previously been exposed to three years of community building through Tuungane. Local institutions were more accountable than one might expect, but there was no sign that Tuungane could take any credit for this. Neither did Tuungane projects display any economic returns – although arguably there was too little time between spending the cash and evaluating the results for such returns to become apparent.
It’s a shame that the results haven’t matched the expectations of the CDD optimists, but it’s hardly a surprise. Tuungane is large in absolute terms, but it’s just a few pounds per person. It would be odd to expect miracles. The safe arrival of cash in the hands of very poor people is surely worth celebrating.
Equally interesting to me would be (a) whether there was meaningful variation in results depending on conditions in recipient villages (I’ve yet to read the paper) — for example, did the project perform less well in linguistically or tribally heterogeneous villages than in homogeneous ones (b) what might be revealed by process tracing about why various aspects of the project did and didn’t work.
The other thing I’m curious about, having witnessed the travails of a large data gathering exercise in the Pacific is whether you can really trust your survey teams to actually go and gather the data, rather than just claiming that they did. Especially if “enumerators risk death and must wade through water up to their necks to reach the villages they are surveying.” I’m not saying that inaccurate data creation is necessarily an issue in this particular study, I’m just saying it is where I work, and I really wish it wasn’t.