There have been growing demands for greater independent evaluation of foreign aid for at least half a decade now. As William Easterly argued as far back as 2006:
We need independent evaluation of foreign aid. It’s amazing that we’ve gone a half century without this. . . . [Truly independent evaluation of aid would] give feedback to see which interventions are working and give incentives to aid staff to find things that work.
The Center for Global Development summarized the need in its report When Will We Ever Learn? Improving Lives Through Impact Evaluation:
Impact evaluations do not have to be conducted in-house. Indeed, their integrity, credibility, and quality is enhanced if they are external and independent.
It is with this understanding that I read the recent Brookings report on aid to fragile states.
Although Laurence Chandy, the author, does an excellent job summarizing the history of aid to these countries and makes a good case why the effectiveness of foreign assistance to them should have improved over the past decade, he fails to prove his central claim that:
The development community is a lot wiser . . . there is mounting evidence that aid to fragile states can work.
Ten years ago a World Bank established taskforce that examined how the development community approached “Low-Income Countries Under Stress” (LICUS) concluded that aid “does not work well in these environments”, “may even be counterproductive” and that “donors are impotent against poverty” when operating in these countries.
The Brookings report argues that these conclusions may never have been true and are in any case invalid today:
Macroeconomic evidence cited by the taskforce turned out to be less than robust . . . The emergence of new aid models that are better suited to fragile states has gone a long way to address the political economy constraints that undermined earlier donor-partner relationships. . . . Finally, and perhaps most significantly, the performance ratings of aid projects in fragile states have undergone a remarkable turnaround.
All of this may be true, but it is hard to know from the information presented, which provides no systematic analysis and is based on almost no independent evaluations.
The conclusion that there has been a “remarkable turnaround” in the performance ratings of aid projects—supposedly the most significant reason to be optimistic—is based on only two sources of information. World Bank data shows that the gap between its projects’ performance in fragile and stable settings has been substantially reduced (with measures for both improving) over the past decade. There is no independent evaluation of this finding. A single report on the Global Fund to Fight AIDS, Tuberculosis and Malaria is cited as representative of the rest of the donor community.
This seems rather tenuous to say the least.
Indeed, the author himself asks:
Can the improvement in project ratings really be believed? A skeptic would likely question the veracity of project evaluations . . . One might also suspect whether the improvement in project ratings reflects donors’ capacity to bring about change or rather donors’ realization of their limited capacity and corresponding adjustments to project design. . . . This we could call the “soft bigotry of low expectations” effect.
Given this hesitation and the paltry amount of data collected, it is hard to understand how he can go on in the very next paragraph to argue that:
These claims . . . cannot invalidate the core inference that the performance of aid in fragile states has improved. Today’s aid projects are informed by a decade of experience, as evidenced by the emergence of new models of donor engagement* that adjust to the political economy challenges of fragile settings. This learning has reaped dividends.
Again, these conclusions may be correct, but it is hard to know with the information presented. Instead, they may be nothing more than the product of wishful thinking (or aid industry group think). We need a much more systematic analysis across a much broader set of institutions, ideally by an independent actor, before knowing the reality.
Therefore, while I would recommend the paper for its examination of an important subject, I cannot recommend its analysis and main conclusions. We really cannot know if donors have improved their performance in fragile states without more information.
* I found the qualitative explanation of how new aid models have improved performance more convincing:
The emergence of new aid models that are better suited to fragile states has gone a long way to address the political economy constraints that undermined earlier donor-partner relationships. Donor-recipient compacts offer a form of partnership particularly suited to fragile states, where the onus is on achieving high-level political buy-in from the partner country and where objectives are defined in terms of the outcomes being sought rather than the means by which they will be reached. Budget support is currently being employed in some of the world’s weakest states—including Afghanistan, Burundi and Central African Republic—through the application of measures that protect against mismanagement and misappropriation, such as budget tracking devices, oversight arrangements, and a reliance on third parties to conduct procurement and audit services. In another example, country ownership has been reconfigured in certain countries, such as Nigeria and Papua New Guinea, to recognize the potential for sub-national governments and communities to substitute for a weak central state in defining development priorities and acting as a reliable partner to donors.
