Contents - Previous - Next


This is the old United Nations University website. Visit the new site at http://unu.edu


Note for the record: comments on the paper ''methods for evaluating the nutritional impact of food aid projects: lessons from past experience'' by David Sahn

Hossein Ghassemi
Senior Nutrition Adviser, UNICEF, New York

Dr. Sahn's paper pinpoints many problem areas in programming and administration of food aid. It states that past definition of objectives has not always been clear. There have been problems in programme design. Sometimes programmes are not based on a solid hypothesis. There have been poor implementation and various methodological problems in evaluation. The paper also concludes that evaluations have often been qualitative, incomplete, and inadequate.

The paper recommends two important lines of action: The first is to design and conduct a number of in-depth studies in order to gain a better insight into programme costs, effects, and proper indicators for evaluation. It also proposes building into ongoing projects a solid, carefully designed element of monitoring and evaluation and suggests that the additional costs of such action are entirely justified.

In reviewing the lessons in evaluation, there are several issues to be raised:

First, it is important to distinguish between good projects that were poorly evaluated and poor projects that had a favourable evaluation.

Second, we need to ask, Evaluation for what' Evaluation has not been approached broadly enough. In general, there is a problem of utilizing evaluation for various purposes.
Evaluation can and should be conducted for three different reasons:

- to improve decisions on choice of policies and programmes,
- to improve efficiency and effectiveness,
- to bring advocacy for generating enough food aid for the needs of developing countries.

Evaluation activities in the past have concentrated more on the questions of efficiency and effectiveness and very rarely have been designed in relation to programme/policy choices, and they have certainly not been effectively utilized for advocacy purposes. As a matter of fact, the field of food aid in the international development community has evolved a rather bad image as a resource that has been poorly used, and some negative effects, such as disincentive for food production, have been of great concern. It is not often understood that food aid is a resource; it is the way it is used that makes the difference.

Therefore, the suggestion is that the field of evaluation should be clearly studied and directed towards meeting all three of these important needs. For major agencies in charge of programming and administration of food aid, there is an urgent need for better knowledge in programme policy choice and also for the whole area of advocacy.

Third is the question, Evaluation by whom? Most evaluation studies have been conducted by the donors or as a part of special studies. It is important that programme management strongly encourage the authorities in the countries to take monitoring/evaluation seriously into account in proper design and implementation.

Fourth, considering all the constraints and uncertainties raised by Dr. Sahn, what can be done now? I am raising this question with some specific suggestions that I believe are feasible and practical. What can we do now, based on the available knowledge and skills? The following immediately come to mind:

  1. There is a need for better integration of food aid projects into overall community development activities. It is common practice now for food aid projects to take a rather isolated, individual approach and as an operational channel within the system. This implies that food aid is not being effectively considered as another resource available in the community that can be pooled and effectively used within the overall frame of priorities and programming in the community, especially in the areas of nutrition and health needs of vulnerable groups. Problems of objectives, implementation, participation, etc. all stem from lack of integration.
  2. A great deal of improvement can achieved in developing better, clearer, and more concrete definitions of objectives for the use of food aid.
  3. There also needs to be improvement in implementation. Three options exist:
  1. Improve management capacity, logistics, targeting, supervision, etc.
  2. Shift responsibility for implementation to the community.
  3. Modify the programming approach. This option is obviously of a more long-term nature.

If one has to make a choice between strengthening programme management- i.e., trying to do a better job through more efficiency on the part of management- and shifting the responsibility to the recipients, it is quite likely that the second option promises much more hope for a better outcome and would be less costly. However, this is a hard decision to make.

With regard to modification of the approach to programming:

First, for quite some time suggestions have been repeatedly made to change the concept from food distribution to food as medicine, specifically targeting on advanced cases of malnutrition. In that concept, the health system has the responsibility of prescribing food but not necessarily distributing and administering it.

Second, food aid can be used through women's programmes and income-earning projects, especially those targeted on increased local food production. This is an approach that would turn the whole operation around and make it possible to use food aid for increasing food production rather than competing with it. This can also be effectively used for helping women to earn more income, and for reasons that I will explain later, this can have a very strong incentive effect on better design and implementation of primary health care/nutrition programmes

Third, food aid can be used in a monetized form for supporting either the initial or the recurrent costs of community development programmes.

It is very clear that concentration on these three possibilities, among many, would change the design and operational aspects of food aid to a great extent and would give it a much more effective role as a major resource element in the total development efforts supported by the international community.

It is important to recognize that costs and benefits of a programme will be strongly influenced by the socioeconomic circumstances relative to advancement in access, infrastructure, income, etc. Such circumstances in turn affect relative size of deprivation and therefore needs and the cost of meeting those needs, and also the magnitude and durability of benefits achieved. It happens that the most disadvantaged are the top priority for humanitarian reasons, but it would often cost more to meet their needs. Hence, it would mean the same benefits for higher costs. Also be aware that small input in relation to large need (e.g., food gap or calorie gap) is very unlikely to produce a measurable impact. This is where the resources are spread too thin. Such cases should be seen as resource allocation, project design, and implementation problems and not as evaluation problems, unfortunately, they are not rare.

A very important point is that the annual cost of eradication or control of hunger in the world has often been cited as five billion dollars and, as the discussion in the Sahn paper indicates, the value of food aid currently available for the specific needs of vulnerable groups such as mothers and children and for nutritional purposes easily adds up to about one billion dollars per year. This is a substantial resource, and proper use of it could have a major impact if we are willing to take a serious look at the possibilities of better use of it and to be prepared to make hard decisions.

Among the very low-income households, there is substantial evidence that programmes that tend to effectively reduce poverty at the household level- such as income-generating activities- would be extremely effective in helping the poor to elevate themselves above the poverty line. Such improvements would then make them much more responsive and active participants in organization and operation of nutrition/primary health care activities in the community.

Therefore, use of food aid in cash (through monetization) for income generation, together with local food production with particular focus on women, offers excellent potential for an integrated, development-oriented approach to nutrition improvement.

Hard decisions must be made in the following areas:

  1. Strong efforts are needed to change the programming approach.
  2. More money should be spent on evaluation and promoting evaluation in participating countries. There have been many arguments that one should be very careful about the amount expended for evaluation. Sometimes the administrators are hesitant to consider using major portions of resources for evaluation, arguing that this is actually the food or resource being taken from the needy. This is a false argument because there is enough evidence that spending an additional 1 or 2 per cent of the total resources on evaluation would contribute to improvements in the whole understanding of programme policy choices, in efficiency, and especially in advocacy, that should help improve the image of food aid and the potential for more flexible aid. In both the short and the long run the return would be much greater than the cost. In this context, it is also important that major attention be given to generating enough political will among the governments to prepare the overall climate for such major changes.
  3. Decisive initiatives should be taken to sharpen the focus on the purposes of evaluation, improvement of provement of project design, and evaluation for advocacy.
  4. There is need for support of in-depth studies. It should be made very clear that in-depth study is not specifically designed for better evaluation of projects. It is a complex exercise designed to develop better, simpler, and more practical methodology at the field level, and this is an expert attempt to improve the methodological aspects of evaluation, which is quite different from assuming that in-depth evaluation at much higher costs is needed to evaluate every project.
  5. Finally, it is important to differentiate between what is possible now and what needs to wait for the future. For example, in the area of evaluation at the project level, it is much more practical at this point to improve and emphasize the process evaluation aspects of the project and give more time to more definitive and effective impact evaluation. This is primarily because of some methodological problems, especially in the area of indicators, that need some time for clarification of a number of controversial issues. At this time, impact evaluation should focus on the best use of reliable methodology and wait for necessary improvements in order to add to the precision of impact measurement.

Contents - Previous - Next