Contents - Previous - Next

This is the old United Nations University website. Visit the new site at http://unu.edu

OTHER APPROPRIATE APPLICATIONS OF THE NUTRITIONAL COST-EFFECTIVENESS APPROACH

As noted previously, whether or not the a-value criterion can be used and how the a value is to be imputed depends on the nature and objectives of the food aid programme and the available commodities. This section discusses selected applications.

The most straightforward application is where the objective of the programme is to convey maximum flexibility to the beneficiaries to use the food aid in ways that best suit their individual preferences and circumstances. In this case, the food package that maximizes its weighted a value is best. The appropriate or value of each food depends on what expenditures are displaced by the food. If the food supplied is less than the quantity of the same food currently purchased, its value would be the price at which the food is purchased. If the food commodity is not in the current diet, or to the extent that it is supplied in excess of its current consumption, its value is equivalent to the price of a food that it displaces in the current diet.

For example, if 1 kg of corn is supplied at a cost of 20 cents per kilogram and the household currently consumes 2 kg of corn at a price of 20 cents per kilogram, the a value is 1.0. If 3 kg of corn is supplied in the food aid package, the a value of the first 2 kg is 1.0, but the value of the third kilogram depends on what food is replaced in the current diet. If it is rice, which the household currently purchases at 40 cents per kilogram, the a value of the third kilogram is 2.0. If sorghum is replaced, which is purchased by the household at 10 cents per kilogram, the a value of the third kg is 0.5. (Strictly speaking, the maximum replacement potential is the amount of the food-aid-supplied commodity the household would wish to consume once the aid is received. Without aid the consumption of corn may be 2 kg, but with the additional "income" conveyed by the food aid package, the household may prefer to consume 3 kg of corn, in which case all of the corn supplied should be valued at the price of corn.)

To estimate the nutritional cost-effectiveness of commodities that are not in the current diet of the beneficiaries and to compare their cost-effectiveness with other commodities requires knowledge and judgements that go beyond a values. The most common case is the inclusion of fortified and blended foods in supplementary food aid packages. The objective is then usually to correct for particular nutrient deficiencies in the diets of the recipients. Several considerations are relevant. First, will the new foods be acceptable, i.e., will they be consumed and substituted for other foods in the existing diet? Second, if they are consumed, their or values will be lower, which means that they will confer less monetary value and probably less energy augmentation of the existing diets per programme dollar. Will they on balance sufficiently increase the consumption of the intended nutrients? What is the size of the trade-off, and is there sufficient evidence that, for instance, the added cost-effectiveness for protein more than compensates for the reduced cost effectiveness on energy?

A fortified wheat flour always costs more than nonfortified wheat. If, on top of reducing the supply of wheat from each programme dollar, the fortified food is not acceptable for reasons of taste or unfamiliarity, its value and, hence, its indirect impact on nutrition will be further reduced. When fed to animals, for instance, it would have much less value than when it is substituted for a food in the existing diet. Clearly, the reduced or value is the appropriate criterion for comparing the non-consumed food with other commodities.

TABLE 7. Comparisons of Cost-Effectiveness of Fortified and Non-fortified Wheat and Oil

* The index of energy cost-effectiveness for all three commodities and the index of protein cost-effectiveness of wheat and oil is a straightforward function of the ??values. The protein cost-effectiveness of the fortified wheat flour is calculated in accordance with equation 3, given that the difference in protein content between the fortified wheat flour and the unfortified variety is 40 grams per kilogram.

Table 7 illustrates cost-effectiveness comparisons of wheat flour with fortified wheat flour when the latter is substituted for the former in the diet of the food aid recipients. The additional protein consumption induced by a fortified food derives from two sources: the higher protein content in the fortified food and the increased consumption of food made possible by the additional purchasing power conveyed by the commodity. The latter depends foremost on the recipients' propensity to increase the consumption of protein out of an additional dollar of purchasing power. (The indices shown in table 7 are calculated on the basis of equations 1 and 3 discussed earlier).

As expected, the fortified wheat flour delivers a larger increase in protein consumption per programme dollar than the unfortified flour. But there is a trade-off with energy and of course also with purchasing power the household can use for non-food expenditures. The comparison with a commodity of high ??value is, however, particularly noteworthy. When the ??value of oil, for instance, is much higher than the or value of the fortified food, it can lead to more additional protein consumption than would be obtained from the protein fortified food per programme dollar, although oil itself contains no protein!

SUMMARY

Our theoretical and empirical analysis leads us to suggest some propositions that we present here by way of a summary.

1. Food aid is more nutritionally cost-effective than money aid whenever the a value of the commodity is greater than unity. This would be the case whenever the value of the commodity to the recipient exceeds its cost to the aid providing agency. We cited several reasons why this could occur, with illustrations of actual cases.
2. We do not know whether food aid is more or less cost-effective than money aid if the a value is less than unity. The marginal propensity to spend on better nutrition may be higher for food aid than for money aid. If such differences do not exist, or if they could be eliminated with nutrition education, nutritional cost-effectiveness could be enhanced by turning into cash commodities that have an a value of less than unity.
3. For foods that are found in the diets of the recipients in excess of the amounts supplied through the supplementary food aid package, the relative nutritional cost-effectiveness of commodities is always in direct proportion to their a values.
4. The precise nutritional impact of any commodity depends on the marginal propensity of the recipients to allocate additional income (conveyed through the food aid) to improving their nutritional status. If the recipients response is judged to be unsatisfactory with respect to energy (calorie) cost-effectiveness, the only possible remedy is to provide nutrition education. The cost-effectiveness of food aid in terms of protein or any other specific nutrient might also be increased by supplying a suitable commodity (such as a fortified food) that will be consumed in lieu of another commodity in the current diet. In this case, there is likely to be a trade-off of, perhaps, more protein for less energy as a consequence of selecting the commodity with the lower a value. This is sensible only if protein content is judged to be more deficient than the energy content in the recipients' diets.
5. Selection of commodities on the basis of the or value criterion is particularly important when the intent of the programme is to improve nutrition indirectly through providing the beneficiaries with an incentive to participate in nutrition education or work programmes.
6. Substitution of the new commodity selection criteria for those currently in use in food aid programmes could bring about a large increase in nutritional cost effectiveness. Moreover, the data requirements are low and could be met at minimal cost.

REFERENCES

1. M. A. Anderson, "CARE Preschool Nutrition Project" (CARE, New York, 1977).

2. J. M. Baertl, E. Morales, G. Verastegui, and G. G. Graham, "Diet Supplementation for Entire Community," Amer. J. Clin. Nutr., 23: 707 (1970).

3. M. G. Hererra, J. O. Mora, N. Christianson, N. Ortiz, J. Clement, L. Vuori, D. Weber, B. de Paredes, and M. Wagner, "Effects of Nutritional Supplementation and Early Education on Physical and Cognitive Development," in R. R. Turner and F. Reese, eds., Life-Span Developmental Psychology Interventions (Academic Press, New York, 1980), pp. 149-184.

4. J. O. Mora, J. Clement, N. Christiansen, J. Suescun, M. Wagner, and M. G. Herrera, "Nutritional Supplementation and the Outcome of Pregnancy: III. Prenatal and Neonatal Mortality," Nutr. Rep. Int., 18: 167 (1978).

5. G. H. Beaton and H. Ghassemi, "Supplementary Feeding Programs for Young Children in Developing Countries," Amer. J. Clin. Nutr., 35: 864 (1982).

6. C. Overholt, S. Sellers, J. O. Mora, B. de Paredes, and M. G. Hererra, "The Effects of Nutritional Supplementation on the Diets of Low-income Families at Risk of Malnutrition," Amer. J. Clin. Nutr., 36: 1153 (1982).

7. H. Delgado, E. Brineman, A. Lechtig, J. Bongaarts, R. Martorell, and R. E. Klein, "Effect of Maternal Nutritional Status and Infant Supplementation during Lactation on Postpartum Amenorrhea," Amer. J. Obstet. Gynec., 135: 303 (1979).

8. O. Knudsen and P. Scandizzo, "Nutrition and Food Needs in Developing Countries," World Bank Staff Working Paper No. 328, (World Bank,Washington, D.C., 1979).

9. J. O. Ward and J. H. Sanders, "Nutritional Determinants and Migration in the Brazilian North-East: A Case Study of Rural and Urban Ceara," Econ. Devel. Cult. Change, 29: 141 (1980).

10. P. Pinstrup-Andersen and E. Caicedo, "The Potential Impact of Changes in Income Distribution on Food Demand and Human Nutrition," Amer. J. Agric. Econ., 60 (3): 402 (1978).

11. India National Sample Survey: 26th Round, 1971-1972.

12. F. D. McCarthy, Nutrition Planning: Analysis of Some Policy Options, INP Discussion Paper No. 9 (International Nutrition Planning Program and Center for International Studies, Massachusetts Institute of Technology, Cambridge, Mass., USA, 1976 rev. 1978).

13. Food Habits Survey: Gujarat and Maharashtra, indict (1969).

14. Food Consumption in Brazil: Family Budget Surveys in the Early 1960s (1970).

15. S. Kumar, Role of the Household Economy in Child Nutrition at Low Incomes: A Case Study in Kerala, Occasional Paper No. 95 (Department of Agricultural Economics, Cornell University, Ithaca, N.Y., USA, 1978).

16. National Research Council, Nutritional Analysis of PL 480 Title II Commodities (National Academy Press, Washington, D.C., 1982).

Proposed guidelines for designing evaluation for nutrition and health programmes

J. B. Mason
Cornell Nutritional Surveillance Program, Division of Nutritional Sciences, Cornell University, Ithaca, New York, USA

INTRODUCTION AND GENERAL PRINCIPLES

Purpose of This Paper

Running a programme requires a management structure to ensure delivery and supplies and to provide training, backup, and supervision for the field workers. A successful programme will provide the right supplies to the right people in a timely manner and see that field staff are adequately trained, supervised, and supported. Programme management therefore involves making decisions about allocation of resources- material and human. These decisions require certain information that differs at different levels of the administrative structure. This paper gives suggestions on methods for setting up procedures to provide the minimum information necessary to make these decisions. Its purpose is to give guidance to those responsible for designing a built-in evaluation mechanism for country programmes under the Joint Nutrition Support Programme (JNSP) of WHO and UNICEF. It is a first attempt to provide such guidance and is intended mainly for the initial needs of those considering evaluation. The scenario envisaged is that either a government planning officer or a consultant or some other such person needs to produce recommendations during the planning of a country programme in a relatively short period of time. The first experiences should be made available as case studies, so that as soon as possible there will be some concrete evidence to build on.

Several assumptions are made here:

- There is a detailed plan of the programme either avail able or being developed, and some guidance will become available on the choice of programme components.
- The evaluation planning is part of the programme planning.
- It is generally agreed that it is important to make a start, and to move towards more efficient and realistic designs based on experience as it comes in; hence the initial designs reached through this process are open to modification and evaluation. - Nonetheless, it is essential to arrive at some initial design for the evaluation procedure with a fair amount of detail, enough to see if the design is likely to be workable. This is the "design document" referred to hereafter.

Two immediate outcomes of using this paper are therefore intended: first, the beginnings of a process of developing the evaluation procedure - involving project planners, management, and supporting institutions, as appropriate, with special expertise; and, second, the design document itself (see the section "The Design Document" below).

It should be understood that, wherever there are existing mechanisms for gathering data, interpreting them, and indeed for evaluation (e.g. as part of wider information systems or as established administrative procedures), these will be used and adapted.

Terminology and Basic Concepts

There are different types of decisions to be made at different points in the management of a programme. A village health worker may need to decide what to do when examining a child, which household to visit with what services, etc. A village health committee may need to check that the correct diagnoses are being made, and where, in general, health and nutrition are improving. The district level needs to know that resources are reaching the right villages, that development of local capability is proceeding, and that nutrition and health are improving-taking the necessary action otherwise. Here it is also important to know, in certain cases, whether changes in health and nutrition are due to the programme, or to part of it, to make more far-reaching decisions on the programme activities at the national level. These include management's decisions and also decisions based on some knowledge of the effectiveness of programme activities; both are important.

The procedures discussed here aim to fulfil these needs for decisions at different levels of management. Unavoidably, there are different aspects to the procedure. The most important concepts to clarify at the start are: - the different types of information, on programme implementation and on health and nutrition, - the distinction between assessing overall change, and change due to the programme: the former may suffice for management, but the latter is needed for policy decisions.

There are a wealth of different terms in use for planning and for evaluation, and it is impossible to choose one convention that will be familiar to everyone. Worse, there is such a confusion in the use of different terms that a "constraint that is often encountered is a certain built-in resistance in principle to accepting evaluation and its results as a valid management tool" (1, pare. 9).

The "evaluation" procedures discussed here are primarily (but not only) an integral part of programme management. Their purpose is the same as that for health programmes: "Evaluation is a systematic way of learning from experience and using the lessons learned to improve current activities and promote better planning . . . for future action" (1, pare. 6). The relation to other terms in current use is, briefly: "evaluation" (here) = "evaluation" (1) = "monitoring and evaluation" (2) = "operational programme evaluation" (other literature).

We distinguish here between routine evaluation, aimed at ensuring satisfactory programme management, and impact evaluation, aimed at assessing the net effect of a programme. Before explaining this, other concepts need to be clarified.

1. There is a difference between checking on ("monitoring") an activity (e.g. number of home visits) and examining its effect (e.g. nutritional status). The information relating to activities is known here as "process." The information relating to effects or outcome is known as "outcome." In other terminologies the equivalents are as follows: "Process" = "review of progress," "efficiency," "effectiveness" (1, pares. 38, 56-70) = "inputs and outputs," or "monitoring" (2). "Outcome" = "output" (1, pare. 67) = "effect and impact," "assessment of results" (2).

2. There is another essential difference between examining changes in outcome (e.g. improvement in nutritional status) for programme participants and in considering whether these are due to the programme. One difference is obviously the changes that would have occurred anyway with or without the programme; one difficulty in assessing this may be because of the way in which programme participants are selected, and so on. The overall change, not allowing for changes that might have occurred anyway, is known as "gross outcome." If attempts are made to determine how much change is due to the programme, this is referred to as "net outcome." The expression "net outcome" is synonymous with "impact." The relationship may be illustrated as: Net outcome (or impact) = gross outcome- changes not due to programme, etc.

This concept is discussed further in Mason et al. (3) and Freeman et al. (4). It is suggested that a portion of the effort put into evaluation should be devoted to impact evaluation, since this is the only way that some estimates of the actual effectiveness, and possibly the cost-effectiveness, of the programme can be assessed.

"Routine evaluation" refers to using information on process and gross outcome to reach conclusions and decisions useful to programme management. Routine evaluation is similar in concept to a management information system. This procedure checks if programme implementation is adequate, by comparison with operational objectives, and if gross changes in health and nutrition are adequate in comparison with impact objectives. This is equivalent to "routine adequacy evaluation" (5): because community participation is central to the programme, it is assumed that flexible (decentralized) management is the norm, and there should be no distinction between the types given in Mason and Haaga (5) between routine adequacy evaluation and routine adequacy evaluation/flexible management. This concept of routine evaluation is equivalent to much of the evaluation process referred to in the WHO Health Programme Evaluation ( 1, pares. 36-70) and to nutritional surveillance for programme management (3). "Impact evaluation" uses information on process and net outcome; it is equivalent to "assessment of impact" in Health Programme Evaluation ( 1, pare. 71 ).

The Joint Nutrition Support Programme distinguishes between "impact objectives" (e.g. reducing infant mortality) and "operational objectives" (e.g. building national capacity). These objectives relate directly to outcome indicators (to assess impact) and process indicators (to assess progress towards meeting operational objectives). Quantification of these objectives in the planning stage gives the criteria against which progress can be assessed by evaluation. This specification should be directly related, again initially in the planning stage, to the management or policy decisions that are to be made if these objectives are or are not met (in management, usually the latter). In the section "The Design Document" examples of output tables are given that include criteria for deciding if the programme is on track in terms of process of implementation and outcome. These criteria are the same as the operational and impact objectives (as conceived of in the JNSP documents). The evaluation procedure is thus a way of making these objectives meaningful and is yet another reason why planning a programme should include planning the built-in evaluation. The level of detail reached in planning the programme, whether prior to implementation, or, often better, as the programme is implemented (learning by doing), determines the level of detail feasible in designing the evaluation. Ideally, these should all be part of the same process.

Impact objectives need to be distinguished as "net outcome objectives" and "gross outcome objectives." Operationally, the outcome objectives needed for routine evaluation (e.g., to reduce preschool malnutrition by so many cases per 100 per year) are gross outcome, but the impact objectives are net outcome. It many be decided in the programme planning that gross and net outcome objectives should be set as the same reduction in malnutrition, but the distinction is crucial for subsequent evaluation.

Objectives of the Joint Nutrition Support Programme and Built-in Evaluation

As the objectives of the JNSP have been reviewed in several documents and are summarized in Mason and Haaga (5, p. 2), which also gives the objectives of individual programmes and of the monitoring and evaluation (p. 4), they will not be repeated here. It should be noted that additional emphasis should be given to promoting organization at the local level (thus, objective 1-A on page 4 of Mason and Haaga should read ". . . effectively deliver certain appropriate goods and services to certain people, particularly through building capability at local level").

Two additional points may be made. The first is that it is important to retain the overall impact objectives of improving child nutrition, reducing infant and child mortality, etc. Although this may be a long-run objective, nonetheless it is the purpose of the operational objectives, including building local capability. As this capability is built up, it should have a fairly rapid effect, at least on nutritional status. There is a risk that, if the human impact objectives are downplayed in favour of organizational objectives, organizations may be successfully set up with no clear idea as to what precisely they are supposed to do. A second related point is that it is crucial to include some element of assessing impact. The present state of knowledge on what is effective in improving nutrition and reducing infant and child mortality is often inadequate. It is most important that this situation be improved and that the JNSP include the objective of gaining better knowledge of what interventions should be concentrated on in future. Otherwise, the opportunity may pass, and it will continue to be difficult to recommend confidently most of the interventions currently being considered, for lack of information on their effectiveness. Moreover, now is a particularly good opportunity to do this since the JNSP programmes are intended to be operational, not pilot, programmes, and to be replicable. One of the major constraints has been that there simply is not much information on impact that has come from pilot projects; Gwatkin et al. (6) give almost all that is currently available. Also, it seems that one of the major reasons that impacts may not be achieved when these are scaled up is precisely because the scaling up involves changes in management, leadership, etc.

Routine and Impact Evaluations

It is considered vital to mix routine and impact evaluations. Routine evaluations are essential for running the programme. Impact evaluations are needed to find out how the programmes work. The point is discussed in Mason and Haaga (5) and is here taken as accepted.

A final point concerns baseline surveys. These may be desirable for impact evaluation. However, they are not needed for routine evaluation, nor are they always necessary for impact assessment. Often it is better to use the first data available as the programme starts. As a general rule, if a baseline survey is essential for programme planning, which is less often than commonly supposed, its design should enable it to be used for impact evaluation. But usually a baseline survey before the project is planned is not necessary.

Other Documents

Other papers of ours that may be useful in considering this subject (3, 5, 7) give more extensive references to the relevant literature.

continue

Contents - Previous - Next