Previous - Next
This is the old United Nations University website. Visit the new site at http://unu.edu
The need and rationale for monitoring and evaluation of nutrition programmes
P. C. Sen
Adviser (Nutrition), Directorate General of Health Services, New
Delhi, India
A number of nutrition intervention programmes have been launched in developing countries to combat the widespread malnutrition among vulnerable sections of society, especially children and expectant and nursing mothers. The successful implementation of any nutrition programme is dependent on several variables:
The implementation of even a very well conceived programme may also pose problems and difficulties that may impede efficient delivery of benefits and their utilization by the beneficiaries. It is therefore necessary that the implementation of these programmes be observed and studied systematically in order to have a realistic assessment of the manner in which the services are actually delivered and utilized. Such efforts may be made with a view to identifying points of divergence between what is envisaged and what obtains in the field and the problems experienced by the implementing agencies and the beneficiaries. Monitoring on these lines is the tool for arriving at modifications in the policies of implementing agencies and strategies employed in the implementation of these programmes.
Monitoring is a continuous, routine collection of data at the field level by non-technical staff. It is not just progress reporting at different intervals on physical and financial targets achieved or the inputs (building, personnel, supplies, etc.) provided for the implementation of the intervention. It is a two-way flow of information both laterally and vertically to aid in decision-making and quick action. The flow of information in a monitoring system would depend the level (centre, state, district, and block) and the phase, i.e., installation or delivery. While at the installation stage, feedback on timely provision of physical and financial inputs should receive emphasis; at subsequent stages, the focus should be on the attainment of the objectives of the programme in terms of projected beneficiaries and extent of coverage, quality of service, etc.
The ultimate success of the programme can be judged by the impact it makes on the health of the beneficiaries. Impact can be seen by periodic evaluation of the programme, which is more specifically intended for programme continuation, discontinuation, modification, revision, or expansion. It is directed towards quantifying achievements attributable to the programme in particular planning periods. Questions to be answered are: Are the goals achieved? What are the necessary changes to be made? Monitoring and evaluation must be viewed as an integral part of the planning process and be built into the scheme at the beginning. Although actual monitoring begins when the programme is initiated, exercises for devising the monitoring systems must start even earlier. Similarly, though evaluation ideally should begin after a scheme has been in operation for some time, the timing of the evaluation and its focus should receive consideration in the initial stages of planning.
Monitoring of any nutrition programme involves the following:
a. Identifying the different units involved in planning and implementation of the programme and establishing the necessary intra- and inter-departmental linkages.
b. Identifying items on which feedback is required and the level at which the feedback on different items is to be given. Both positive and negative indicators are important.
c. Identifying the personnel at different levels and defining their role in monitoring.
d. Developing programmes for reporting. This should be simply structured with complete instructions given and be capable of being handled by the field staff.
e. Determining the periodicity of reporting, which will depend on the item, need for information, stage of implementation, etc.
f. Processing information received, analysing it in terms of projected progress, indicating the positive and negative elements, the nature and extent of short-fall, reasons why, and suggesting corrective action.
g. Sending the relevant information to the decision making points and transmitting the decision to the different implementing levels.
h. Storing data and developing facilities for their retrieval when necessary, particularly when needed for evaluation of the programme.
i. Identifying the critical and vulnerable points in the implementation of the scheme.
j. Determining the degree of precision expected in reporting information. Since precision demands time, money, and manpower (both in numbers and professional competence), it will be necessary to decide on the kinds of information for which approximation will suffice for decision-making.
To build an efficient monitoring system, provision of appropriate manpower, training of personnel, supervision, etc., are necessary, with importance given as much to field-level requirements as to the intermediate and apex level. One of the prerequisites for proper monitoring is the clear specification of objectives of nutrition interventions. This contrasts with the aim, which is always in general terms. It is surprising that in plans for most schemes these objectives are not clearly specified; they are stated in general terms, making the task of monitoring and evaluation rather difficult.
A manual of operations to guide the field staff will greatly facilitate the implementation of the scheme and raise the levels of programme performance. Monitoring at the field level can be carried out on the basis of the standards laid down.
EVALUATION
The role of evaluation in judging the effectiveness of a nutrition programme can hardly be overestimated. With commitment of public funds of a fairly large order, concern is naturally expressed about the impact of the programme, whether any dent has been made in the problem of malnutrition, and whether, for example, food supplements are an efficient, economical, and feasible way of tackling the problem. The purpose of evaluation is not just to locate lapses, but to give objective data on various aspects of the scheme and thus serve as an aid to decision-making.
Another use is the identification of the vulnerable points in such programmes. It would cover:
a. the concept behind the scheme, the assumption underlying the same, and the extent to which these are found relevant and valid;
b. the costs of implementing the scheme and whether it is possible to get more mileage out of funds available;
c. the operations involved in the delivery of services, administrative organization and structure, personnel, standards of services, etc.;
d. the impact of the scheme-i.e., an indication of the extent to which it has been able to tackle the problem and produce the desired results; negative impact, if any, should also be indicated.
The immediate task in evaluation is the statement of primary and secondary objectives, which may be one or more of the following:
In each case, the targets should be expressed as clearly and precisely as possible. Once this is done, the instruments or means to attain the objectives have to be identified, the operations listed, the agencies performing the operations determined, the target group identified, the duration and service and its coverage specified, and the indicators for measuring progress in the attainment of objectives worked out. It is also necessary to indicate clearly the criteria for defining the target group and the procedure for identification of individual beneficiaries.
The questions most frequently asked by policy-makers and administrators relate to the impact of the programme: which of the inputs (singly or in combination) contemplated in a scheme as a package produce the maximum impact and which lead to only marginal benefits. A basic assumption in such programmes is that the action (independent variable) produces an effect (dependent variable).
1. Is the information likely to lead to the anticipated change? A number of research designs are adopted for undertaking evaluation, including before-after, experiment-control group, and beneficiary response analysis. Several difficulties arise in the case of supplementary feeding; complications are created by the fact that the net availability of food in the family (a primary determinant of the nutritional status of the child) is, if at all, only marginally influenced by the scheme. More effective interventions may include health and nutrition education and health services. Indeed, the greatest influences on the net availability of food in the family are economic-the extent of the family's landholding, or the availability of employment.
2. Within what time frame is an impact expected? Another important factor to be considered is the time by which the anticipated impact is expected to occur. This may vary for different objectives. Further, in the case of certain parameters, the problem of recall-lapse should also be considered when asking an evaluation question.
3. Can experimental groups be matched with controls? The matching of experimental groups with control groups also creates problems. For operational convenience, the matching is sometimes undertaken with broad socioeconomic and demographic parameters. Other important variables like dietary intakes may be more relevant, but data on them may not be available. A before after design would not only require studies at two points in time but also would need to reckon with the social and economic changes in the area caused by other factors. These may have an important bearing on prices, wages, and employment and therefore also on nutritional status.
4. Sample survey or case study methodology? Another area of choice is between a cross-sectional research design covering a large area and selected case studies of centres and their beneficiaries that provide a more thorough process analysis. Complications in impact evaluation are also created by factors such as disruptions/irregularity in inputs, the nature and extent of which are likely to vary from place to place.
5. Relationships between evaluation and monitoring. Evaluation is facilitated and qualitatively improved if information is regularly collected during the operation of the scheme, both for purposes of monitoring and for later use for evaluation, and the data must be properly analysed and stored. In fact, evaluation is dependent on an efficient monitoring system. Without proper feedback from the monitoring cell, no evaluation will be correct.
6. Dissecting out the impact of an individual component. Sometimes ad hoc evaluation is done to see the efficacy of a programme. There is also terminal evaluation possible in a time-bound nutrition programme, but most of the programmes in nutrition are related to general health services like MCH (maternal and child health), immunization, health education, water supply, etc. The parameters often used to determine the impact of a nutrition programme are not sufficient to assess an individual programme's impact.
CONCLUSION
Systematic monitoring and evaluation should be developed within the general health services structure. Countries may differ in their existing infrastructure for health-care delivery systems when it is a question of primary health care. Nutrition as a component of primary health care should also be assessed by its own parameters included within the information regularly kept by the primary health centre functionaries. The evaluation of package programme efficiency needs to be oriented according to the country's need. Thus, the need for monitoring and evaluation of nutrition programmes becomes part and parcel of the whole health care package.