Previous - Next
This is the old United Nations University website. Visit the new site at http://unu.edu
Projects to programmes: An international perspective
James Greene and John Kevany
Programme design and development
An important consideration in the transition from projects to programmes is understanding the nature and extent of the problem. The initial definition of vitamin A deficiency at a national or regional level may be based on relatively weak estimates of magnitude, derived from service records or unrepresentative surveys using variable criteria and procedures. In the selected project area, where precise objectives are set and rigorous evaluation carried out, more exacting procedures and criteria may be applied, with a resulting loss of comparability between project and programme conditions. To maintain the validity of assumptions underlying expected programme effectiveness, systematic corrections to epidemiological rates for the area may be required or, alternatively, the population may have to be reassessed using the project criteria.
This situation is further complicated if new and more sensitive biological indicators have been introduced at the project stage to improve the assessment of response. If biochemical or functional (dark adaptation) assessment of vitamin A deficiency has been used in the project to improve the sensitivity of response measurement, the question arises as to whether this should be used at the programme level to redefine the magnitude of the problem in public health terms. Such redefinition may greatly increase the size of the population perceived to be at risk and lead to a loss of precision in targeting interventions.
In addition, the scope of VITAD-prevention programmes is changing over time. New and increasingly reliable research information has emerged in the past few years on the role of vitamin A supplementation in controlling mortality from common infections in childhood. The Canadian International Development Association funded a recently completed meta-analysis of all major mortality intervention studies which concluded that an average reduction of about 23% in mortality among children under five years of age may be expected as a result of supplementation with vitamin A concentrates in populations with some frequency of overt eye disease. This means that programme objectives and expectations may now be more extensive than those of precursor projects. Mortality indicators have not always been used in the first generation of pilot projects; setting realistic objectives for reduction in large-scale follow-up programmes may present difficulties.
Similar attention is necessary with respect to strategy definition and the selection of programme interventions. Earlier efforts to control eye disease were based principally on capsule distribution and were located mainly if not exclusively in the health sector. Although longer-term dietary change was sometimes incorporated in the design, it was usually a secondary component, and its implementation was less closely monitored or critically evaluated. Current international policy places more emphasis on dietary diversity as an avenue for the long-term prevention of vitamin A deficiency. Accordingly, a higher priority may be accorded to dietary measures in the programme plan than in the pilot project, which will have implications for the estimates of future programme effectiveness based on project performance.
Whereas we have good evidence for the effectiveness of some interventions, such as capsule distribution, the evidence for others is less complete. By extension, the effectiveness of different strategies based on various combinations of interventions is even less clear, and the law of diminishing returns may apply. The addition of further components to a single-intervention strategy may be expected to produce further improvement in overall effectiveness, but the question remains as to whether the incremental gain can be justified in terms of the additional cost.
The effect of changes in intervention strategy between project and programme phases applies also to inter-sectoral cooperation. If dietary change as a preventive measure is made a priority, adequate supplies of foods rich in vitamin A (and ,8-carotene) at the household level must be developed, requiring improved production, processing, and distribution systems as well as increased beneficiary demand. Whereas supplementation is delivered almost exclusively through primary health care systems, dietary intervention requires effective cooperation among agriculture and food-processing and distribution interests and their acceptance of health and nutrition status as an internal priority. In view of the critical role of agricultural production in export-led economic growth and the associated priority assigned to the production of (often non-food) cash crops, this may not be easy to achieve widely over the short run.
Centralized fortification of dietary staples with vitamin A is also likely to involve the active cooperation of the private and public food industries. Community-level fortification may be technically and managerially feasible but probably still requires negotiation with local food suppliers.
Within the health sector, programme managers also have to decide whether the new VITAD intervention is to be given priority over competing activities that use the same delivery system. Capsule distribution and associated tasks such as inventory maintenance require additional time and effort from care providers. So do nutrition education for dietary diversification, and communication and coordination with staff of other agencies involved in the production, storage, and distribution of food supplies.
Most health interventions at the primary level are delivered through the community health worker, whose job description almost universally is heavily overloaded. The addition of even relatively simple tasks such as the above must involve formal and informal trade-offs with other work unless personnel resources can be increased. Where human resources are severely limited, care must be taken to ensure that the introduction of measures to control vitamin A deficiency does not adversely affect the performance of other essential functions and, by extension, of the supplementation process itself. The true extent of the incremental effort required is often greatly underestimated in making decisions about up-scaling simple interventions to the programme level.
Training, supervision, and management
In moving from smaller projects to large-scale programmes, the key issues shift from technical effectiveness and feasibility to the tougher question of operational impact and costs. It is increasingly clear that the principal determinant of programme impact, cost-effectiveness, and efficiency is what happens at the intersection of the three most crucial aspects of delivery systems: training, supervision, and management.
These three aspects are critical because populations targeted for vitamin A and other nutrition programmes tend to be highly differentiated by a number of factors, frequently including socio-cultural characteristics. Reaching such people effectively requires flexible training, supervision, and management systems so that services and communications activities can be varied locally in the light of local conditions, priorities, beliefs, and behaviours. One way to promote a better fit between strategy and services (and to generate local commitment) is for communities themselves to take the lead in recognizing their vitamin A problems and deciding what to do about them.
Unfortunately, under most delivery models, service providers usually have relatively few built-in performance incentives. In a situation characterized by low technology and knowledge and high uncertainty about outcomes, the tasks of service providers often are not clearly defined or are intrinsically hard to measure and therefore not easily monitored. Furthermore, a lack of demand from the clients and their generally marginalized status act as disincentives to workers' concentration on services to them. Clearly, management approaches have to take account both of the services to be delivered and of their organizational and cultural contexts.
Whatever is done in these key areas has to derive from decisions on what services are to be delivered to the clients, by whom, how often, and with what expected results. The cutting edge of that delivery process is work routines resulting from an iterative process of hypothesis, testing, and refinement. A basic consideration is to define task priorities and their time implications. Those priorities should be consistent with the nature and magnitude of the problem, and must also reflect a level of technology that the workers can deliver at a reasonable cost and with a reasonably assured outcome.
Several decisive elements are involved in going to scale. Training at all levels must emphasize problem-solving, field practice, learning by observation, and imparting better knowledge, attitudes, and skills. Pre-service training should emphasize practices: what to do, how to do it, why it is important. Workers should leave pre-service training confident of their ability to deliver services and of why these particular services are more important than others. Nextin-line supervisors have to be trained to supervise, not just to administer. In-service training has an important complementary role in solving problems, incrementally increasing skills, and orienting staff to seasonal or other changes in programme emphasis.
There are probably more than a few cases where, as in India, nutrition and complementary health services are delivered by different workers reporting to different departments. Where, as is most likely, nutrition services and staff are added to an already functioning health system, difficulties may be encountered at least initially in getting the different worker streams to interact satisfactorily. Joint training may help to establish an initial climate for collaboration and procedures, but much more durable impact comes from management signals through supervisors and other aspects of the bureaucracy.
Supervision, particularly at the periphery, ought to be seen as essentially providing on-the-job training and solving problems rather than as an inspection function. Therefore, it must be sensitive, consistent, flexible, and frequent. Both quantity and quality have to be considered. Tasks that vary in intensity and sequencing require more frequent supervision. The question of whether the performance benefits of more, rather than less, supervision justify the additional costs is seldom explored sufficiently.
It is possible to analyse supervision functionally, determine how much time is required for each task, and then work out the marginal costs and output of varied supervision regimes as long as the output measures are clearly defined in evaluable terms. Some programmes have determined ways for community representatives or committees to supervise or monitor the service-delivery staff. Advantages are a presumably better linkage between demand and supply in terms of both quantity and quality, and a stronger sense of community ownership of the services. A prospective disadvantage of so-called community control is the possibility of manipulation by the local elite, which can be countered when recognized if the commitment to community empowerment is real.
Supervision becomes complicated when two administratively independent worker streams deliver complementary services, particularly if the workers are at different technological levels. This will apply specifically where dietary diversification is a principal component of the VITAD-prevention strategy. Service inputs from agricultural extensionists, home economists, schoolteachers, and health workers have to be coordinated systematically at the local level to make this intervention work effectively. Some mechanism is necessary to send the right signals to front-line workers and for supervisors to meet and sort out issues at the field level. This is a tough problem that few major programmes have resolved successfully. Nevertheless, despite constraints, several have demonstrated high levels of worker and supervisor commitment, maybe partly because the design of the project fostered self-evaluation and worker self-respect.
Supervision at the district or comparable levels of intermediate management also requires attention. This is the highest tier at which rapid operational feedback to the field staff is possible. It is the point at which performance data converge for consolidation, analysis, and transmittal to the project management. It is also the level from which operations research questions are posed for higher-level consideration. An essential tool of intermediate field management is a good monitoring system to provide early warning of performance problems in particular geographical or functional areas. This is essentially a level for planning, programming, budgeting, and administrative decisions based on interaction with the top project management on strategic questions and feedback from below on operational matters.
Special programme requirements
Data collection and analysis
Although the quantity, quality, and timeliness of data collection and analysis are usually enhanced at the project level, particularly where effectiveness is being tested, it is not always necessary to establish special systems and units for this purpose. Furthermore, external sponsoring agencies often become involved directly in this process to meet their own wider information needs. Under these conditions, the real costs of monitoring and evaluation can be underestimated, as data are essentially managed outside the project. When a follow-up programme is initiated, it is usually essential to establish a special monitoring and evaluation unit for the first few years to keep track of performance in larger populations and under more variable conditions, thereby adding an additional but worthwhile element of programme costs.
Financing and resources
Responsibility for financing and resource allocation may change in the transition from projects to programmes. As mentioned above, small-scale effectiveness projects are often initiated by agencies or groups with external or additional funding, whereas the larger and longer-term commitment of programme resources is usually the exclusive responsibility of health and other national services. In making decisions about scaling up, it is therefore essential to determine the capacity of national agencies to sustain particularly recurrent costs over the long term. When incremental costs as well as benefits are likely to be substantial, some form of continuing external assistance may be warranted, perhaps on a declining scale, until the programme stabilizes.
Vitamin A deficiency resulting in eye disease remains a serious public health problem in many poor countries. The evidence that it also can affect mortality and severe morbidity in infancy and early childhood further strengthens the case for comprehensive and effective prevention. An array of technically effective intervention measures are already available, with the potential for global control of the appalling effects of the disorder. However, decisions to apply those technologies on a large scale are often made without adequate attention to the programme context. Strategic planning may ignore the potential of operations research to reveal the cost-effectiveness and efficiency implications of programme choices and how to optimize them. In some cases, suitable mechanisms for translating small-scale projects into large-scale programmes are simply not yet in place; the health sector is not alone in its experience with successful pilot projects that never reached operational maturity.
What is needed is a better marriage between available control technologies and the essential systems for delivering them efficiently and on a meaningful scale. Where the delivery systems are weak, strengthening them is critical to successful large-scale interventions. Only by focusing as much attention on the application of the technology as on the technology itself can we be sure that VITAD interventions will be translated into effective and sustainable large-scale action.