Previous - Next
Section introduction
32. Use of rapid assessment procedures for evaluation by UNICEF
33. Institutionalizing the use of rapid assessment procedures in rural service agencies
34. RAPing in Chad
35. From qualitative community data collection to programme design: Health education planning in Niger
36. Use of rapid assessment procedures for nutrition programme planning, project reorientation, and training in Malawi
The papers in this section
look at RAP from the perspective of researchers within the donor
community which have funded many RAP studies and training
workshops. Aubel discusses the issues of institutionalization
from within a national setting and also in an organization such
as the World Bank. She finds encouragement amid numerous
constraints. Pearson and Kessler add the perspective of UNICEF,
one of the strongest supporters of the development and use of
RAP. They call for a strong pragmatic approach and describe the
many uses to which their organization has put methods that they
see as falling within a broad and highly practical framework for
RAP. Watson's paper was warmly received at the conference as she
eloquently presented an analysis of the achievements and
constraints on institutionalizing RAP in Chad. She points out the
difficulties both within the Government and within her own UNICEF
office to an innovative methodology such as RAP. Aubel presents
another analysis of the difficulties and strategies needed to
overcome them in having RAP accepted in Niger as a new tool for
planning. Berggren concludes the section with a description of
the use of RAP in Malawi where the research process built in an
iteration project planning that included a loop back to the
community. The use of RAP in training and sensitizing planners
and project workers to community problems provides another
dimension to institutionalization of these methods.
Background on the evolution of RAP and its uses in UNICEF
Use of RAP in evaluation in UNICEF
Conclusion
Acknowledgement
Endnotes
References
By Roger Pearson and Susi Kessler, M.D., Evaluation Office, UNICEF, New York1
Roger Pearson is a Senior Evaluation Officer at UNICEF. Susi Kessler, M.D., is a Senior Advisor to the Central and Eastern European Unit of UNICEF.
More than any agency other than the United Nations University, UNICEF has supported development and use of RAP. UNICEF has funded several RAP workshops, assisted work on RAP guidelines, and supported the videotape used to promote RAP. UNICEF both supports use of RAP and just as importantly, orients its staff in this methodology.
The RAP conference had over 20 participants from UNICEF field offices and its Evaluation Office in New York. Papers by UNICEF staff members are contained in this volume and several authors refer to UNICEF assistance to their studies or training.
This paper reflects the professional experience of Parson and Costlier in RAP-related staff training and use of qualitative techniques in project evaluations and assessments in Asia, Africa, the Americas, the Middle East, Central and Eastern Europe, and the Commonwealth of Independent States. It also shows the pragmatic, action-orientation of UNICEF, and the increasing professionalism in qualitative research among UNICEF staff. Eds.
DURING THE PAST ten years we have, in UNICEF, increasingly been using rapid assessment procedures in our evaluation work. We have also collaborated closely with a number of institutions to expand the approach and make it more relevant to UNICEF's field work. We have found RAP to be useful for providing information to decision makers at every stage of the programming cycle - design, monitoring and evaluation. Our use of RAP is eclectic and reflects the fact that we see it as a process which is relevant for problem solving.
We view RAP not as a technique, but as an approach which utilizes many methodologies - many tools - both qualitative and quantitative. We view it as a flexible constellation of means for investigating. We use it to make a diagnosis of situations, to help us in problem solving, in getting data for decision making, to identify how programmes can be improved and what lessons can be shared. We see it as a useful approach - not only because of rapidity (rapidity, as a number of presenters have noted, is relative) but because it is relevant. It allows us to include the viewpoints and opinions of those people for whom programmes are intended, beneficiaries as well as policy makers and service providers. The assessment process and the decisions that may be affected by it thus can be shared and thereby contribute to the empowerment of those who are affected.
We also find RAP important because it is an approach which provides information but which in the very process of acquiring information can alter the situation. While conducting a RAP, evaluators involve participants and give them feedback. They also provide expert consultation and often build consensus by bringing people together. Thus the use of RAP can be part of the change process.
What do we consider some of the hallmarks of the RAP process?
1. RAP IS ACTION ORIENTED. Information gathering through RAP is specifically geared to programme improvement, problem solving, decision making, extension of experience. This explicit focus casts the data gathering process in a pragmatic light. Rapidity is important in this context. If findings are to be used to modify actions, it is useful to obtain the information before the questions are forgotten or while decision makers remain highly motivated.
2. RAP IS INVESTIGATIVE. It attempts to discover new information or find new interpretations in addition to testing hypotheses. It attempts to discover not only what is happening but why it is happening, or in some instances, not happening. It looks both for the anticipated and the unanticipated. The process allows the evaluator to follow leads, check discrepancies, and obtain many different viewpoints from which a holistic picture can emerge.
3. RAP IS PROCESS ORIENTED. It analyzes not only the end results but the process of getting there. It attempts to discern the facilitating factors and the constraints.
4. RAP ATTEMPTS TO ASSESS SITUATIONS HOLISTICALLY. The process looks at many angles of a situation from many perspectives. It places problems in context; most quantitative assessments by their nature tend to be reductionist, looking at a limited number of variables. RAP also seeks to look at interactions, patterns and evolution over time.
5. RAP DERIVES CONFIDENCE from the fact that multiple key informants are involved. By piecing together information and insights from various informants RAP team members are likely to obtain a "true" picture. Key informants include a broad range of people - policy makers, managers, service providers, both within and outside the sector - and most importantly beneficiaries are also principal key informants.
6. RAP PLACES GREAT IMPORTANCE ON "INFORMED JUDGEMENT". It attempts to ensure that judgements will be informed by putting emphasis on observing, describing, listening, and getting different viewpoints. It encourages arriving at collective judgements, often by consensus.
7. RAP IS EFFICIENT by emphasizing optimum use of existing data, and collecting and analyzing available information and documentation. RAP may include multiple techniques - qualitative and quantitative.
8. RAP EMPHASIZES THE USE OF INTERDISCIPLINARY TEAMS. Because perceptions and interpretations are often conditioned by disciplines, RAP emphasizes bringing together experienced observers with different backgrounds and expertise. We put great importance on assembling a team with multiple skills and identifying a team leader good at facilitating interaction and coordination.
9. RAP TEAMS INCLUDE "INSIDERS" AND "OUTSIDERS". We find it useful in most teams to include both. The insiders, both UNICEF and government, familiar with the project or programme, bring in-depth knowledge, comment on practicality of recommendations and above all are more likely to use recommendations if they have been involved in the RAP. Outsiders on the other hand, specialists familiar with the field or country or culture, can bring fresh, less biased perspectives.
10. RAP CAN FACILITATE COMMUNITY INVOLVEMENT. All members of the community are viewed as potential informed observers and can be drawn into the investigative process. Open ended interviews allow people to express opinions, offer suggestions, make recommendations. "Listening and learning" are emphasized. The process is interactive. Findings and recommendations are presented and discussed with programme planners and implementors so their views can be taken into consideration. The process promotes dialogue.
Much of UNICEF's work in
developing countries involves, hand in hand with government
agencies, the planning, management, monitoring and evaluation of
development programmes.
In the 1970s, as development aid expanded, there were two main ways of getting information to plan, monitor and evaluate the work of development agencies. There was the field trip - an activity carried out by professional development workers themselves - and there was long drawn-out research, often involving centrally planned sample surveys, and with gestation periods measured in months or sometimes years; consultants from developed country universities usually carried out the latter. Both ways of collecting information are, of course, extremely useful. RAP has evolved as a third and complementary process. To understand the need for RAP, it is worth dwelling on the pitfalls of field trips and on more academic research.
Field trips are a useful way of getting a "feel" for things, and getting first hand information quickly. But this type of appraisal method came in for an increasing amount of criticism detailed by Robert Chambers in his book, Putting the Last First [1]. There was concern that the level of contact of development professionals (particularly people from large international development agencies) with poor people was being reduced to "development tourism". Most impressions of what was happening in a development programme were fleetingly obtained from looking out of the window of a speeding vehicle, along with all the biases inherent in this way of gleaning impressions. The more senior one became, the more one was trapped into this way of becoming informed.
Chambers listed a set of biases associated with the field trip that are worth repeating here, for they still apply today. Dry season bias - traveling to remote places only when the roads are in good condition; spatial bias - only seeing what is happening close to roads, especially tarmac roads, and seeing more of life in urban compared to rural areas; project bias - being directed to where development funds are being spent, particularly towards showpiece projects; person bias - listening to and observing only the people with whom one comes into contact during the fleeting visits; people who are fit, still alive, male, elite and using the service being developed; diplomatic bias - visitors being deterred by combinations of politeness and timidity from coming into contact with poorer people; professional bias - professionals tending to look and find what they look for in their own narrow field of specialization where a more holistic approach might be called for.
The other main way of deriving information on what was happening - large centrally planned and executed surveys, whether of a quantitative or a qualitative nature - was often useful, even indispensable, but sometimes produced misleading, irrelevant, obtuse, or late information. Reports were often written in a way that was difficult to interpret. Again this observation still applies today2.
So in the late seventies the feeling started to arise that what little time was available to interact with the public, especially the rural poor and people living in urban slums, could be put to use in a more systematic manner; that ways could be found to be able to listen more closely to what people had to say, to learn from them; for the public to be able to take a greater part in deciding course changes for development programmes that affected their lives; that another middle way between the fleeting field visit and the more detailed methods could be devised. Rapid assessment procedures (RAP) grew out of this need.
Another motivation stemmed from the wish of UNICEF staff to have more opportunity for direct contact with populations. UNICEF staff tend to be social scientists, although a large proportion are engineers or from backgrounds in the humanities. The nature of life for UNICEF staff directly involved with development work is that of a member of the middle classes, normally living in a well-off neighbourhood in a capital city. Most staff members spend their time in offices, or sometimes in provincial towns attending to administrative details. They have little time left for work at the front end of development work with the very people whom UNICEF is trying to help.
Table 1. Some of the Characteristics of Rapid Assessments
|
Exposing
UNICEF staff in a methodologically rigorous way to the rapid
anthropological methods of collecting information, as laid out in
the Scrimshaw and Hurtado manual [2], has also been used to
familiarise junior staff on how to take a more rigorous approach
during field trips (UNICEF staff are encouraged to spend about 20
percent of their time on field trips) so as to avoid Chamber's
pitfalls as much as possible.