Contents - Previous - Next

References

1. Paul B. Health, Culture and Community. New York: The Russell Sage Foundation, 1955.

2. Kottak C. When People Don't Come First: Some Sociological Lessons from Completed Projects. In: Cernea M., ed. Putting People First: Sociological Variables in Rural Development. Washington, D.C.: Oxford University Press, The World Bank, 1985: 325-356.

3. Cernea M. Putting People First. New York, London: Oxford University Press, 1985.

4. Cernea, M., ed. Putting People First: Sociological Variables in Development Projects. New York: Oxford University Press, 1991.

5. Cernea, M. Re-tooling in Applied Social Investigation for Development Planning: Some Methodological Issues. In: Scrimshaw NS, Gleason GR, eds. Rapid Assessment Procedures: Qualitative Methodologies for Planning and Evaluation of Health Related Programmes. Boston, MA; International Nutrition Foundation for Developing Countries, 1992: 11-33.

6. Chambers R. Pacey A, and Thrupp L, eds. Farmer First: Farmer Innovation and Agricultural Research. London: Intermediate Technology Publications, 1989.

7. Kendall C. Public Health and the Domestic Domain: Lessons from Anthropological Research on Diarrheal Diseases. In: Coreil J and Mull DJ, eds. Anthropology and Primary Health Care. Boulder: Westview Press, 1990: 173-195.

8. Arensberg CM and Neihoff AH. Introducing Social Change: A Manual for Community Development (2d ed.). Chicago: Aldine-Atherton, Inc., 1971.

9. Desowitz RS. How the Wise Men Brought Malaria to Africa: And Other Cautionary Tales of Human Dreams and Opportunistic Mosquitos. In: Klein N. ed. Culture Curers and Contagion. Novato: Chandler & Sharp Publishers, Inc., 1979: 64-71.

10. Miner H. Culture Change Under Pressure: A Hausa Case. Human Organization, 1960; 19(3): 164-167.

11. Chambers R. Participatory Shortcut Methods of Gathering Social Information for Projects. In: Cernea MM, ed. Putting People First: Sociological Variables in Development Projects. New York: Oxford University Press, 1991: 515-537.

12. Scrimshaw SCM and Hurtado E. Rapid Assessment Procedures for Nutrition and Primary Health Care: Anthropological Approaches to Improving Programme Effectiveness. Los Angeles: UCLA Latin American Center, 1987.

Spanish edition 1988, French edition (Kidima, Scrimshaw, and Hurtado) 1991, Chinese edition 1991, Portuguese edition forthcoming 1992.

13. Bentley ME, Pelto GH, Straus WR, Schumann DA, Adegbola C, de la Pena E, Oni GA, Brown KH, Huffman SL. Rapid Ethnographic Assessment: Applications in a Diarrhoea Management Program. Social Science and Medicine, 1988; 27(1): 107-116.

14. Chambers R. Rapid Rural Appraisal: Rationale and Repertoire. Public Administration and Development, 1981; 1: 95-106.

15. Beebe J. Rapid Rural Appraisal: Evolution of the Concept and Definition of Issue;. In: Proceedings of the 1985 International Conference on Rapid Rural Appraisal: Rural Systems Research Project and Farming Systems Research Project, Bangkok: Khon Kaen University, 1987: 47-68.

16. Grandstaff TB and Grandstaff SW. A Conceptual Basis for Methodological Development in Rapid Rural Appraisal. In: Proceedings of the 1985 International Conference on Rapid Rural Appraisal: Rural Systems Research Project and Farming Systems Research Project, Bangkok: Khon Kaen University, 1987: 69-88.

17. McCracken J, Petty J, and Conwey G. An Introduction to Rapid Rural Appraisal for Agricultural Development. London: International Institute for Environment and Development (IIED), 1988.

18. Scrimshaw SCM and Hurtado E. Field Guide for the Study of Health-Seeking Behaviour at the Household Level. Food and Nutrition Bulletin, 1984; 6(2): 27-45.

19. Epstein TS. A Manual for Culturally Adaptive Market Research (CMR) in the Development Process. East Sussex: NWAL, 1988.

20. Special Issue on Rapid Epidemiological Assessment. International Journal of Epidemiology. 1989; 18 (Supp. 2).

21. Scrimshaw SCM. Combining Quantitative and Qualitative Methods in the Study of Intra-household Resource Allocation. In: Rogers BL and Schlossman NP, eds. Intra-household Resource Allocation. Food and Nutrition Bulletin, Supp. 15. Tokyo: United Nations University Press, 1990: 186-198.

22. Pelto PJ and Pelto GH. Anthropological Research: The Structure of Inquiry. New York: Cambridge University Press, 1978.

23. Patton MQ. Qualitative Evaluation and Research Methods (2d ed.). London: Sage Publications, 1990.

24. Scrimshaw SCM. and Hurtado E. Anthropological Involvement in the Central American Diarrhoeal Disease Control Project. Social Science and Medicine, 1988; 27(1): 97-105.

25. Scrimshaw SCM, Carballo M, Ramos L, and Blair BA. The AIDS Rapid Anthropological Assessment Procedures: A Tool for Health Education Planning and Evaluation. Health Education Quarterly, 1991; 18(1): 111-123.

26. Munguti KJ. A Study of Knowledge, Attitudes and Practices Relating to Visceral Leishmaniasis (Kala-Azar) in Tseikuru Location, Kitui District, Kenya. Thesis for Master of Arts Degree, University of Nairobi, Kenya, 1986.

27. Scrimshaw SCM, Mitzner K, and Scrimshaw N. eds. Health Seeking Behaviour: Household Perspectives on Primary Health Care From Sixteen Developing Countries. Tokyo: United Nations University, in press.

28. Long A, Scrimshaw SCM, and Hurtado E. Rapid Assessment Procedures for Epilepsy: Anthropological Approaches for Program Development and Evaluation. Landover: Epilepsy Foundation of America, 1988.

29. Scrimshaw SCM, Carballo M, Carael M, Ramos L, and Parker RG. HIV/AIDS Rapid Assessment Procedures: Rapid Anthropological Approaches for Studying AIDS Related Beliefs, Attitudes and Behaviours. Boston, MA: International Nutrition Foundation for Developing Countries (INFDC), in press.

3. Qualitative and quantitative: Two styles of viewing the world or two categories of reality?

The origins of survey research
The problems and limitations of survey research
Qualitative or quantitative: Two styles of viewing the world or two categories of reality?
Combining quantitative and qualitative methods: Triangulation
The pros and cons of rapid assessment methodologies (RAM)
References

By Duncan Pedersen

Duncan Pedersen is affiliated with the International Development Research Center (IDRC) in Canada.

This paper skillfully sets parameters for discussions of quantitative and qualitative research. The author reviews the origins of systematic data collection research, outlining the early recognition of the power of information for control and the growth of information collection as a specialty. He also discusses the differing epistemiological and ontological assumptions behind qualitative and quantitative methods. A growing acceptance in development circles of rapid assessment methods is shown despite the continued bias toward quantitative data in the social science community. This paper describes each methodology well without choosing one as a superior method among the quantitative, qualitative and integrative pragmatic approaches. It addresses a number of important and commonly made suppositions of the differences in methodological approaches. These include the facts that surveys are in fact often more "rapid" and more easily designed and implemented than RAP-based studies. Researcher bias, often seen by "RAPers" as an exclusively quantitative issue, is brought home to RAP as well. In its conclusions, the paper cuts through many of the more frequently debated but less important issues concerning RAP. The author contributes substantially by concentrating his analysis on those basic tenets that should most concern those who gather information in and about communities. - Eds.

SOCIAL RESEARCH ON the health-disease process and health services delivery programme and evaluation has often been characterised as two opposing approaches.

On the one extreme there are those who, in their attempt to identify causes and disease distribution in the social and natural environment, tend to simplify reality to such an extent that the complex network of factors and the human experience of illness is lost in the search for establishing empirical generalisations for the sake of presenting reliable results. Generally speaking, this approach is identified as the quantitative-experimental and deductive model, and is based on the paradigm of the natural sciences. Hence the tendency to use numbers as a language (hard data), disregarding the subjective and phenomenological human experience. Emphasis is placed on the explanation of phenomena from the point of view of researchers, that is, from the outside (etic approach).

At the other extreme, there are those who conduct research using only qualitative methods. This approach is based on the social sciences paradigm that aims at understanding the human dimensions of the phenomena through qualitative research whose language is mainly verbal (soft data); its methods are non-intrusive, naturalistic and inductive. Emphasis is placed on coming to terms with reality from the actor's point of view, from within (emic approach).

I would like to make it clear from the outset that I do not intend to de fend either of these approaches, nor to promote an "infallible" methodological quantitative-qualitative blend. Let me state clearly at the beginning: the presentation that follows is based in two premises: first, that what researchers do is essentially very straightforward: to look, to ask and to read, and occasionally to think. Observation, interviews, questionnaires and other tools, under the title of research methods, are not necessarily quantitative or qualitative per se. Second, any attempt to quantify involves a qualitative judgement, and vice-versa. Qualitative statements imply a certain hierarchy, number and magnitude that give form to meaning [1].

In the following paragraphs, I will be referring to the origin of surveys, the limitations of various methods, and to the antagonistic, reciprocal and complementary relationships existing between quantitative and qualitative methods in health research. Finally, I will take a critical look at rapid assessment methodologies and review some of their premises.

The origins of survey research

The systematic use of registers of vital events and population surveys for health planning and assessment is a relatively recent phenomenon whose remote origins lie in population enumeration for tax or military recruitment purposes. Much later, the use of census and surveys broadened substantially in order to meet the political and administrative needs of ever-expanding colonial powers.

In the 1930s, and since the Second World War in particular, surveys became much more rigorous and adopted the scientific method [2].

Whereas surveys in the industrialised nations are carried out to determine the population's opinion of a given aspect of social and political life, in countries of Africa, Asia and Latin America they have been used for other reasons, related to the process of domination, control and the exploitation of resources. It is quite clear that the first colonial nations, and later the countries of the North, needed specific base line information about native populations - demographic data and information about what people say or do, have or have not - for the purpose of planning and executing administrative functions, and in this way to establish the terms for economic and cultural exchange, or religious and ideological imposition or domination. The implicit purpose of the surveyors was to collect information from those who had no decision-making powers in order to make decisions for them. For this purpose, political authorities, administrators, the military, religious orders and later, of course, social scientists and health professionals, all collaborated.

Despite the popularity of survey research, it soon came to light that what is needed to survey and measure in one population, ethnic group or social class, is not necessarily valid in another population, ethnic or socioeconomic group. The questions that the surveyors asked over and over again were whether the same survey instruments could be used in different population groups and contexts. Later, when the need to adapt or develop new survey instruments became evident, surveyors asked themselves whether data collected in this way were reliable and comparable, and moreover, whether the results could be extrapolated to the general population. The various sampling strategies and the application of standardized questionnaires in cross-sectional surveys are two techniques developed to facilitate the generalisation of results; to reduce errors and biases; to shorten time frames and lower costs; and to establish a basis for comparison between the population under survey and the referral group.

I would like this historical perspective to reveal, not only how methods and techniques evolved as societal aims changed, but more importantly, to stress the transformation that came about with the process of appropriating knowledge. In other words, research evolved from a systematic and isolated observations into a systematic series of observations and comparisons, with increasingly technical interpretations of reality. This process led to the transfer of the power to create knowledge from the people to those who held the required skills and controlled the application of methods and techniques [3].

The problems and limitations of survey research

The dissatisfaction experienced with conventional approaches to survey research led to the search for new approaches, strategies and alternative research methods. I would like to review some of the general limitations recognised by different researchers [3] and point out others more specific to health research.

One general limitation attributed to survey research is the oversimplification of social reality. The arbitrary design of questionnaires and multiple-choice questions with pre-conceived categories represents a biased and overly simple view of reality. Individual responses to questions lead to the arithmetic manipulation of figures, creating frequencies, averages and rates that represent "average replies", ratios or proportions that carry no real significance on their own, and rather mystify reality (e.g., 37.5% of respondents reported a health problem within the last two weeks). Cross-sectional surveys lead to the reading of a static or "photographic" image of what is, in reality, an interactive and dynamic process.

A second important concern in survey research has been the problems related to validity and reliability of results. The inconsistency of collected data can be attributed either to the dynamic and genuine variability or fleeting occurrence of the phenomenon observed (e.g., blood pressure, morbidity episodes, attitudes, etc), or to the lack of truth or consistency in the replies given. Even when questions are correctly formulated and well-intentioned, they often end up being inadequate or even irrelevant with respect to the culture and values of the respondents. Survey research techniques are clearly blemished by prejudices, or influenced by the ideology and value system of the researchers. Although sampling strategies and changes in questionnaire construction have improved the application and acceptability of surveys, they have proved once again to be insufficient in overcoming these prejudices.

Today, a large proportion of health research corresponds to cross-sectional surveys and KAP (knowledge, attitudes and practices) studies, on samples of rural or urban populations undergoing acculturation and rapid change. These studies often involve the collection of information about births, deaths and family reproduction history, food availability, distribution and intake, child-rearing and child-care practices, sexuality, contraceptive use and abortion, income, use of drugs, alcohol and tobacco, defecation and the disposal of waste, and other more or less intimate or "clandestine" behaviours. Survey research often demands clear-cut answers to questions related to illness perception, beliefs, health-seeking behaviours and therapeutic usages, and reasons for using or not using available health technologies and services. This kind of survey study, which explores the intimate and discreet behaviour of everyday life, leads to questionable results, and about half of collected data are considered erroneous or misleading [4], and therefore of poor reliability and dubious validity.

The use of closed questionnaires and pre-coded forms often elicit an incorrect, evasive or deliberately wrong answer. There are few references to the occurrence and importance of lying informants in survey research [5,6,7] and most agree that respondents do not lie without good reason. Often, lies are resorted to as a mechanism of escape from an embarrassing situation, created by the subject the question evokes.

Many authors have reported that information given by mothers on past illness episodes of their children, health care and child-rearing practices, or health services utilisation, presents such large discrepancies from reality that about a third of all responses should be invalidated. The conclusions speak for themselves: the reliability of responses given by any segment of the population decreases with the lengthening of the recall period (telescopic memory effect), and whether, because of omission, imprecision or deliberate distortion, half of the time what is reported bears little resemblance to actual behaviour [8].

In synthesis, survey methods are an effective tool in collecting objective data, but "weak and wasteful" in collecting subjective and attitudinal data, particularly when dealing with illness beliefs and health behaviour. Experience in using health surveys on populations in Third World countries has demonstrated additional limitations in their application, and problems with regard to reliability and validity of data. It is regrettable that all the rigour and expense involved in study design and stratified random sampling is actually wasted if the data collected are of poor validity, leading to unreliable results [9]. The point in question is whether this is due to the conceptual perspective, the type of questions and methods used, the researchers themselves, the culture of the respondents, or to a combination of all these factors. To this debate, we can add two epistemological approaches, one based on the social sciences and the other on the natural sciences' paradigms: the qualitative and the quantitative.

Qualitative or quantitative: Two styles of viewing the world or two categories of reality?

The polarisation of the debate between the qualitative and the quantitative has centred on the capacity of the data, as collected by one or the other method, to describe, understand and explain social phenomena. A growing number of researchers have adopted a more eclectic position from which they claim that no method per se has the monopoly on inference. They argue that quantitative and qualitative approaches should not be considered antagonistic, but rather as complementary [10, 11].

Within this debate of qualitative vs. quantitative, three stereotypes have evolved: the purists, the eclecticists and the integrationists or pragmatics [12].

The purists or segregationists, argue that quantitative and qualitative paradigms are incompatible, deriving "...from different, mutually exclusive epistemologic and ontologic assumptions about the nature of research and society" [13]. From this perspective, methods are based on opposing assumptions and therefore lead to profoundly different visions of the world.

The eclectics claim that both approaches are valid. The application of one or another depends on the situation, and although both sets of methods can be used complementarily for the study of the same subject, they still represent distinct assumptions. Many evaluative studies advocate this perspective, where quantitative and qualitative methods are used in parallel or sequentially, as the situation dictates.

Finally, the pragmatics or integrationists maintain that there is a false dichotomy between the quantitative and the qualitative, and argue for the integration of both methodologies for the same study subject. From this position, polarization is seen as representing the extremes of a continuum along which there is a gradient of possible combinations of quantitative and qualitative methods, both supporting each other and enhancing the credibility of study results.

In the health field in general, and in epidemiology in particular, there is growing consensus that applying both sets of methods in an iterative mode can increase the reliability of data and lead to a more complete understanding of the phenomenon under study. The combination of methodologies for the study of the same phenomenon has been coined "triangulation" [14].

Combining quantitative and qualitative methods: Triangulation

Some researchers experienced in using a combination of qualitative and quantitative methods have reported effects and results that transcend mere complementarily. Blending and integrating methods and data in studying the same phenomena can "...capture a more complete, holistic and contextual portrayal" of the subject under study, by eliciting data leading to new hypotheses or conclusions, for which single methods would be blind.

In most research designs using triangulation methods, Jick and other authors have pointed out the hidden assumption of triangulation: that the weaknesses and limitations of each individual method will be counterbalanced by the other method, exploiting the assets, and neutralizing, rather that compounding, the liabilities [15].

The use of multiple methods in research has been applied for over a decade. Its introduction into the health field has been relatively recent, and yet there seems to be reluctance to accept it. This is partly because the dominant paradigm of the natural sciences in biomedical research. has looked with disdain at the use of qualitative methods proposed by social scientists. On the other hand, norms and requirements of scientific publications have placed rigid criteria for review and acceptance of manuscripts, introducing biases in the selection of studies for publication. Refusal is often based on quantitative criteria: "lack of replicability," "small samples" invalidating generalisation of results, or "no statistically significant differences."

Furthermore, most existing research training manuals emphasise the use of single methods, either quantitative or qualitative, and there is a lack of instructional material that will guide students and researchers in the actual collection, analysis and interpretation of data from different perspectives using the "triangulation" approach. The analysis of qualitative-quantitative data requires experience and skills in the processing and interpretation of both "hard" and "soft" data.

The pros and cons of rapid assessment methodologies (RAM)

To conclude, it is worthwhile pointing out some areas of the advantages and disadvantages in the application of qualitative RAM to health research. In order to do this, we should call to mind the premises that uphold rapid assessment strategies and ask ourselves what the benefits are in proposing an abbreviated time frame and a "new" set of field research tools in health and disease.

First of all, the introduction of rapid assessment methodologies should be recognised as an effective strategy for finding wider acceptance for qualitative and phenomelogical research in the scientific community and amongst health professionals. As we saw above, although the triangulation strategy has been applied successfully to health research, resistance is still met when it comes to adopting innovations in the use of qualitative methods. The incorporation of qualitative methods in health research is a sine qua non for expanding the conventional epidemiological and biomedical model; for re-orienting health plans and programmes; and for designing more effective health interventions and evaluative models.

It has been said more than once that RAM is just a bit more than organised common sense. Chambers rightly warned RRA (Rapid Rural Appraisal) enthusiasts of the dangers of superficiality and error in this method. Above all - affirms Chambers - rapid assessment techniques are not supposed to save time, but " ...should release time for more contact with and learning from the poorer rural people" [16].

Unfortunately, there are many examples of health research amongst high risk groups as well as longitudinal ethnographic studies of groups or ethnic minorities whose results are untimely and at times irrelevant for the groups under study. However, the duration of field work is probably only one of the reasons for this. It may take a year or more of intensive field work to complete a longitudinal study with participant observation, in-depth interviews, and prospective follow-up of illness episodes and therapy-seeking behaviour. Conversely, it usually takes a couple of weeks to design a survey questionnaire, and once the sample is drawn, only a few days (depending on resources available) to complete it. The former is labour-intensive and requires a great deal of personal commitment, and long exposure to the field conditions; the latter is much easier to apply, to analyse, and more likely to be published and disseminated.

In reviewing the premises on which "rapid" methods of research and assessment are based, it is claimed that prolonged field work leads to the unnecessary accumulation of ethnographic material that is not always relevant to the subject under study. As a result, it is proposed that field visits be shortened and efficiency increased so that only information considered necessary is collected. In order to do this, a list of subjects considered universally relevant is drawn up, and recommendations are made for the combined use of quantitative and qualitative methods in the collection of data. The underlying assumption is that the adherence of the researcher to this "prescription" or list of subjects, and the use of combined methods, will render data collection both efficient and reliable.

This is an incomplete premise, and today we should take it upon ourselves to review this partial (and reductionist) concept of scientific research that often leads to an oversimplification of reality. A broader approach departs from the assumption that each phase or stage in the research process is in dynamic interaction with the other phases and components, and simultaneously, with the whole. Once again, we have to insist on the adoption of an "expanded" view of scientific rigour and the research-evaluation process as a whole [17].

Scientific rigour in research cannot (and should not) be restricted to the discussion of data collecting methods, nor to the efficiency or rapidity with which it is carried out. Therefore, scientific rigour should not be tied clown to the selection of techniques and the proportions to which the quantitative and qualitative methods be applied, but rather to the quality of decisions that researchers make throughout the research process. The definition of the problem, the conceptual framework, the generation of hypotheses, field work and the selection of informants, and the analysis and interpretation of results all form an integrated whole, to which data collecting methods and instruments are added.

Moreover, the RAM approach assumes that other prejudices and sources for error in the research process, such as the ideology and value system of researchers and informants, either do not exist, or are neutralised by the effects of the method, and by the type of information gathered.

Information collected cross-sectionally and over a short period of time may be efficient from the researcher's viewpoint, but it runs the risk of being incomplete and of presenting a static image of reality. Research in general "...should be a dialectic process, a dialogue over a long period of time'' and the "dialogue" cannot be restricted to certain stages such as the collection of data. The process of collecting and interpreting is iterative, which is why it should be done on a continuum, each stage helping the other. This does not mean we should extend field work and analysis indefinitely, but the time allotted should be sufficient to allow for the analysis of information in situ and, if necessary, to return to gather additional data.

Another unsolved problem with regard to RAM is the interpretation of information and the use of data collected. Various alternative routes can be followed in the interpretation of data. Results can be laid out in such a way that they describe a programme or interpret a health problem or assess the impact of an intervention, but data have to be analysed not only in order to understand, evaluate and explain reality, but also to transform it.

Finally, the generation of new methods, and the substitution of some research techniques with others does not solve the problem of the monopoly of knowledge. I would like to emphasize here that all research and assessment (rapid or conventional) should involve the people and the community who have up to now been excluded from the process. This leads us to ask once again: What do we really need in order to conduct health research? Do we need a greater number of researchers qualified in the application of more sophisticated scientific research techniques?

The application of RAM should not underestimate the knowledge and experience of local researchers and informants. The participative research approach, which brings together decision-makers, professional researchers and representatives from the local community in the research process, represents a valid alternative for increasing efficiency, reducing time frames, ensuring timely feedback and the democratisation of the process of the production and utilisation of knowledge.

Contents - Previous - Next