Contents - Previous - Next

This is the old United Nations University website. Visit the new site at http://unu.edu

II. Methodological approaches to measurement

Methodological approaches to measurement
6. Combining quantitative and qualitative methods in the study of intra-household resource allocation
The meaning of cultural things
7. An approach to the study of women's productive roles as a determinant of intra-household allocation patterns
8. Household organization and expenditure in a time perspective: social processes of change
Micro-social research on household organization and expenditures in buenos aires, Argentina

Methodological approaches to measurement

BEATRICE LORGE ROGERS AND NINA P. SCHLOSSMAN

Part I discussed a set of conceptual frameworks for identifying important factors in intrahousehold resource distribution. Economics, anthropology, and psychology focus on different aspects of the allocation process, tend to identify different sets of variables as central to the process, and so tend to use different approaches to the measurement of these variables. In this section, three papers address methodological issues related to specific techniques used by different social-science disciplines to obtain the most complete understanding of intra-household allocation patterns and their determinants.

Scrimshaw's paper proposes an integration of qualitative methods such as those used in traditional ethnographic studies with more quantitative data-collection methods characteristic of survey research in economics. Each approach, the qualitative and the quantitative, has both advantages and disadvantages; combining the two strengthens both. In fact, the sequence from qualitative description to structured direct observation to survey research is now the method of data collection commonly followed by anthropologists who recognize the importance of statistical reliability and by economists who recognize that they cannot construct adequate models of behaviour without identifying and incorporating culturally specific variables.

Scrimshaw outlines a sequence which might be followed to obtain information for the assessment of programme impacts on the household and its allocative processes. Data collection starts with exploratory ethnographic work, followed by structured observations and guided but open-ended interviews, which lead to the design of structured surveys of statistically representative samples of the population.

Bennett, in her paper, describes an application of these combined methods to one particular problem: the effects of women's work on child welfare. She elaborates the research which would be required to address the various aspects of this question. In so doing, she demonstrates the effective combination of ethnographic investigation, direct, structured observation of behaviour, and quantitative, statistically representative data collection.

To these methods should be added a review of existing literature, which should precede the initiation of any primary data collection, and discussions with professionals - both scholars and development specialists - to obtain information and insights about the location, culture, and environment in advance of field-work. Both published and unpublished literature in the fields of anthropology, sociology, economics, and the specific subject area of the programme or project being assessed (e.g. health, food, nutrition, or agriculture) should be reviewed for pertinent information.

Secondary data is an important and often overlooked source of information for designing and evaluating programmes. Longitudinal, in-depth studies conducted by local research institutes, or by doctoral students from universities in the country or abroad, can be a rich source of information on the local culture and on household processes. Such studies cannot be accepted uncritically, of course, but they can certainly be used to develop questions and hypotheses for later empirical validation.

Jelín's paper in this section describes a study carried on in a small number of households over an extended period of years, using a time-intensive, personal method of gathering information. Her method of data collection is clearly unsuited to the constraints of a donor agency with a limited time horizon. None the less, the insights obtained from the study are of value in understanding how households use resources. Jelín demonstrates that the apparent irrationality of certain household consumption decisions has a logic of its own, based first of all on the expectation of continued high inflation (which makes debt a rational strategy), and secondly on the value of prestige, obtained from specific kinds of publicly observable consumption. She also reveals the importance of the regularity, timing, and reliability of income (at least from the household's perspective) in how the income is used. Once recognized, these factors could be built into more quantitative models of income and its uses. Moreover, the fact that small, regular increments of income are more likely to be used for such expenditures as food should have significant implications for the design of effective income or employment schemes.

Another means of obtaining information is to exploit existing large-scale surveys, such as population censuses, household income and expenditure surveys, health and other special purpose surveys, for what they can reveal about the household. Extracting information about households and their internal dynamics through the analysis of existing large surveys poses its own set of problems. In many countries, large data bases may contain useful data on households. The analysis of Indian census data by Rosenzweig and Schultz (1982) represents one imaginative approach. They compared survival rates of different cohorts of girls and boys, using the known expected ratio of girls to boys at different ages as a baseline, to draw conclusions about differential investment in male and female children.

To be applicable to the analysis of household processes, large data sets must contain disaggregated age/sex characteristics for each household. As Rosenzweig and Behrman indicate in their papers in this volume, individual employment data on each household member can serve as an accurate indicator of the alternative uses of each member's time, and of the total time constraints on the household. Individual education level can reveal both the investment in different categories of individuals and the economic value of their time. Surveys of income will be significantly more useful for household-level analysis if they contain information on individual sources of income (earnings, transfers, wealth income), and even more applicable if earned and wealth income can be associated with the appropriate individual in the household. Special purpose surveys, covering, for example, health, nutrition, or food consumption, often include valuable information on individual welfare outcomes, which then may be associated with individual characteristics and with household composition and income data.

Of course, interpreting the analysis of such large-scale surveys requires an understanding of the cultural context in which the data were collected. The culturally specific meanings of the variables must be identified before the analysis can be performed. Knowledge of the local culture is essential also for assessing the probable reliability and validity of the data. For example, numerous studies have found that women's market work is underreported in censuses and agricultural surveys because of cultural norms which hold that women are primarily housewives. Not just men, but the women themselves, may report that they (the women) are unemployed, even though they are actively engaged for significant amounts of time in market-oriented production. Another example is the definition of household headship. In many cultures, women are unlikely to be identified as heads of household, even if there is no adult male present. Husbands who are absent longterm, or even pre-school-aged sons, may be reported as household heads in such cases.

It should be noted that computer analysis of such large survey data sets is not cheap, easy, or quick. Typically a great deal of data cleaning is required, and additional modifications may be necessary before the data can be read by a large, main-frame computer. Access to the data itself may be difficult to obtain, especially if it is controlled by a government department or a local research institute. If the problems can be resolved, such secondary analysis is an invaluable tool. Just as data from large-scale surveys must be used with caution and with cultural sensitivity, so must designers of largescale surveys provide more careful training to their enumerators on culturally appropriate questions of definition, and on how to probe for accurate and meaningful responses.

In the Appendix (table A), we have suggested a sequence of activities to be followed in planning and evaluating the effects of development programmes on the household and its individual members. This series of steps starts with a careful specification of the activities anticipated by the project, and their expected relationships with the desired project outcomes. The likelihood of these relationships occurring can then be assessed using information from existing published and unpublished studies, expert opinions of scholars and development specialists, and field studies employing a combination of qualitative and quantitative methods.

These steps are an integral part of project planning. Moreover, the combination of methods discussed in these papers should be used not only in advance of project implementation, but also in the monitoring and long-term evaluation phases of development programmes. In reality, projects take place in a changing environment, and households may adapt differently to changes in the short and the long run. No substitute therefore exists for ongoing monitoring of intra-household processes as they change over the life of a project and beyond.

6. Combining quantitative and qualitative methods in the study of intra-household resource allocation

SUSAN C. M. SCRIMSHAW
School of Public Health, University of California, Los Angeles, California, USA

INTRODUCTION

Two fundamental methodological questions plague attempts to measure intra-household resource allocation. The first is how to measure accurately the actual behaviours, motivations, feelings, and outcomes related to intra-household resource allocation. The second is how to ensure an accurate understanding of the meaning of the behaviours and concepts to be measured. Both of these relate to validity, the accuracy of measurements. Only when validity has been established to a reasonable degree does it make sense to add a concern with reliability (representativeness) and replicability.

Quantitative methods used in isolation tend to jump ahead to a focus on reliability and replicability, but if validity is compromised these efforts are wasted on data which do not reflect reality. Qualitative methods can capture actual behaviour with great accuracy, and can produce detailed information and insights applicable to both the development of testable hypotheses and the interpretation of quantitative data. This is particularly important in cross-cultural research. For example, as Rogers, Messer, and Heywood discuss in other chapters in this volume, "household" and "family" are not synonymous. Family members important to intra-household resource allocation (such as grandmothers) may reside outside the household. Conversely, household members may not all be family members: they may be hired help, friends, or even anthropologists.

The importance of relationships and thus the power of individuals regarding food allocation will vary from culture to culture. To complicate matters further, cultural ideals (e.g. the male head of household makes all decisions regarding how money is spent) may be circumvented in actual behaviour (e.g. the female head is observed making purchases without his authorization). We cannot even make assumptions about the meaning and value of resources. Milk, for instance, is considered children's food in many cultures (and with reason in locations where lactose intolerance is prevalent), so it is not a resource which would interest adults.

This paper will first elaborate on the need to combine quantitative and qualitative research methods. It will then discuss the cross-cultural meaning of concepts such as family, allocation, and resources, and consider why these definitions matter in project design. It will conclude with specific suggestions for measuring the intra-household allocation of resources in different cultures.

WHY COMBINE QUANTITATIVE AND QUALITATIVE RESEARCH METHODS?

In general, quantitative research is assumed to mean large-scale survey methods, including mail surveys, self-administered questionnaires, and telephone or face-to-face interviews (Babble, 1982; Bailey, 1982; Sanders and Pinhey, 1983), while qualitative research is seen as traditional ethnographic field-work conducted by anthropologists and some sociologists and psychologists. Conventional images of qualitative research methods conjure up an anthropologist sitting in a village taking notes on everything including conversations, observed behaviour, life-histories, and material cultural objects (see table 1). In reality, most present-day ethnographers employ various combinations of participant-observation, observation-only, in-depth unstructured or semistructured interviews, and structured interviews and surveys (Pelto and Pelto, 1978; Spradley, 1979). Moreover, some observational techniques ethnographers use involve the quantification of many minute observations.

This is illustrated by the work of Johnson and others on time allocation (Johnson, 1975 and this volume), Harris and Dehavenon (Dehavenon, 1978) on video tapes of family hierarchies and food-handling behaviour in East Harlem, New York, and recent research on labour and delivery in women of Mexican origin in Los Angeles (Moore and Scrimshaw, 1983). Thus, the boundaries between quantitative and qualitative work can be nebulous, as ethnographers employ questionnaires and interview schedules or quantify detailed information collected through systematic observation (see table 1).

Researchers trained in disciplines which place a heavy emphasis on quantitative methods (i.e. economics, demography, sociology, and epidemiology) frequently do not understand the methods of qualitative researchers (i.e. anthropologists, some sociologists, and some psychologists) and consequently may view their work as unscientific, unreliable, or biased. Researchers who underrate qualitative methods do not realize that these often include random sampling, standardized questionnaires, and repeated observations which are then quantified. Furthermore, when samples are not random and formal interview schedules are not used, this is because the kinds of information (such as the meaning of an activity) needed at that point in the research are not readily or accurately available through quantitative means. Qualitative data are not, as some view it, a poor substitute for the "right way to do research." They provide instead an essential method in the array available to students of human behaviour.

The debate on the scientific value of qualitative versus quantitative research is well summarized by Pelto and Pelto (1978). They define science as "the accumulation of systematic and reliable knowledge about an aspect of the universe, carried out by empirical observation and interpreted in terms of the interrelating of concepts referable to empirical observations" (Pelto and Pelto, 1978, p. 22). According to this definition, the fields of anthropology, sociology, social psychology, and economics, which must come to terms with human behaviour, can join experimental fields such as phy sics in the category of "sciences." The Peltos add that "if the 'personal factor' in anthropology makes it automatically unscientific, then much of medical science, psychology, geography. and significant parts of all disciplines (including chemistry and physics) are unscientific" (Pelto and Pelto, 1978, p. 23).

Table 1. Basic ethnographic methods

Source: Adapted from Scrimshaw and Hurtado, 1987, p. 5.

In fact, scientific research is not truly objective, but is governed by the cultural framework and theoretical orientation of the researcher. This idea will be discussed later in the section on the cultural variations in the meaning of concepts to be measured. As Johnson comments: "It is a vexing ethnocentricism to assume that science is or even can be completely culture-free" (Johnson, 1978, p. 2). The issue then, is to be aware of one's research orientation and potential biases and to collect data as accurately and as objectively as possible.' There is hardly a consensus on how to do this, but, increasingly, researchers are arguing that a combination of qualitative and quantitative data-collection techniques is essential (Steward, 1950; Pelto and Pelto, 1978; Johnson, 1978; Van Maanen et al., 1982).

The differences between standard surveys (interview schedules or questionnaires) and the frequently more qualitative anthropological approaches (table 1) and their advantages and disadvantages are summarized in table 2.

The methodological concepts of validity and reliability provide a common foundation for the integration of quantitative and qualitative techniques. Validity refers to the accuracy of scientific measurement, "the degree to which scientific observations measure what they purport to measure" (Pelto and Pelto, 1978, p. 33). For example, in my work in Spanish Harlem, New York City (Scrimshaw and Pasquariella, 1970), the phrase "¿sabe como evitar los hijos?" (do you know how to avoid [having] children?) elicited responses on contraceptive methods and was used as the first in a series of questions on family planning. By not using family-planning terminology at the outset, we were able to avoid biasing respondents. The same question in Ecuador, however, produced reactions like "I would never take out (abort) a child!" If the New York questionnaire had been applied in Ecuador without testing it through semi-structured ethnographic interviews, the same words would have produced answers to what was in fact a different question. In another example, Zborowski's classic work on pain perception in different cultures showed that while Jews and Italians (in New York) both tended to verbalize feelings of pain to a greater degree than the other culture studied, the underlying meaning of pain was different. To the Italians, it was a discomfort which was quickly dismissed when analgesia solved the problem. Jews tended to reject analgesics because of concerns over side-effects and viewed pain as symptomatic of more serious health problems (Zborowski, 1952).

Table 2. Plusses and minuses of qualitative and quantitative methods

Source: Adapted from Scrimshaw, 1985.

Reliability refers to replicability: the extent to which scientific observations can be repeated and obtain the same results. This means that a study of household-level food-procurement behaviours in a community would yield the identical results with repeated trials. In practice, such replications are complicated by non-controllable factors such as seasonal change and alterations in the economic climate. Even if the first study identified factors leading to particular behaviours, it would be difficult to interpret variations in results from a replication since more than one confounding variable (or circumstance) could come into play. While all research ideally employs meticulous data collection to ensure maximum reliability and validity, in practice investigators must settle for less than perfection. Careful research procedures, including thorough training and monitoring of interviewers, are essential components that should be used in project planning and implementation to minimize error. In general, qualitative methods are acknowledged to be more accurate in terms of validity, while quantitative methods are considered to be better in terms of reliability or replicability.

Surveys are effective tools for collecting data from a large sample, particularly when the distribution of a variable in a population is needed (e.g. the percentage of women who obtain pre-natal care) or when rarely occurring events (e.g. neo-natal deaths) must be assessed. Surveys are also used to record people's answers to questions about their behaviour, their motivations, their perception of an event, and similar topics. While surveys are carefully designed to collect data in the most objective manner possible, they often suffer inaccuracies based on respondents' perceptions of their own behaviour, or their desire to please the interviewer with their answers.

Surveys have difficulty revealing motives (i.e. why individuals behave as they do), nor are they likely to uncover behaviours which may be consciously or unconsciously concealed. For example, programmatic attempts to follow up on the consumption of supplementary foods distributed to pregnant and lactating women and young children yield an inaccurate picture if the women alone are interviewed. They may give answers they know the interviewer wants to hear, rather than the facts (i.e. that the male head of household consumes the foods). Alternatively, individuals surveyed may be unaware of their actual behaviour and thus report it inaccurately. In Barbados, comparisons of responses to interviews with 48hour observations of actual behaviour revealed that women nursed their babies twice as often as they thought they did (Scrimshaw, 1969).

A less frequently recognized limitation to survey research is that results can be very different depending on who is interviewed. In Los Angeles, nurses thought Latina women came into the hospital "too early" or in "false" labour in greater proportions than other patients. Observations followed by postpartum interviews with women delivering at that hospital over a two-month period revealed, however, that Anglo women experiencing their first pregnancy were in fact the most likely to come in early. The nurses selectively remembered the Latina women because they were more "difficult patients" owing to language barriers and anxiety about pain and risk of death during childbirth (Scrimshaw and Souza, 1982). In this instance, participant observation helped to establish the most likely source of inaccurate information. More accurate data was available from the patients than from the nurses, although interviews and participant observation with both revealed the problem as seen from both sides, staff and patient.

In addition to the aforementioned difficulties in collecting accurate data, surveys may focus so narrowly on specific variables that they may fail to elicit important behaviours underlying a situation. Because anthropologists take a holistic approach which allows them to remain open to new information and to add categories to the data-collection guidelines, they can more readily identify these unanticipated links. For example, Caldwell and Caldwell (1977) point out that demographers examining African birth histories were aware of the limitations of their quantitative approach. They found they were unable to tell whether long birth intervals represented a real social constraint on fertility, defective sampling, or inaccurate reporting. Anthropologists, studying the same situation with the methods described in table 1, established that these intervals were in fact due to institutionalized postpartum abstinence, a cultural pattern with important consequences for total fertility rates (Page and Lesthaeghe, 1981).

In another classic instance, physicians and epidemiologists puzzled over the mode of transmission of the uniformly fatal degenerative nervous system disease, kuru, among the Fore people in the New Guinea Highlands. Kuru affected adult women and children and adolescents of both sexes proportionately more than adult men. By the early 1960s, the most accepted of the prevailing hypotheses was that it was genetically transmitted. Yet this did not explain the sex differences in infection rates in adults but not in children, nor how such a lethal gene could persist. Working with Gadjusek of the National Institutes of Health (NIH), cultural anthropologists Glasse and Lindenbaum used in-depth ethnographic interviews to establish that kuru was relatively new to the Fore, as was the practice of cannibalism. Women and children were more likely to engage in the ritual consumption of dead relatives, which was culturally less acceptable for men. Lindenbaum and Glasse suggested the disease was transmitted by cannibalism. In order to confirm their hypothesis, Gadjusek's team inoculated chimpanzees with brain material from women who had died of kuru and the animals developed the disease. Since then, the practice of cannibalism has declined and the disease has now virtually disappeared (Lindenbaum, 1971; Gadjusek et al., 1967).

Research on childbirth among Mexican women in Los Angeles illustrates how a combined quantitative and qualitative approach can work more effectively than either alone. The study was aimed at understanding the relationships between cultural and behavioural factors and the biomedical aspects of labour and delivery. The methods included observation, participant observation and ethnographic interviews, which yielded information and strategies used in the design of pre- and postpartum questionnaires. These were applied to a sample of 518 women delivering in two Los Angeles hospitals. A subsample (not random, but opportunistic) of 45 women was observed throughout their labour and delivery using pre-coded forms for obtaining five-minute time samples of behaviour. Cultural/behavioural factors such as acculturation, social support, anxiety, knowledge of birth, and use of pre-natal services showed logical and predictable relationships when survey data on the entire sample (518 women) were analysed. The medical variables also interrelated in expected ways: the use of pain medications predicted lower baby Apgar scores, labour complications predicted baby complications, and so on. However, the survey data provided few connections between the cultural/behavioural and the biomedical variables, even though we knew some links had to be there. For instance, Mexican women who had arrived most recently in Los Angeles had a higher probability of significantly longer labours, yet there was no evidence for any physiological differences by recency of migration. Three-quarters of the women had arrived in the United States within seven years of the study.

Our behavioural observations of labour and delivery of 45 women helped resolve this problem. In this subsample, the data showed cultural/behavioural influences on factors such as the use of pain medication in labour. We identified relationships between nonverbal behaviour, noise levels, nurse behaviour, and the use of pain medications. Noisier women initially obtained more nurse attention (measured by the amount of time nurses spent in the labour room and by verbal and non-verbal interactions with the patient), and were more likely to be given medication for pain relief. Once the women were medicated, high noise levels no longer were associated with more attention from the nurses (Moore and Scrimshaw, 1983). None of this observed behaviour emerged in the postpartum interview. Women seemed to gloss over the details of their labours. The interview method did not permit time to explore this in depth, nor could already long interviews be extended. A combination of survey and observation techniques greatly enhanced the value of the study results.

It is clear that there are advantages and disadvantages to either methodology alone (Scrimshaw, 1985), and great potential value in combining the two approaches. Johnson suggests that research must balance "the appeal of straightforward techniques to testing hypotheses by using statistics and quantitative analysis against the necessarily complex business of developing cross-culturally valid methods and concepts for the study of human behaviour" (Johnson, 1978, p. vii). The process of developing cross-culturally valid methods and concepts is necessarily dependent on understanding the meaning of the concepts to be measured within each cultural context. Projects which never seek or fail to achieve this understanding run the risk of generating data which reflect superficial findings rather than deeper realities.

Contents - Previous - Next