Contents - Previous - Next

This is the old United Nations University website. Visit the new site at http://unu.edu

Results

Comparison of absolute levels of prevalence

Table 2 shows the prevalence of underweight as estimated from clinic data and survey data in each of the seven clusters, for infants and children separately. In this table the survey results were statistically weighted to match the seasonal distribution of measurements in the clinic data. Among infants the clinic prevalence was significantly lower than the survey prevalence in three clusters, significantly higher in two clusters, and not significantly different in two.

TABLE 2. Prevalence of underweight among infants and children in clinic-based data and MMCN survey data

The survey data are weighted to match the seasonal distribution of measurements in the clinic-based data, and were done separately for each cluster and age group (see methods).

a. Z is the result of a statistical test of the difference between the two prevalences, which is significant at the .05 level if the absolute value of Z exceeds 1.96.

where P1 and P2 are the prevalence from clinic and survey data respectively, P is the overall prevalence, Q equals 1—P. and n1 and n2 are the sample sizes for the clinic and survey samples respectively.

TABLE 3. Prevalence of underweight among infants and children in clinic-based data and MMCN survey data- -data from period I only, December 1988 to March 1989

In absolute magnitude the estimates differed by a factor of two to three in all seven clusters. A similar picture was seen among children. in that the clinic prevalence was significantly lower than the survey prevalence in three clusters (with a fourth approaching significance with Z= - 1.85), significantly higher in one cluster, and not significantly different in two. Again, most of the prevalence differed by a factor of two to three between the two data sources.

Table 3 presents similar comparisons, but based only on data from the first period (December 1988 to March 1989). This period is singled out for comparison because that is when most of the child measurements (12-59 mo) were recorded in most clusters. This is because the child data used here represent the first visits in the current year, as distinguised from subsequent visits on the reporting form. This distinction helps avoid bias that might arise from using duplicate measurements on chronically sick or malnourished children.

The results are much the same as those in table 2, in that clinic prevalence differed significantly from survey prevalence in most clusters. In this case there was a greater tendency for the clinic data to underestimate grossly the survey prevalence in most clusters, as seen in the negative sign attached to the Z statistic in all cases except one. When the data from all three periods were used, the direction of the difference tended to vary from one cluster to another.

TABLE 4. Pearson correlation coefficient between prevalences, means. and ranks in clinic data and MMCN survey data

a. Survey data are weighted to match the seasonal distribution of measurements in clinic data.

Comparison of cluster rankings and correlations

Table 4 shows the Pearson correlation coefficients for infants and children between clinic and survey prevalences. The results are shown separately for analysis using data from all three periods as well as data from period I by itself. As shown, there was a clear tendency for a negative correlation between the prevalences, as well as between the corresponding cluster rankings. This tendency was evident among infants and children, but approached statistical significance only among children. This negative correlation is not materially affected by using logit transformations of the prevalence data, by weighting the analysis in favour of those clusters with greater sample size, or by the use of survey means rather than prevalence (results not shown).

Figure 2 (see FIG. 2. Relationship between prevalences from clinic and MMCN survey data (all periods combined)) illustrates the nature of this relationship in greater detail. Among infants the negative correlation was dictated to a large extent by two clusters that had very high prevalence in the clinic data, Elangeni and Muhuju/Ngong'a, and medium to low prevalence in the survey data. However, there was little or no correlation between clinic and survey prevalences even among the other five clusters. Among these five points the survey data reveal a good range of prevalence from 5% to 27%, but the clinic data fail to detect these differences, instead remaining within a fairly narrow range of 6% to 9%. Similarly, the negative correlation among children's prevalence is heavily influenced by one cluster (Muhuju/ Ngong'a), but there is no evidence of a positive correlation even among the remaining six points.

Another method of comparing these data is through the use of cluster rankings rather than prevalence or means. This may be particularly relevant in the case of surveillance data if the desire is simply to identify those areas with the greatest nutrition problems in order to target resources there. Table 2, based on seasonally weighted data, reveals substantial differences in cluster rankings when they are based on clinic data versus survey data. Among infants and children the clusters ranked as most malnourished by one data source were not similarly ranked in the other data source. Indeed, as shown in table 4, a negative correlation was seen in the cluster rankings, similar to that observed in the prevalence and means. In practical terms, clinic data for infants suggest that resources should be targeted toward Muhuju/Ngong'a, Elangeni, and Mphompha as the worst three clusters, whereas the survey data suggest they should be targeted toward Chinteche, Enkweleni, and Yakuwata. The data for children produce similarly inconsistent results, except that one cluster (Elangeni) would be targeted as high priority by both sources.

For reasons outlined earlier the above analyses were repeated using the data for period 1 only. With regard to the correlation among prevalence, the moderate to strong negative correlations were not seen in period 1 data. Instead, they were very close to zero or weakly positive. These results were not affected by using logit transformations, by weighting according to sample size, or by using survey means rather than prevalences (results not shown). Inspection of scatterplots reveals the presence of one grossly aberrant cluster (Elangeni) in the infant comparison, but even if this were excluded, there is no evidence of a positive correlation among the remaining six points (not shown). Similarly, the scatterplot for children reveals that the weak positive correlation arises from one cluster (again Elangeni), with no evidence of a trend among the remaining points (not shown).

Finally, the period 1 results were also compared with respect to cluster rankings. In this case, the three priority clusters as identified by clinic data included two that were also priority on the basis of survey data. This was seen in infants as well as children. Note that, with seven data points, there is roughly a 50% probability of this occurring by chance alone.

Discussion

Methodological considerations

The results explicitly took account of factors such as seasonality, child's age (which covaries with clinic coverage), and statistical considerations in comparing clinic data with community-based data. However, it is also necessary to address the question of whether the MMCN survey data can be used as a basis for validating the clinic data. This issue has three dimensions, involving the reliability of weight-forage measurements, stability of cluster rankings, and representativeness of MMCN data. In addition, it is important to examine to what extent these seven MOM facilities are representative of the larger set of facilities from which they are drawn. These issues have relevance to the present study and to future research efforts along these lines.

Reliability of weight measurements

Reliability of weight measurements is influenced by numerous factors, including variation in technique within and between enumerators, variation between weighing scales or within scales from one day to the next, and other sources of non-nutritional variation between or within subjects from one day to the next, due to variation in weight of clothing, levels of hydration, stomach contents, and so on. Of these, the last is probably the most significant in general, but tends to be random rather than systematic, thereby having less effect on means than on prevalence. In the MMCN study special effort was made to measure children with minimal clothing, with the same scales that were calibrated regularly, and by the same enumerators within any given round of data collection. In addition, measurement technique was checked regularly by supervisors during routine visits A formal test-retest study performed under classroom-type conditions between updates I and 2 revealed that only 1.7% of the total variance in weight among these children was due to intra-observer and inter-observer error, comparable to results obtained in other field studies [12].

Stability of cluster rankings

Given the objectives of the present study, a particularly relevant question is the extent to which the MMCN estimates of underweight at cluster level are replicated from one period to another. A high degree of replicability would provide greater confidence in using the MMCN data as a basis for validating the MOH data. A low degree of replicability, on the other hand, is more difficult to interpret because it may reflect methodological variation in the MMCN study (e.g., changes in enumerators, equipment, clothing) and/or true variation in the extent of seasonality across clusters. Cluster-level replicability was examined by comparing weight-forage means, prevalence, and ranks across the three periods, with and without logit-transformation and weighting by sample size.

As shown in table 5, there is a strong positive correlation (r=.87) between prevalence (and their ranks) in periods I and 2 for infants, and a moderately strong correlation for children (r=.57-.69). These correlations are also shown in figure 3 (see FIG. 3. Relationship between prevalences in periods I and 2 in MMCN survey data). This suggests fairly good stability in estimates of prevalence and ranks between periods 1 and 2. However, correlations involving period 3 (i.e., period 1 with 3, 2 with 3) are much lower. In the case of infants this is caused by two clusters (Enkweleni and Elangeni) that have very small sample sizes and prevalence of zero.

When correlations are calculated on the cluster means rather than prevalences (to reduce the influence of small sample sizes), the three coefficients for infants and the three for children all reveal moderately strong positive correlations in mean WAZ across the three periods (r=.50-.60). Similarly, when the cluster ranks are assigned on the basis of means rather than prevalence, the correlations are moderately strong and all positive. This provides some measure of confidence in the validity of the MMCN data for the purpose of estimating and ranking mean WAZ across clusters, but much less so for prevalence when sample sizes are small.

TABLE 5. Pearson correlation coefficients between weight-for-age prevalence and means in MMCN survey clusters measured during three periods^a

1. seven clusters in each contrast; periods as defined in figure 1
2. Note that two clusters have a prevalence of 0.0 in period 3. thereby distorting the estimated correlations. Correlations based on means provide a more accurate estimate (r =.5-.6).

Representativeness of MMCN data

As noted earlier, each MMCN cluster contains several adjacent villages, but these were not randomly chosen from all villages in a given clinic catchment area. Rather, the entire cluster of villages was chosen as a single block. This raises the possibility that any lack of correlation between MMCN and MOH data may result from the fact that they are drawn from overlapping but somewhat different populations. One way to examine the potential impact of this is to examine the extent to which mean WAZ varies among villages within an MMCN cluster. This was made possible by the fact that each cluster contained 3 to 10 villages. To do this the mean WAZ for each village was subtracted from the mean for the cluster to which it belongs, and the standard deviation, standard error and 95% confidence interval were calculated on the basis of these differences.

The results show that the 95% confidence interval for estimating the cluster mean based on a sample of village means is 0.21 Z score units when averaged across all the clusters. This is not a large amount of error considering that the cluster level means have a range of about 0.70 Z score units between the lowest and the highest clusters. Moreover, inspection of scatterplots reveals a clear tendency for the mean WAZ in most villages to be located in the vicinity of the overall mean for its respective cluster [13].

MOH data

Another methodological consideration relates to the nature of the MOH data. Most of the clinics did not report on a monthly basis, resulting in incomplete data from a seasonal perspective and fairly small sample sizes overall, especially for children. Even though the survey results were weighted to match the seasonal distribution of clinic measurements, it is possible that they may be influenced by incomplete reporting and the lack of attention to detail it signals. Thus, the present study cannot strictly address the question of how valid the clinic data might be if they were faithfully collected and reported each month; rather, it can only examine their validity under the circumstances prevailing in these clinics during 1989.

This raises the related question of whether and how the clinics might differ from others in Malawi in terms of their reporting procedures, frequency, and so on. This question was investigated by comparing the distribution of reporting frequency in these clinics (i.e., what percentage reported once, twice, etc., up to 12 times a year) with the distribution for a larger sample. The results show that the distribution for all Malawi clinics includes 22% that submitted 10 to 12 reports in 1989, 47% that submitted 7 to 9,14% that submitted 4 to 6, and 16% that submitted fewer than 4 reports. The three districts were fairly similar in their reporting frequencies, although Mzimba submitted somewhat fewer than the other two. The clinics did not vary in any systematic fashion from these overall tendencies; three submitted 10 to 12 reports in 1989 and the remaining four submitted between 5 and 9 reports. Thus, it does not appear that they were peculiar with respect to reporting frequency.

It should be borne in mind that this analysis is based on disaggregation of data to the clinic level, whereas other levels may be more appropriate for surveillance purposes, for example, district-level targeting. This implies that the results might not be readily generalized to higher levels of aggregation.

District-level analyses would have larger sample sizes and the possibility that local (clinic) variations in weight for age may be averaged away during the aggregation process. Such variations may arise from measurement and reporting problems as well as local-level variations in factors affecting nutrition status, such as health conditions and circumstances concerning household food security. Thus, the validity of using clinic data at a higher level of aggregation may require specific investigation at that level, which might be accomplished in Malawi by comparing prevalences from CBD with those from the Ministry of Agriculture's Food and Nutrition Information System [2].

Validity of clinic-based data

This study suggests two important conclusions regarding the use of CBD to estimate the level of malnutrition in the general population. First, the differences in prevalence between clinic and survey data are significant not only statistically, but also in policy terms. The prevalence estimated from one source often differs by a factor of two to three from that estimated from the other source. This is true for infant as well as child prevalence. The second important conclusion is that the direction of the bias is not constant. In some clusters the prevalence is higher in clinic data whereas in others it is higher in survey data. This means that at a clinic level it is not even possible to state whether the CBD provide an upper or lower boundary for the value of the true prevalence in the general population. Thus, it appears that CBD do not provide a reliable guide to the overall magnitude of malnutrition in the general population.

The results pertaining to the use of CBD for targeting purposes are similarly disappointing. As revealed here, there is no evidence to support the use of CBD to distinguish high-prevalence areas from others. Indeed, the results based on all three survey periods, appropriately weighted to match the clinic data on a seasonal basis, suggest that resources might actually be targeted to the better off populations if the CBD were used for this purpose. It is unclear whether this is a peculiarity of the present small sample of clinics or whether it reflects a systematic characteristic of clinic data.

A possible explanation for this phenomenon may relate to selective attendance at clinics. It may be that villages with poor nutrition status are symptomatic of populations that generally do not attend clinics for preventive or minor curative care. In such villages it may be the more highly motivated and health-conscious mothers who preferentially attend clinics and, since children of such mothers should be better nourished than average, this may artificially lower the prevalence in that clinic. It is notable, for instance, that the high-prevalence clusters tended to be in Mzimba and Nkhata-Bay, where the levels of maternal literacy were not nearly as high as in Rumphi. At this point this explanation must be considered conjectural.

Finally, it should be noted that this study provides some insight into the use of individual clinics as sentinel sites as an alternative approach to nutrition surveillance. Sentinel sites are sometimes used when it is not possible to collect and/or trust the data coming from a large number of clinics. The implicit assumption is that the levels and trends in malnutrition reflected in the sentinel site, where greater consistency can be ensured in reporting and weighing techniques, are indicative of the levels and trends in the larger population of the district or region. The present study suggests that this approach may not provide valid indications of malnutrition levels and trends.

The results from individual clinics varied in unpredictable ways from the results from the general population. Moreover, as noted earlier, analysis of the MMCN data by themselves revealed that the trends in weight-for-age prevalence and means (seasonal and year-to-year trends in updates 2-4) vary from one study cluster to another. This variation probably reflects differences in weather patterns, food availability, and disease patterns, which had a highly localized dimension when the MMCN study was conducted. Thus, despite the fact that sentinel sites may be the only feasible approach to obtaining some data on nutrition status in difficult circumstances, the validity of these data remains to be demonstrated.

Comparison with previous studies

One of the ways to assess the extent to which the Malawian results are typical of what might be expected from CBD is to compare them with findings from other countries. Studies in five other countries related to this issue: from El Salvador [14], Swaziland [15], Jamaica [16], Botswana [17], and Indonesia (unpublished).

As shown in figure 4 (see FIG. 4. Relationship between prevalences of underweight in clinic and survey data from four countries), none of the studies provides strong evidence in favour of using CBD for targeting purposes. The Swaziland study found no association between prevalence in CBD and survey data; the El Salvador CBD distinguished two major regions with vastly different prevalence, but did not distinguish subregions within each; the Jamaican CBD underestimated survey prevalence by variable amounts at different times and did not reflect the changes in prevalence suggested in the survey data; and the Botswana CBD were able to distinguish three areas with far higher prevalence than average, but could not distinguish variations among the remaining 17 areas. Finally, the Indonesian study failed to find an association between CBD and survey prevalence and revealed that the direction of the bias varied from one location to the next (J. Haas, personal communication).

In addition to results within countries, these studies shed some light on the ability of CBD to reflect differences in the absolute levels of malnutrition in a given country. Thus, in Botswana, Swaziland, and Jamaica the overall prevalences from CBD and survey data were reasonably close in absolute terms; in El Salvador they differed substantially, with clinic data always providing overestimates of survey prevalence. In Malawi CBD tended to provide underestimates of survey prevalence. It is notable, however, that a much larger sample of Malawi clinic data collected in 1977 suggested an overall prevalence of 51%, which is substantially higher than the 27% to 34% estimated in the National Sample Survey of Agriculture [18, 193.

The overall conclusion that emerges from all of these studies is that the validity of CBD for nutrition surveillance within countries for targeting purposes is highly suspect. Of all the studies available there has yet to be a single one that concluded that the data are adequate for distinguishing high-prevalence populations. Although this does not mean that CBD are invalid for this purpose in all situations, the accumulated evidence is certainly sufficient to indicate that their validity in each setting should be demonstrated rather than assumed. Some of the methodological principles outlined here may be useful in planning future investigations, with particular attention to validity at higher levels of aggregation and validity in estimating trends in prevalence over time rather than point estimates at one time.

Acknowledgements

This research was supported by grants from UNICEF/Malawi and the Inter-Agency Food and Nutritional Surveillance project administered by UNICEF/NY. The Malawi Maternal and Child Nutrition Study on which it is based is a collaborative project between the University of Malawi's Centre for Social Research and the Cornell Food and Nutrition Policy Program. The overall project has received support from WHO, UNICEF/Malawi, USAID/Malawi, IDRC, the Carnegie Corporation, the Rockefeller Foundation, the Thrasher Research Fund, and a Cooperative Agreement between the USAID Office of Nutrition and Cornell University. The authors acknowledge the assistance of Dr. Chitsulo of the Ministry of Health Research Unit in Malawi.

References

1. Cornell Nutritional Surveillance Program. Trends in nutritional status of children in rural Kenya: review and policy implications. Cornell Nutritional Surveillance Program working paper. Ithaca, NY, USA: Cornell University, 1984.
2. Ministry of Agriculture. Household food security in Malawi. Food security and nutrition monitoring report. Lilongwe, Malawi: Ministry of Agriculture, 1991.
3. Mason JB, Habicht H. Tabatabai V. Valverde nutritional surveillance. Geneva: WHO, 1984.
4. Centre for Social Research. The Malawi maternal and child nutrition study: interim findings from the baseline census. Presented at the Household Food Security and Nutrition Workshop, Zomba, Malawi, 29 August 1988. Cornell Food and Nutrition Policy Program mimeo.
5. Cornell Nutritional Surveillance Program. An examination of the characteristics of data from underfives clinics in the northern region of Malawi: its potential usefulness for nutritional surveillance and its implications for MCH services. Prepared for Centre for Social Research, Zomba, Malawi, 1987. Cornell Food and Nutrition Policy Program mimeo.
6. Pelletier DL, Low JW, Msukwa LAH. The Malawi maternal and child nutrition study: study design and anthropometric characteristics of children and adults. Am J Hum Biol 1991;3:347-61.
7. Martorell R. Habicht JP. Growth in early childhood in developing countries. In: Falkner F. Tanner JM, eds. Human growth: a comprehensive treatise. Vol. 2. New York: Plenum, 1986;241-62.
8. Beckerson SA. Seasonal labour allocation food supply and nutrition in subsistence and semi-subsistence farming households in Malawi, Africa. Master's thesis, University of Guelph, Canada, 1983.
9. Chimwaza BM. Food and nutrition in Malawi. Doctoral degree thesis, University of London, UK. 1982.
10. Pelletier DL, Msukwa LAH. The use of national sample surveys for nutritional surveillance: lessons from Malawi's national sample survey of agriculture. Soc Sci Med. 1991 ;32(8):887-98.
11. Snedecor GW, Cochran WG. Statistical methods. Ames, Iowa, USA: Iowa State University, 1980:7.
12. Pelletier DL, Low JW, Msukwa LAH. Sources of measurement variation in child anthropometry in the Malawi maternal and child nutrition study. Am J Hum Biol 1991;3(3):227-38.
13. Pelletier DL, Johnson FC, Chitsulo D. The validity of clinic-based nutritional surveillance data from selected sites in northern Malawi. Report submitted to Ministry of Health, Lilongwe, Malawi. Cornell Food and Nutrition Policy Program mimeo, 1992.
14. Trowbridge FL, Newton L, Huong A, Staehling N. Valverde V. Evaluation of nutritional surveillance indicators. Bull Pan Am Health Org 1980;14(3):238-43.
15. Serdula MK, Herman D, Williamson DF, Binkin NJ, Aphane JM, Trowbridge F. Validity of clinic-based nutritional surveillance for prevalence estimation of undernutrition. Bull WHO 1987;65(4):529-33.
16. Grosh ME, Fox K, Jackson M. An observation on the bias in clinic based estimates of malnutrition rates. Human Resources Division of the World Bank, Technical Department, Latin America and Caribbean Region. Washington, DC: World Bank, 1990.
17. UNICEF. Report on the evaluation of Botswana's nutritional surveillance system. Nairobi, Kenya: UNICEF Social Statistics Programme Eastern and Southern Africa, 1983.
18. Jere SMC. Clinic based nutrition surveillance. Malawi: Ministry of Health, 1984.
19. National Statistical Office. National sample survey of agriculture. Zomba, Malawi: Government Printer, 1984:1980/81:3.

Contents - Previous - Next