Contents - Previous - Next

This is the old United Nations University website. Visit the new site at http://unu.edu

The validity of clinic-based nutrition surveillance data: A study from selected sites in northern Malawi

David L. Pelletier and F. Catherine Johnson

Abstract

Many developing countries collect on the weight for age of children attending health facilities as one element of a nutrition surveillance system. This study compares the estimates of malnutrition from seven health clinics in northern Malawi with estimates derived from nearby community-level surveys. The results show that prevalence of underweight in clinics does not accurately reflect community prevalence. Clinic estimates often differ by two- to threefold from community estimates, and the direction of the bias is not constant across clinics, making these data an invalid basis for targeting programmes according to nutritional need. Similar results were reported in five other studies in the literature, indicating that the Malawi results are not unusual. It is suggested that, contrary to current practice, cross-sectional clinic-based data should be assumed invalid for targeting purposes unless proved otherwise in a given country. Trend data at regional and country levels require further validation.

Introduction

Most nutrition surveillance systems in developing countries include information on the nutrition status of children as one of the central components. In some cases, as illustrated in Kenya [1] and more recently Malawi [2], this information is obtained through sample surveys conducted at regular intervals. In many more cases, however, data from health clinics are used [3]. Such data have the advantages of being readily available without the extra survey costs, being available more frequently than survey data (monthly or quarterly rather than every few years), and being able to support highly disaggregated analysis (e.g., to district or even throughout clinic level). The last may be a particularly desirable feature when the primary application is for identifying areas with high rates of malnutrition in order to target programmes or other resources to them. It is seldom possible to disaggregate data from sample surveys to the extent necessary for such targeting applications.

Despite these advantages of clinic-based data (CBD) for nutrition surveillance, a number of potential disadvantages and concerns exist. One disadvantage is that collecting, aggregating, reporting, and computerizing these data do have an associated cost in the form of staff time at clinic and national levels, and possibly at intermediate levels depending on the details of data flow and processing. Although the time devoted to these activities may not be as visible as that associated with the nutrition survey methodology, there is none the less an opportunity cost in terms of service delivery.

Of equal importance is the fact that the quality and regularity of reporting from clinics to national level is often variable at best, and frequently quite poor. Thus, if CBD are to be used for nutrition surveillance, a major decision must often be made on whether they are worth the cost of retraining staff, possibly revising the reporting formats, setting up computerized systems, and ensuring good supervision of the system. That decision should be based on considerations of cost, specific uses of the information at national or other levels, and alternative strategies available for obtaining the desired information. In addition, the decision should be based on knowledge of whether or not CBD are reasonably valid for supporting the expected policy and programmatic decisions.

Two categories of factors may affect the validity of CBD. One is related to inaccuracies or incompleteness in weighing and determining age, and plotting, tallying, and reporting data. These are the types of problems that might be remedied by improved training and supervision. The other category is more problematical and is related to the fact that children who attend clinics are typically only a fraction of children in the catchment area and are probably not representative of all children in the general population. A common pattern in developing countries is to have reasonably good coverage (>75%) in the first year of life when children are brought for immunizations, but very low coverage thereafter ( < 10%) unless specific services are offered that are highly valued by mothers, such as supplementary food. Children who do attend clinics after the first year of life are likely to do so either because they are sick and seeking care or because their mothers are exceptionally well motivated and health-conscious. In either case the coverage is incomplete and probably biased, especially after the first year of life.

Like many countries, Malawi possesses a clinic-based reporting system with potential utility for nutrition surveillance. Indeed, selected results from this system have been used since the late 1970s to examine the extent of malnutrition in different parts of the country. More recent efforts include computerization of the data and production of wall maps to convey information in a more usable form. This paper examines the validity of CBD in Malawi and compares this with results of studies in other countries.

Methods

Generation of CBD in Malawi

In Malawi, CBD are based on children attending weekly under-five clinics serving sick and healthy children, nutrition clinics serving severely malnourished children who receive supplementary feeding and education, and those seeking routine health care. The surveillance system consists of recording weight for age on mother-retained growth charts and tallying the number of infants or children who are underweight and normal weight. (In Malawi, NCHS standards are used, but growth charts are based on girls only. Underweight is defined as more than 2 SD below the median.)

The tallies are compiled and submitted on a monthly basis to the district health office and the Ministry of Health (MOH). Separate tallies are kept for infants (0-11 mo) and children (1-4 yrs), and for different types of visits: first ever visit, first visit in the current calendar year, or second and higher visits in the current year. The present analysis is based on the first visit in 1989 for each child, to avoid potential bias arising from several visits of sick children in a given year.

As noted, surveillance in this setting is limited to children who attend the under-five or nutrition clinics, or who seek treatment in health centres or hospitals. In Malawi, as in many countries, use of health services declines dramatically after immunizations are completed (i.e., after the child is one year old) [4]. Thus, clinic-based surveillance usually covers between 70% and 80% of infants, those who regularly attend health centres for immunizations, and only 2% to 5% of those between one and four years [5], who typically attend owing to severe illness.

Comparative data for validating CBD

The Malawi maternal and child nutrition (MMCN) study is a three-year study of the causes and consequences of malnutrition among mothers, newborns, infants, and children. Approximately 950 pregnant women were enrolled over a two-year period beginning in January 1987 and visited monthly until the end of the study in January 1990. These women resided in 17 study clusters from northern Malawi (Nkhata-Bay, Rumphi, Mzimba districts), with each cluster consisting of several adjacent villages. The clusters were chosen to provide contrasts in agroecological characteristics. Three broad agro-ecological zones were identified based on staple foods (cassava v. maize), rainfall, and elevation. These zones became the strata for collecting a stratified random sample of 17 study clusters and 86 villages, as described elsewhere [6]. These samples were not chosen for the purpose of validating CBD, but are used in an opportunistic fashion for the purpose of this paper.

The Rockefeller component of the MMCN study, which provides the data used in the present analysis, consists of a greatly enlarged sample of children from these same 17 clusters. The purpose of this component was to monitor the nutrition and vital status of all children below age five in these clusters through a series of surveys conducted roughly every six months. Since the surveys were intended to update the information collected in the baseline census in early 1987, they are referred to as updates. A total of four updates were conducted between March 1988 and March 1990, each requiring three to five months to complete.

Table 1 shows the weight-for-age mean and the prevalence below —2 Z scores for infants and children, based on all of the data from updates 2 to 4. In these and all subsequent results, data are provided for the seven study clusters for which corresponding MOH data exist from nearby health facilities. Ten study clusters had no corresponding MOH data, because the data either were too sparse or suggested gross reporting inaccuracies. The table reveals considerable variation in malnutrition across the seven clusters, with the prevalence of underweight varying from 11% to 36% among infants and 23% to 38% among children. This is especially important in the present study because of the need to relate this variation to the patterns reflected in clinic-based data. As reported elsewhere [6], most of this underweight is due to low height for age rather than low weight for height, in common with the situation in most other developing countries [7].

TABLE 1. Mean and prevalence of nutritional indicators by cluster and age

For all three indicators the prevalence cut-off point is < 2 Z scores. This table includes information for all months for updates 2,3 and 4. Later tables are restricted to the periods with MOH data.

Seasonal matching of MMCN and MOH data

Although the MMCN updates cover the period from November 1988 to March 1990, the present analysis only uses data from December 1988 to December 1989, to correspond to the period during which MOH data were collected. As described below, preliminary analysis indicated that seasonality has a significant influence on the nutrition status of children in the MMCN study. Moreover, further analysis revealed that the pattern of seasonality in nutrition status differs from one year to the next in some clusters, probably reflecting variations in household food consumption and morbidity. Thus, analysis of the MMCN data suggested that the samples from updates 2 to 4 had to be chosen carefully to match the seasonal distribution of MOH data and without averaging data from two different years.

The seasonal matching of MMCN data to the MOH data was done by defining three periods of the year. Period I extended from December 1988 to March 1989 and represented a time of food shortages, peak morbidity, and maximum time constraints on women [8, 9]. Period 2 extended from April to September 1989 and represented the totality of the harvest and early post-harvest period in the northern region. Food was generally available in all households during this time, morbidity was lower, and women's time was not as constrained. Period 3, October to December 1989, was a transitional one for many households in terms of these characteristics.

Figure 1 (see FIG. 1. Mean weight-for-age Z score (WAZ) by cluster and period) shows the mean weight-for-age Z scores (WAZ) of infants and children in the MMCN study, according to the three periods. The seasonal changes in mean WAZ are seen most clearly among children. The most common pattern, as occurred in six of the clusters, involves relatively low WAZ in periods I and 3 (corresponding to high prevalence of underweight) and improved WAZ during period 2. This corresponds to what one would expect on the basis of the seasonal occurrence of morbidity, time constraints, and household food supply. The one exception to this pattern is in Chakoma cluster, where the improvement in WAZ continued into period 3 rather than returning to high levels as in period 1. Among infants the situation was similar in so far as mean WAZ improves between periods 1 and 2 in six of the seven clusters. However, the return to poor levels of WAZ in period 3 was seen in only three clusters, with the other three showing no change or continued improvement from period 2 to 3. The inconsistent results for infants compared with children may reflect the smaller sample sizes for this group or differences in nutritional ecology with age [10].

Preliminary analysis revealed that the seasonal distribution of measurements was quite different between the MOH and MMCN data sets, which could confound the results if not taken into account in the analysis. This was addressed by comparing period-specific data in some cases and by calculating weighted averages or weighted prevalence in the MMCN data, using the distribution of MOH measurements across the three periods as the statistical weights as follows:

where:

P(T) is the weighted annual prevalence of underweight in a given cluster;

p(i) is the prevalence for the cluster in period i (where i = 1, 2, 3);

wt(i) is the cluster-specific weight for period i, based on the proportion of MOH data from that cluster derived from period i;

represents the summation across all periods.

This methodology was applied to infants and children separately, using weights derived from the two age groups in the MOH data. The methodology was also applied in estimating weighted means for WAZ by substituting cluster-specific means in the equation rather than prevalence.

Statistical methods

The MOH data were compared with MMCN data from two perspectives. First, the absolute levels of underweight prevalence were compared to identify any systematic bias in clinic-based data across all clinics (i.e., under- or overestimation of prevalence). The significance of differences between MOH and MMCN prevalence was assessed for each cluster separately by calculating the normal deviate (Z) of the difference between proportions [11] and taking note of the direction of the difference across the seven clusters.

Second, a variety of methods was used to examine the extent to which the prevalence of underweight across the seven MMCN clusters was associated with the prevalence in the corresponding MOH reports. This question is somewhat distinct from the first, in that biased clinic data may still be useful for surveillance, as in targeting by geographic area, so long as the direction of the bias is consistent. The association in prevalence between the MMCN and MOH data was examined by ranking the clusters in one data source from lowest to highest and comparing this with the ranks from the other data source, and by calculating the Pearson correlation coefficient between the two sets of prevalence estimates.

In applying these methods, consideration was given to a number of other statistical complications. First, the MOH prevalence was compared with MMCN means as well as prevalence whenever appropriate, to minimize the influence small sample sizes may have on estimates. Second, the comparisons were performed before and after applying a logit transformation to the prevalence data, to stabilize the variances across a range of prevalence and allow the inferential statistics (correlation coefficients and t tests) to be used with greater validity. Finally, because the MMCN sample sizes varied across the seven clusters, correlation coefficients were calculated before and after applying weights to each cluster's estimate. In this fashion clusters with small sample sizes have less influence on the estimated coefficient. The weights used in this analysis simply reflect the proportional representation of each cluster in the total MMCN sample size for a given analysis. It should be noted that the results are not greatly affected by applying these adjustments and transformations. Thus, this paper presents only the results of simple comparisons.

Contents - Previous - Next