Previous - Next
Stages of analysis and interpretation of findings
Establishing the trustworthiness of information
Presentation of findings
Implementation of findings
By the time
you finish using the methods and tools you have selected for your
study, you will have several sets of information organized and
stored in notebooks, files, and index cards according to
chronological order or by method/tool used, or both. This chapter
deals with the processes of conducting overall analysis of all
the information gathered and reviewed; checking its
trustworthiness by triangulation; interpreting or making sense of
findings; presentation and use of findings. As documentation is
one of the most important outputs of a hygiene evaluation study,
we shall demonstrate how investigation and analysis link up to
report writing in practical terms.
There are four main stages
in the analysis and interpretation of qualitative information.
These are discussed in more detail in several text books
including Patton (1986, 1990), Miles and Huberman (1994), and
Silverman (1994). Here, we shall concentrate more on the
practical tasks, rather than on theoretical issues.
Descriptive Analysis
Description and analysis of qualitative information are closely linked, hence the phrase descriptive analysis. This includes some description of the purpose of the study, the study site, and people involved which is normally presented in the introductory sections of a report. However, descriptive analysis focuses on the information gathered in relation to how it was gathered, where, and by whom. This involves reviewing the information, identifying links, patterns, and common themes, arranging the facts in order, and presenting them as they are, without adding any comments on their significance. This is usually presented in the Results section of a study report. The order in which the results are presented may be chronological, following the order in which the facts were obtained; or hierarchical, in order of their relative importance to the heart of the investigation. The introductory description and the descriptive analysis (results) sections of a study report should enable you to answer basic questions. For example:
Introductory Sections
• Where was the study conducted? What are the physical and climatic conditions in which people live?
• When was the study conducted? Why?
• What were the study aims, objectives, and intended outputs?
• Who conducted the study? Which methods/tools were used? Why?
• How did people participate in the study? Which ethnic, language or other groups were involved? How does the level of participation achieved in your study compare with your project's general ethos concerning (community) participation?
Results Section
What does the information gathered consist of:
• by method/tool of investigation used;
• by cluster of hygiene practices;
• by any other relevant order?
Answers to these questions require rigorous analysis and description, but not interpretation (see Box 24 for an example of how results are distinguished from discussion of findings or interpretation).
BOX 24. An Example to Demonstrate How Reporting Results Differs from Interpretation In a hygiene evaluation study conducted in rural western Kenya, several methods and tools were used including mapping, Three-pile sorting, Spot-check Observations, and Semi-structured (Informal) Interviews. With regard to latrine use, the findings were as follows: Mapping Maps created by study participants in both villages revealed that most latrines (seventeen out of twenty-one in Village 1, and twenty-five out of twenty-six in Village 2) were located outside the courtyards. Three-Pile Sorting The picture of a VIP latrine with a curtain which did not reach the floor (so that the feet of the person using the latrine could be seen) was categorized as bad in both villages. Observations Children's faeces were noticed in the compound only if the mother was absent. In both villages, very little faecal contamination was observed in both the domestic and the public environment. Informal Interviews Others reported that they normally train their young children to defecate in a specially designated place within the compound... after defecation, the child would let the mother know and she would dispose of the faeces either by taking it to the latrine (with a hoe), or by digging and burying it in the ground. These and other findings were then put together, crosschecked and interpreted, and presented 'In the Discussion section of the report as follows: "In
the Luo culture, it Is generally held that contact with
human faeces is defiling and thus to be avoided at all
costs... Firstly, there are clear gender-specific rules
about latrine construction and maintenance ... Secondly,
if a latrine is to be used, and used by everyone, then it
should be located appropriately... If a latrine is
located within the compound, it cannot be shared by
in-laws... the use of a latrine inside the compound of
one's in-laws is seen by the Luo a, tantamount to undressing
or being naked in front of one's in-laws even
though nobody actually sees the act of undressing or the
state of being naked. Such notions of nakedness relate to
privacy which is a very important and well recognized
requirement for latrine acceptance and use...The results
of the three-pile sorting activities certainly support
the privacy argument." |
Sufficient detail should be included in the descriptive analysis to enable the reader to see the investigative steps you have followed, how you made methodological decisions, or changes of direction. and why. Remember that the facts have to be presented clearly, coherently, and fully before they can be interpreted. A very important feature of the descriptive analysis is the checking and crosschecking of information in order to establish the quality or trustworthiness of the findings. We shall deal with this separately in detail in "Establishing the Trustworthiness of Information.''
Interpretation
The second stage is to determine what the results mean and how significant they are in the specific context to which they belong. The reasons behind certain hygiene practices and to what extent they are influenced by sociocultural factors can be teased out when the study team's multiple perspectives are brought to bear on the results. Wider issues concerning our understanding of the links between hygiene practices and health can also be explored in the light of the findings.
The following are some of the questions for the study team to answer when interpreting the study results:
• What do the results mean?
• Why did the results turn out the way they did?
• What are possible explanations of the results?
• Have all the why questions been answered? Do some of them require further investigation?
The interpretation of findings should ideally reflect the comments and suggestions made by members of the study population(s) during the feedback sessions that are built into the use of investigative and analytical methods/tools, such as those described in Chapters 5 and G. This will help minimize the biases that can creep into the interpretation of results, making sure that they are not separated from the context in which information was gathered (see Box 24).
Judgement
Descriptive analysis and interpretation of results ultimately lead to judging the findings as positive or negative or both, and stating the reasons why. The values of the study team and other stakeholders are brought to bear on the study findings. For example, the findings may show what is good, bad, desirable, or undesirable in the way the project has promoted improved water supply, sanitation, and hygiene/health, in the way people have responded to external interventions. and why. The question to be answered here is:
• What is the significance of the findings to the various stakeholders in this particular setting?
• to your project?
• to the study population?
• to applied researchers interested in the links between particular hygiene practices and health?
The interpretation and judgment of results are usually presented in the Discussion section of a report. It is important to strike a fair balance between the positive and negative aspects of the findings. For example, positive findings should be emphasized without brushing over negative ones. Similarly, negative findings should not only be listed, but discussed in a way that explores possible practical solutions or feasible remedies. The discussion section should be followed by the conclusions which may be presented in the same section or separately under Conclusions.
Recommendations
The fourth stage is to draw some recommendations for action to be taken on the basis of the analysis, interpretation, and judgement of study findings. The Recommendations section of a report normally follows the discussion and conclusions and should address the following questions.
• What are the implications of the findings, based on your analysis, interpretation, and judgements? What are the implications:
• for your particular project?
• for other projects that may be interested to learn from your findings?
• for any other interested parties, such as researchers?• What should be done by your project and other stakeholders on the basis of the analysis, interpretation, and judgement of your study results?
The more
the different concerned parties or stakeholders are involved in
the interpretation and judgement of the study results, the easier
it will be for you to reflect their interests in the
recommendations. Practical and feasible suggestions should be
clearly included in the recommendations.
As discussed in
"Putting in Place Data Quality Checks" in Chapter 4,
the criteria for establishing trustworthiness of qualitative data
are essential components of the study design and conduct which
enhance the trustworthiness (or goodness) of the
information gathered. Unlike the statistical significance or
goodness-of-fit tests applied to quantitative data, the criteria
for trustworthiness of qualitative data are not a set of tests to
be applied to the information after it has been collected, but
in-built checks that are put in place before information
gathering begins, and monitored throughout the conduct of
investigation (see Chapter 4).
You should be able to judge the trustworthiness of the information you have gathered by applying all the criteria you put in place when designing the study while conducting it. The number of criteria applied may vary from one study to another, depending on the resources (human, material, time). and other constraints on the study design and execution. However, the following key criteria constitute the minimum requirements that should be met in order to establish the trustworthiness or the quality of qualitative information.
• Prolonged or intense engagement of the study team with the study population. The duration of the study will be determined by resources available and the study team's familiarity with the study population. A lot can be done in a couple of weeks, especially if field workers know their study population very well. If not, a longer time will be required for the team to establish rapport with the population and minimize biases introduced by unusual manners and the unnecessary separation of the study team from the community. Be clear and honest in reporting your estimate of biases that might have crept into the study due to the type of engagement between the study team and the population(s).
• Triangulation of sources. methods, and investigators. As discussed in Chapter 4, it is often not feasible or practical to design a study in which means of triangulation of sources, methods, and investigators can all be put in place and applied. For example, one study may be conducted by using focus group discussions with caretakers of young children, semi-structured interviews with the same category of respondents and spot-check observations of selected households, and the study team may consist of very few individuals with similar disciplinary backgrounds. Another study may employ a larger study team with diverse backgrounds and skills and sufficient resources to enable them to use participatory investigative and analytical tools as well. Crosschecking information can be done in both cases through triangulation of sources and methods, or triangulation of methods and investigators. The most important thing is that trustworthiness of the results is checked and crosschecked by triangulation. Your report should include a clear account of the triangulation carried out.
• Feedback and discussion with the population. This will help in finding possible paths for the interpretation of findings and should be documented in the report.
• Peer review/checking. When peers, independent reviewers, including perhaps some of your colleagues who were not directly involved in the investigation processes, check your results, they may identify areas where you may need to provide more information or justification for the conclusions drawn. This means that your study report has to include rigorous description and analysis, with an attached diary of activities containing sufficient detail on when and how the study was carried out, for reference.
Peer reviews are most productive when criticisms are put to the study team clearly and constructively. However, you should be prepared to respond to difficult questions and/or not-so-constructive criticisms as well. You may need to review and, if necessary, clarify major decisions and changes of direction made during the conduct of the study. To help in preparation for such eventualities, self criticism during the processes of investigation and analysis should be encouraged among members of the study team, in an atmosphere of trust and openness.
Study reports that include very little or no detail on how the study was conducted, when, and why methodological and other decisions were made may arouse suspicion in the reviewer's mind about the trustworthiness of the findings, and may even jeopardize the investigators' credibility and status.
The results of your hygiene
evaluation study may be reported in different ways depending on
the target audience or readership. To begin with, you will have a
written report which will contain a complete record of the study
processes and findings. Once you have completed the report, you
may decide to extract parts of it, and prepare short summaries
for dissemination among the various stakeholders who will expect
to learn about your results. In this section, we will deal with
the complete report first and then suggest additional ways in
which it may be disseminated among specific audiences or
readerships.
Writing a Complete Study Report
At the end of the investigation and analysis processes, you will find yourself with considerable amounts of fieldnotes, charts, and other written records of what you have done. These will all need to be systematically organized, kept in notebooks, and files compiled by hand or on a computer, if available. You can then start putting them together following a report outline, as shown in "Stages of Analysis and Interpretation of Findings" in this chapter. Box 25 provides an example of a report outline.
Writing Separate Summaries for Specific Readers or Interest Groups
You may need to send short summaries such as an executive summary to your project funders, the study population, local community groups, governmental, and/or non-governmental counterparts. It is important to balance well the positive and negative findings when reporting in short, executive summary format. By definition, an executive summary does not allow the reader the benefit of seeing the findings in the context. Evaluation study results are seldom entirely positive or entirely negative, but a combination of the two. Whether they are interpreted as positive or negative depends on who is interpreting and using them.
You may also want to prepare short articles summarizing your findings for dissemination in local and/or regional networks of practitioners working in the fields of health/hygiene education, water supply, and sanitation; research network such as the global applied research net work (GARNET) ) which has a topic network on Hygiene Behaviour, the working group on Promotion of Sanitation, and so on. You will need to bear in mind the interests of each of these groups when deciding what to include, and what language and style to use.
Making Verbal Presentations to Selected Groups and Inviting Their Comments and Suggestions
You may find it beneficial to present partial or full results of your investigation to some of the most important stakeholders in the study in order to elicit their responses to the analysis and interpretation of your findings. For example, in Chapters 5 and 6, we looked at a number of participatory tools for information gathering (mapping, historyline, seasonal calendars, pocket chart) which included the presentation of information gathered to the study participants there and then. Charts, graphs, and other visual displays can be used to present the findings in ways that will interest and stimulate participants. However, only overall results should be given and not details of individual interviews or households.
BOX 25. Outline of a Report
|
Your project may already have trained personnel (e.g., trainer or project spokesperson) who can present the study findings at workshops, meetings and conferences where various audiences may be interested in hearing about your findings.
The type of visual and other materials you can use to present your results will depend on the resources available. Often, summaries of findings written on flip-charts using thick marker pens and big letters (including diagrams, charts, and graphs where appropriate) are the most effective ways to present findings to large groups in both rural and urban areas. These require less financial resources to prepare and can be more creative and fun to do.
Organizing a Discussion or Debate the Findings in Which Opposing Points of View Can Be Aired
This is a
particularly good idea if the level of participation of the
different stakeholders is high and if your findings are likely to
be interpreted significantly different by groups according to
their opposing interests. In the final analysis, comparisons must
be made carefully and appropriately to avoid the drawing of wrong
conclusions.
Many of the methods and
tools described in this handbook lead naturally from collecting
and analysing data (i.e., establishing what the problem is) to
planning what needs to be done to address the issues raised. For
example, a healthwalk may reveal that part of a community is
using a water source particularly vulnerable to pollution for its
drinking water. Indeed, we have seen in Chapter 5 the impact of
information gathered during a healthwalk on project design and
implementation. Similarly, information from focus group
discussions and semi-structured interviews may reveal a higher
incidence of diarrhoea among this group. Presentation of these
findings to the community will almost inevitably lead to a
discussion of what needs to be done to remedy the situation,
moving the emphasis from data collection to implementation. Thus
a hygiene evaluation study does not end with the presentation of
findings. It should lead to follow-up action on the basis of the
findings.
Whether or not participatory approaches are given importance in the evaluation, the end result of the study will be the identification of high risk hygiene practices which currently exist, embedded in a context of local physical conditions, beliefs, and ideas. You will almost inevitably advocate that follow up action should include hygiene promotion activities. The goal of any hygiene promotion project must be to influence people to abandon the high risk practices identified in favour of low risk, safe practices. But, what influences people's decisions to change their normal practice? Many studies have shown that the answer to this question is "not received knowledge alone." Commonly, four factors influencing behavioural change are identified:
• Facilitation. The new practice makes life easier for the person adopting it.
• Understanding. The new practice makes sense in the context of existing local knowledge/ideas.
• Approval. Important and respected people in the community approve of and have adopted the practice.
• Ability to make change. It is physically possible for the person concerned to make the necessary changes.
Below are some examples of how information gathered using this handbook may be fed into an implementation process that takes these four factors into account:
Facilitation. In order to get people to use safe water for drinking purposes, it may be necessary to ensure that there are sufficient protected water sources throughout the community to make it easier and more convenient to use as opposed to traditional, unprotected ones. In planning terms, this may mean continuing a mapping exercise that identified existing sources instead of using the map, with the community, to plan the location of new water points.
Understanding. Hygiene promotion messages and activities are not received by people in a vacuum. Rather they are assessed, accepted, modified, or rejected by people within the context of their existing health concerns and beliefs about illness. A number of similar evaluations have, for example, elicited the local concepts of hot and cold illnesses that need to be treated by controlling diet and reducing intake of some foods. In a number of cases, the promotion of ORS has run into difficulties because diarrhoea is classified as a hot illness requiring treatment with cooling substances, while sugar, a major constituent of ORS, is categorized as hot, therefore rendering ORS an unsuitable treatment. Project implementers have found various ways to overcome such problems including substituting honey (considered a cooling substance) for sugar in one case, and in another, encouraging people to use ORS in conjunction with herbal teas made from guava leaves - a traditional remedy considered cooling and seen to overcome the perceived heating effect of the sugar in ORS.
Approval. In order to enhance the desirability of change, it may be necessary to target hygiene promotion at certain groups of trend setters, such as traditional healers, local leaders, or young mothers who are likely to be copied by their peers. Often this would best be done through a continued use of the group discussion techniques used earlier in the evaluation.
Ability. If behavioural change requires resources, it may be beyond some people's abilities to make the change. Promotion of latrines, for example, may need careful planning with communities, using many of the techniques discussed earlier to enable targeted assistance/subsidies to be allocated to those who would otherwise be unable to make the change.
In projects
where the promotion of low risk hygiene practices
has been achieved, the follow-up action to evaluations may
involve tackling other issues that are next in the list of
priorities. Whatever the outcomes of your study are, we shall be
interested to learn about your experiences of using this handbook
(see Evaluation Sheet at the back of the book).