A Methodological Crisis in the WASH Sector: Crunching Numbers and Throwing Them over the Wall

Introduction

It has been a shock to many practitioners in the Water Sanitation and Hygiene (WASH) Sector, that there is now increasing skepticism in academic circles as to whether improvements in water, sanitation and handwashing are in fact sufficient to reduce diarrhoea and stunting [1]. The WASH Sector has been built upon the conventional ‘germ theory’ which has long argued that clean water, safe sanitation and good hygiene is an essential cornerstone of any public health intervention to improve primary health [2]. One much cited review claimed that handwashing with soap can reduce diarrhoea by 47% [3], whilst an older classic calculates that safe drinking water alone can reduce diarrhoea by 15%, sufficient drinking water by 27%, safe hygiene by 35% and safe sanitation by 37% [4]. Based on such scientific reassurance countless millions have been spent on improving WASH facilities since the Water Decade Opinion in the 1980’s. At the turn of the century there was further strong international reinforcement when the United Nations through the Millennium Development Goals, called on the developed world to halve the remaining number without clean water and safe sanitation by 2015 [5]. Whilst many countries, particularly in Sub Saharan Africa, failed to achieve these goals, some progress seemed to have been made by the WASH Sector despite gains being undermined by rapid population increase. Since 1990 the Burden of Disease attributable to inadequate WASH appears to have fallen from 4.2% to 1.5% of global mortality. However, there is still much left to do, with diarrhoea alone still estimated to kill almost a million children <5 worldwide annually. It is estimated (and still believed by many) that many thousands of deaths per annum could be prevented by effective control of harmful pathogens through clean water, non-risk hygiene and safe sanitation [6].

The Disillusionment

In the past five years, increasing evidence provided by Randomized Control Trials, (normally considered the gold standard of rigorous research) are demonstrating the opposite. Two recent systematic reviews found little impact on health due to different methodologies used in recent years to achieve improved hygiene and sanitation: Community Led Total Sanitation (CLTS) which aims mainly to achieve Open Defecation Free villages [7]. and Social Marketing which targets mainly handwashing with soap as a means to prevent diarrhoea [8]. These unexpected findings were compounded when clustered-Randomized Control Trials (cRCT) in Kenya, Zimbabwe and India all found little impact on diarrhoea and no impact on growth of children in simple WASH interventions [9].

A group of leading donors and academics tried to explain away the situation with convoluted language: ‘these three trials do not show that WASH in general cannot influence child linear growth, but they do demonstrate that these specific interventions had no influence in settings where stunting remains an important health challenge [10].’ As ‘settings’ of this nature are to be found in virtually all developing countries where WASH interventions are introduced this qualified explanation does little to reassure practitioners. As policy is informed by such science the relevance of most of the activity of the WASH sector is now in jeopardy, unless we can find some convincing explanation for the failure of such RCTs. Lack of investment in WASH would impact on the lives of millions, denied the opportunities they might otherwise have enjoyed.

Perhaps practitioners should accept these disappointing findings with the humbling realization that in the WASH sector we still have not done enough to achieve the goal of improved community health. That the multiple pathways of transmission of diarrhoea are much more complex than we ever imagined and will take a much more all-inclusive approach to achieve any effect on health. More, not less, investment is needed.

No Magic Bullet

It has taken the WASH Sector over half a century to come to terms with the fact there is no short cut to progress, no magic bullet: neither installing boreholes, hand washing with soap nor achieving ‘Open Defecation Free’ (ODF) villages apparently improves community health. But how did we ever imagine that one or two face-to-face interactions with the community would change behaviour or improve sanitation for any length of time? At last, after 15 years of rolling out CLTS, influential agencies in the WASH Sector are calling for ‘revitalizing CLTS’ or ‘WASH++’ [10] i.e, not just ‘ODF’ but a whole range of hygiene measures (pot racks, rubbish pits, clean utensils, safe food storage, and personal hygiene as well as the usual, ‘handwashing facility & soap’ and ‘improved sanitation’. This and much more is what Community Health Club approach has been advocating from the start [11], where a six-month period involving weekly sessions is considered the minimum if hygiene behaviour change is to be adopted for life [12]. However, a long-term commitment to this holistic approach has been largely side- stepped by donors looking for a quicker fix.

The Rwandan Success Story

Rather than despair at the perceived lack of health impact we could revisit areas where CHCs have been operational for many years. In Zimbabwe, for example, in an area where CHC had been operational for 9 years with members in 80% of households in the catchment of Health Centre, reported cases of diarrhoea, bilharzia, malaria, as well as skin, eye and respiratory disease decreased consistently over that period [13]. Pilot projects of CHC have been started in over 20 countries reaching over two million people [14] but Rwanda is the only country worldwide which has systematically scaled up the Community Health Club approach, implementing a nation-wide Community Based Environmental Health Promotion Programme (CBEHPP) since 2010 [15]. This programme resulted in over 14,000 Community Health Clubs being started in virtually all of the villages in the country by 2015. Rwanda was one of only five countries in Africa that made good progress in meeting the sanitation MDG [16]. Monitoring records from NGOs in districts where CHCs had provided at least 20 health promotion sessions were showing that hygiene standards were improving [17].

The Effect of Number Crunching

Convinced there would be some impact on health from this comprehensive programme, it was a shock to such practitioners when the cRCT published its controversial interpretation pronouncing that they ‘question the value of implementing this intervention at scale for the aim of achieving health gains.’ [18]. One of the trialists shrugged off this bombshell by saying; ‘We just crunch the numbers and throw them over the wall!’ But numbers affect policy; policies affect people’s living standards.

If such recommendations had been heeded by major donors in Rwanda, this counter-intuitive finding could have derailed a successful programme. It was to the credit of the Rwandan Ministry of Health that they decided to use ‘common sense’. The anecdotal observations from the field which had been showing a consistent pattern of high community response were believed; the premature conclusions of the cRCT skeptically received.

It is not unreasonable to expect that if over 80% of the rural population in Rwanda is exposed to regular training for sufficient time in Community Health Clubs there will, in time, be some decrease in diarrhoea and perhaps even stunting. Indeed, a small one-year post-intervention study in one of the most successful CHCs of the RCT District did indeed demonstrate that relative risk reduction values were statistically significant for intestinal worms and malnutrition [19].

Time Enough for Change

An insightful paper in the debate over RCT ability to measure health impact in community settings, has highlighted how the time constraints of most vertical research interventions precludes full community response. For sufficient improvement in hygiene and sanitation facilities to be achieved by a critical mass of the population, it takes longer than just the limited time available to most RCTs [20]. This indeed seems to ring true in the case of Rwanda. Two years after the publication of the disputed cRCT data in Rwanda, regular Ministry of Health monitoring data taken in the very same Community Health Clubs as were sampled by the cRCT, were showing strong response in all the same indicators as those which had apparently registered little change at the time of the RCT [21]. Ignoring the inexplicable cRCT results, the government enhanced the CBEHPP and extended the role of CHCs to include not only WASH but integrated it with a strong nutrition component to further address stunting in an updated roadmap in 2020, which now is known as INWA (integrated Water and WASH) [22].

History may teach us patience: although the transmission of cholera was identified by John Snow in 1830 [23], it was only when the sewerage system was finally completed in London in the late 19th century that public health began to improve. However, only well into the 20th century did infant mortality due to diarrhoeal disease start to decrease in London. Brazil has taken three generations to show improved child growth and development due to universal access to basic WASH infrastructure [9]. Rwanda, with such a consistent and nationwide approach to addressing lack of water, poor sanitation and hygiene is likely to follow this trajectory ahead of most other countries in Africa which still have no real public health strategy to prevent communicable diseases.

Conclusion

Statistical significance should not be allowed to trump honest observation of good development practice. With an obsessive fear of committing selection or interviewer bias, are we not prone to allowing the glittering science of the gold standard to blind us to our own direct observation?

Even if there were no impact on health, and even if it takes more effort than we have given to-date, there are ample justifications for investment in WASH programmers: dignity, convenience, human rights and gender equity [24], and we may also find lives are being saved if we can only find a more reliable way to measure such outcomes than an RCT. Some pre-digital ‘Common Sense’ is urgently needed to salvage the WASH sector before the baby WASH is thrown out with the RCT bathwater.