At Wonkblog, Brad Plumer highlights a new NBER paper that’s disappointing to those who hoped that distributing cleaner cookstoves in India and other countries would be an easy way to improve respiratory health and help slow global warming. Many low-cost, traditional cookstoves belch soot, which is bad for the lungs of people who spend long hours near the stoves and for the ice that melts more quickly when soot particles settle on it. Cleaner stoves would improve respiratory health and could run on less fuel, and these changes could be of particular benefit to women, who often spend hours each day collecting stove fuel and cooking.

Plumer explains that aid groups have noticed the potential for getting big health improvements from wide distribution of relatively cheap improved stoves. But, he explains, new research suggests it won’t be so easy:

One problem is that having a cheap, clean technology is no guarantee that it will be properly adopted. In a new NBER paper, economists Rema Hanna, Esther Duflo and Michael Greenstone note that there’s been very little evidence on whether these stoves work in the real world. They looked at a randomized control trial that handed out cleaner stoves to 15,000 people in Orissa, one of India’s poorest rural areas, and tracked the results over five years. The stoves were a bargain, costing about $12.50 a pop, and they used a chimney to keep smoke away from the users.

What Hanna and her colleagues found is that in the first year of using the stoves, households saw a serious drop in smoke inhalation. The cleaner cookstoves were working exactly as they did in the laboratory. But in the years after that, the stoves stopped working effectively. “We find no evidence of improvements in lung functioning or health and there is no change in fuel consumption (and presumably greenhouse gas emissions),” the authors write.

So what went wrong? Basically, none of the earlier evaluations of the clean cookstoves had taken into account how households in places like India would actually use the things. In early tests, there were trained technicians on hand at all times to inspect and repair the stoves. Not surprisingly, households used the stoves frequently. But when the technicians departed and the owners had to clean the chimneys themselves, they lost interest over time. People were spending too many hours conducting repairs and eventually just preferred to switch back to indoor cooking fires.

The study has several strengths. It it followed households for four years after they were initially offered new stoves, which the authors note is “virtually unprecedented in evaluations of health interventions or other new technologies where households learn about the benefits and maintenance needs over time.” The researchers conducted air monitoring and also studied specific health endpoints, including lung functioning (measured with spirometry), infant birth weight and mortality rates, and blood pressure. The paper authors are all affiliated with the Abdul Latif Jameel Poverty Action Lab, which specializes in conducting randomized evaluations on poverty-reduction programs. (New Yorker subscribers can read more about J-PAL in Ian Parker’s 2010 profile of Esther Duflo, “The Poverty Lab.”)

I’ve seen some evidence (e.g., in another gated New Yorker piece, Burkard Bilger’s 2009 “Hearth Surgery“) that aid groups pay attention to the kinds of cookstoves that will be accepted in the areas targeted for stove distribution. They have to fit easily into people’s homes, do a good job cooking staple foods like tortillas or injera, and require straightforward feeding with fuels that are readily available locally. They should be made with local materials and would ideally be affordable to local residents (though subsidies could be offered to lower households’ costs). It would be wrong to think all aid groups have simply assumed that households everywhere will readily adopt the same US- (or European-) designed stove.

What seems to be unique about this particular study is that it considered how well stoves would be maintained and used over the long term — in other words, would the presumed reductions in soot emissions and related health improvements continue past the initial year or two of new stove use? The results are disappointing, but important for aid organizations and local governments to consider.

Maintenance has turned out to be a stumbling block for many well-intentioned global health interventions. Water and sanitation projects also have a mixed track record when it comes to long-term success. Water pumps or shared toilets can break down and never get fixed, either because the parts are too expensive or hard to obtain, or because there’s no institutional structure in place to ensure ongoing maintenance and upkeep. I’ve heard that some of the savvier donors are starting to require grantees to monitor and report on longer-term outcomes for water projects, and that trend should continue. Maintenance may not sound exciting, but it’s important for health.