The EPA's Secret Science

"Independent Institute Finds Key Pollution Studies Are Sound Science," blared a recent press release from the American Lung Association.

Yet one of the most controversial "junk science" issues of recent years — whether air pollution causes premature death — will remain controversial for the foreseeable future despite activist claims and media headlines.

The ALA goes on to say studies show "once again that air pollution shortens lives, and strong [Environmental Protection Agency] standards are needed to protect public health."

But the reality is somewhat different — at least according to Health Effects Institute, the organization whose research the ALA cited.

The saga started in November 1996 when the EPA proposed more stringent regulation of airborne particulate matter, otherwise known as soot. The EPA initially claimed further regulation would prevent 20,000 "premature deaths" every year at an annual cost of $8 billion. By valuing each life saved at about $5 million, the EPA estimated the monetary benefits of saving these lives at $100 billion per year.

But the validity of the EPA's estimates is in question. In particular, the estimate of premature deaths was based on the "Pope study," so called for its lead researcher, C. Arden Pope of Brigham Young University. The Pope study reported that airborne particulates were associated with a 17 percent increase in premature deaths. But this result constituted only a weak statistical correlation — not scientific proof of a cause-and-effect relationship between air particulate pollution and premature death.

Although the study included more than 550,000 people, the researchers did not measure the level of exposure to air pollution for even one study subject. Instead, they guessed how much pollution these individuals might have encountered. The researchers also failed to look at the subjects' diets, income, health histories, genetic predispositions to illness, exercise habits and social habits — all well-established risk factors for premature death.

The study did adjust for some factors — including smoking habits, education level and occupational exposures — but additional adjustments could easily negate the purported 17 percent increase in risk.

A further problem was that no one has ever demonstrated how typical levels of airborne particulates could cause premature death.

In this context, the reported correlation could easily have been a statistical artifact. According to the National Cancer Institute, "In epidemiologic research, [risks of less than 100 percent] are considered small and usually difficult to interpret. Such increases may be due to chance, statistical bias or effects of confounding factors that are sometimes not evident."

But the EPA did not back down — even when an elementary statistical error in the EPA's calculations was discovered that knocked down the lives-saved estimated to 15,000 or when some economists estimated the proposal's cost to exceed $100 billion annually.

The controversy prompted Congress to ask the EPA to produce the Pope study's raw data so independent scientists could check the results.

The EPA initially balked, saying there was no purpose in any re-analysis of the EPA-funded study. Finally, after issuing the regulations, the EPA did provide access to the data — to the Health Effects Institute, a Massachusetts-based research organization funded by the EPA and the auto industry.

HEI issued its report last week, three years after the regulations were finalized. The results are not quite as the ALA press release touts.

Using essentially the identical methodology as the Pope study researchers, HEI produced virtually the same results — hardly a surprise. A naive media reported this "replication" in headlines such as "Research on Air Particles Passes Muster" and "Studies Back Particulates' Link to Death."

But the re-analysis has essentially the same shortcomings as the original study. The researchers did not factor into their analysis the effect of diet or genetics on death rates. Data on exercise habits were factored in, and ended up reducing risk estimates by almost 30 percent. But the quality of the data on exercise is debatable; incredibly, it indicates no difference in death rates between non-exercising study subjects and heavy-exercising study subjects. Better quality exercise data could have an even more dramatic effect on the numbers.

There is a similar data-quality problem with health history. The re-analysis paradoxically indicates that healthy study subjects had twice the risk of premature death of diseased study subjects.

Neither of these data problems is surprising given how the study data were collected. The American Cancer Society amassed the data by having 70,000 volunteers — not trained data-collection specialists — go to their friends, family and neighbors and ask personal, health-related questions. No effort was made to verify or validate the study subjects' responses.

It's no wonder the HEI concluded its report: "It is important to bear in mind that the results of our re-analysis alone are insufficient to identify a causal relation with mortality."

So why did the ALA jump to its conclusion? That question perplexed HEI's president Daniel Greenbaum, who noted the ALA issued a press release last April titled "New Health Research 'Vindicates' EPA; Soot Particles Are Deadly, Lung Association Notes" — three months before the HEI report was made available to the public.

The explanation, though, lies with the finances and politics of the ALA. As first reported by Investors Business Daily in January 1997, "The ALA has had a long — and lucrative — relationship with the EPA." In the years before the EPA air-pollution proposal, the agency gave the ALA almost $5 million — despite the ALA suing the EPA almost every year claiming the agency wasn't complying with the nation's air-pollution laws.

"If you think the EPA is upset with the ALA suing them, think again," said Scott Segal, a Washington, D.C.-based attorney. "Truth be known, the EPA wants to be sued, because every time they are sued it expands the reach of the Clean Air Act."

Unfortunately for the ALA and the EPA, another lawsuit has pre-empted the claimed "vindication" of the Pope study. A federal appellate court overturned the EPA regulations in May 1999 on a number of grounds. The case is pending before the Supreme Court.

No doubt more research will continue into the potential health effects of air pollution. But another byproduct of the Pope study controversy may change the shape of the next scientific debate. Because of difficulty in obtaining the Pope study data, a federal law was enacted in October 1998 requiring that federally funded scientific data used to support federal policy must be publicly available through the Freedom of Information Act.

With any luck, the next debate over the potential health effects of air pollution won't be hampered by EPA's "secret science."

— Steven Milloy is a biostatistician, lawyer, adjunct scholar at the Cato Institute and publisher of Junkscience.com.