"You don't have to be able to smell or see air pollution to die from it." That's that how the Associated Press reported news of the latest study on air pollution.
The study in the New England Journal of Medicine (Dec.14) claims to be "consistent evidence that the levels of fine particulates in the air are associated with the risk of death from all causes and from cardiovascular and respiratory illnesses."
"The findings should squelch criticism that earlier research at the Environmental Protection Agency, Harvard and elsewhere was inconclusive, said James H. Ware, dean of the Harvard School of Public Health," reported the AP.
Hardly. The study is simply statistical flim-flammery.
Johns Hopkins University researchers compared death rates with air pollution levels measured by regional monitors for the 20 largest counties in the U.S. The researchers reported a 0.51 percent increase in death rates per 10 micrograms per cubic meter of airborne particulate matter. A 0.68 percent increase was reported for the rate of death from cardiovascular and respiratory causes.
Los Angeles' had the highest level of PM measured at 46 micrograms per cubic meter. Dallas-Ft. Worth had the lowest level at 23.8 micrograms per cubic meter. The results translate into about a 2 percent higher death rate in Los Angeles than in a theoretical area with no particulate air pollution.
The researchers concluded, "The epidemiologic evidence that levels of particulate matter are associated with the risk of mortality and morbidity has prompted the promulgation of a new standard for [particulate matter] in the U.S. and a rethinking of guidelines for particulate matter in Europe. Our analyses provide evidence that particulate air pollution continues a to have an adverse effect on the public's health and strengthen the rationale for limiting respirable particles in outdoor air."
Although the researchers are professors in Hopkins' departments of epidemiology and biostatistics, they apparently would fail introductory-level courses in these disciplines. The study methodology can't possibly produce the conclusion reached.
The study is an "ecologic"-type epidemiology study. Ecologic studies are designed to look for statistical correlations using demographic or population-level data, in this case county death rates and regional air pollution levels.
In contrast to other types of epidemiologic studies, ecologic studies do not consider data on individuals, including how much pollution they were exposed to, their lifestyles and their actual causes of death. In the Hopkins' study, not one death in the 20 counties examined was clinically attributed to air pollution.
Ecologic studies are subject to the "ecologic fallacy." The communities being compared likely differ in many factors. Any one or combination of those factors may be the reason for correlations identified. Ecologic studies cannot prove cause-and-effect relationships because they aren't designed, and don't contain the necessary data and analyses to do so.
Jonathan Samet, the study's lead researcher, once discouraged the use of ecologic studies writing in the journal Health Physics that, "The methodologic limitations inherent in the ecologic method may substantially bias ecologic estimates of risk."
Even putting aside the ecologic fallacy, the study is dead on arrival.
Another basic reality of epidemiology is that weak statistical associations are considered meaningless. The National Cancer Institute put it best stating, "In epidemiologic research, relative risks of less than 2 are considered small and usually difficult to interpret. Such increases may be due to chance, statistical bias or effects of confounding factors that are sometimes not evident."
In the Hopkins study, the reported "relative risks" are 1.051 and 1.068 — barely greater than the "no-effect" level of 1.0 and a lot smaller than 2.0 The Hopkins study's relative risks aren't results; they're random error.
So the Hopkins study provides no scientific basis for concluding that particulate matter increases death rates.
Ware's accompanying editorial states, "The findings of Samet et al. are consistent with those of time-series studies in Europe, and cohort studies in the United States. Thus the evidence in support of an association between ... particulate air pollution and the mortality rate is consistent, is not affected by differences in statistical methods and can be generalized."
But the only generalization that can be made is that all these studies are ecologic in nature and all report similar weak statistical associations. No study scientifically links airborne particulate matter with higher death rates.
Ecologic studies are not without utility, though. They may be useful in identifying areas for future scientific — as opposed to simply more statistical — research.
Instead of scientific research, though, the Hopkins researchers have leapfrogged to urging more stringent regulation — consistent with recent efforts by the EPA, which provided half the funding for this study. The EPA's 1997 rules ratcheting-down air quality standards — estimated to cost as much as $100 billion per year — are now being reviewed by the Supreme Court.
A federal appellate court tossed out the EPA rules in May 1999 ruling that Congress had not given the EPA sufficient instructions on how to set air pollution standards. The court ruled the EPA was, in effect, impermissibly making law, not enforcing it. Only Congress can make laws.
Coincidentally, the Supreme Court just ruled in Bush v. Gore that hand-recounts of votes without standards violated the equal protection guarantee of the Constitution. With any luck, the Court will rule that science without standards is similarly problematic.
— Steven Milloy is a biostatistician, lawyer and adjunct scholar at the Cato Institute and publisher of Junkscience.com.
Get all the stories you need-to-know from the most powerful name in news delivered first thing every morning to your inbox