Did the economy create jobs during President George W. Bush’s first term?

Until last Friday’s jobs report, it seemed that the American jobs machine was firing on all cylinders. But the Labor Department’s payroll survey shows 78,000 new jobs created in May (in contrast with the 376,000 new jobs in its household survey). This will embolden the pessimists who insist that the first Bush term was a failure because relatively few jobs were created during those four years.

Leave aside the fact that jobs are only one of the measures of economic performance. The rapid growth rate in GDP, stable inflation, a housing boom, and world-beating productivity growth are all just as important and are clearly successes for Bush. But even if jobs are the sole measure, Bush’s first term is still one of the best ever.

First off, the Bush years ended with more Americans working than ever before. The answer is clouded, however, by a lingering controversy over which of the two Labor Department surveys mentioned above is the best measure of job creation. The payroll survey, which polls employers, indicates a razor-thin gain of 120,000 jobs between January 2001 and January 2005. The household survey, which contacts workers directly, indicates a net increase of roughly 2.5 million employed. Which is correct?

The president’s critics prefer the payroll survey, and they point to Labor Department papers that say it’s better. But the payroll survey is certified as more reliable only in the short run, on a month-to-month basis. Over longer periods of a year or more, payroll data gets flaky. Many entrepreneurs, LLC partners and self-employed workers aren’t counted on any payroll. And then there are the 250,000 to 1 million phantom jobs “lost” to changing turnover rates, a flaw acknowledged by Labor but not widely known, even by economic reporters.

Even Labor admits it cannot fully explain the disconnect between the two surveys. It helps, though, to take a closer look at a statistical earthquake in the economic indicators that occurred in 2002 and 2003. The middle two years of the Bush presidency is where uncertainty plagues the data.

In 2002 and 2003, payrolls were stuck in a “jobless recovery” while the number of working Americans grew by 2 million, according to the household survey. Last summer, the disparity between surveys was so big that the Labor Department had to publicly defend its payroll survey. The household survey said employment was up by 629,000 in July 2004, while payrolls grew a meager 32,000. It echoes the situation this month.

But this month’s data include something else -- revisions to the data from July 2004. Today Labor says that payrolls actually rose by more than 80,000 that month, triple the initial estimate. (Payroll numbers are always revised during the first two months after their preliminary release, then again annually to update methodological quirks.)

Payrolls began their recovery in August 2003, and since then the two surveys seem generally in sync. This doesn’t mean the controversy is over. Something important happened to make the two surveys diverge before August 2003.

One should consider what happened at the end of 2001 that caused a sharp change in how the workforce behaved, and then figure out how that changes survey responses. We can rule out the recession itself as a cause of the household-payroll divergence, because it hit in early 2001, before the two surveys diverged. Besides, recessions haven’t caused a divergence like this before. The seismic event that seems most likely to have changed work behavior is 9/11 and the war mentality afterwards.

Indeed, a reduction in job changing, caused by 9/11, probably caused the payroll phantoms mentioned earlier. As explained in the Winter 2005 issue of The Public Interest:

“[J]ob-changing from one employer to another, which had averaged 3 percent per month in the 1990s, declined by about 0.2 percentage points per year after 2001, settling at 2.4 percent in 2003, where it remains today. This seemingly small change meant ... roughly one million fewer workers were being double-counted on payrolls, a statistical change that the payroll survey registered as one million ‘lost’ jobs.”

To this day, the payroll survey incorrectly measures jobs and job losses by not accounting for the impact of job turnover in its methodology.

One thing that policymakers should be able to agree on is the need for accurate economic data. Stimulating an economy at the wrong time or failing to see economic weakness until it is too late can inflict unnecessary damage. That is why the recent efforts by Reps. David Dreier, R-Calif., and John Boehner, R-Ohio, to help the Labor Department improve its methodologies are a step in the right direction.

For the time being, though, payroll data should be reported with an asterisk. America needs to realize its economy actually is much stronger than that information indicates.

Tim Kane, Ph.D., is a research fellow in macroeconomics in the Center for Data Analysis at The Heritage Foundation.

Tim Kane is a full time Research fellow at the Hoover Institution. Kane is a graduate of the U.S. Air Force Academy and holds a Ph.D. in economics from the University of California, San Diego. He is co-author (with Glenn Hubbard) of "Balance: Why Great Powers Lose It and How America Can Regain It," (Simon & Schuster 2013).