Has science lost its way?

close

Video

Bill Nye among demonstrators at 'March for Science' in DC

James Rosen reports from Washington D.C.

There were many impassioned political orations delivered at Saturday’s so-called March for Science. Sadly, I heard nothing about the truly serious problem plaguing science today.

The single greatest threat to science right now comes from within its own ranks. Last year Nature, the prestigious international science journal, published a study revealing that “More than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments.”

Reproducing experiments is to science’s credibility what verifying financial statements is to a company’s credit rating. The astronomical failure to confirm research that was published in highly respected, peer-reviewed journals suggests something is very wrong with how science is being done. Fifty-two percent of the 1,576 researchers surveyed for the study call it “a significant crisis.”

The crisis appears to be exacerbated by a marked reluctance among scientists to out the epidemic, to contact one another about irreproducible experiments. “That may be because such conversations are difficult,” Nature speculates. “If experimenters reach out to the original researchers for help, they risk appearing incompetent or accusatory, or revealing too much about their own projects.”

The crisis afflicts even science’s most revered ‘facts,’ as cancer researchers C. G. Begley and Lee Ellis discovered. Over an entire decade they put fifty-three published “landmark” studies to the test; they succeeded in replicating only six – that’s an 11% success rate.

A major culprit, they discovered, is that many researchers cherry-picked the results of their experiments – subconsciously or intentionally – to give the appearance of success, thereby increasing their chances of being published.

“They presented specific experiments that supported their underlying hypothesis, but that were not reflective of the entire data set,” report Begley and Ellis, adding this shocking truth: “There are no guidelines that require all data sets to be reported in a paper; often, original data are removed during the peer review and publication process.”

Another apparent culprit is that – and it’s going to surprise most of you – too many scientists are actually never taught the scientific method. As graduate students, they take oodles of courses in their chosen specialty; but their thesis advisors never sit them down and indoctrinate them on best practices. Consequently, remarks University of Wisconsin-Madison biologist Judith Kimble: “They will go off and make it worse.”

This observation seems borne out by the Nature study, whose respondents said the three top weaknesses behind science’s reproducibility crisis are: 1) selective reporting, 2) pressure to publish, and 3) low statistical power or poor analysis. In other words, scientists need to improve on practicing what they preach, which is: 1) a respect for facts – all of them, not just the ones they like, 2) integrity, and 3) a sound scientific method.

No one is suggesting science is rife with evil-minded cheats or incompetents – certainly, I am not. As a scientist – who slept an average of five hours a day pursuing PhD studies in physics, math, and astronomy at Cornell – I’m all too familiar with the pressures that can cause my honest, well-intentioned colleagues to succumb to subtle errors in practice and judgement.

The attendees of the so-called March for Science made a lot of noise about wanting more money and respect from the public and government – what group wouldn’t want that? But nary a whisper was heard from them or the media about science’s urgent reproducibility crisis. Leaving unspoken this elephant-sized question: If we aren’t able to trust the published results of science, then what right does it have to demand more money and respect, before making noticeable strides toward better reproducibility?