(iStock)
Selective reporting of positive results from drug clinical trials is a well-documented problem, but a new study finds that trials of surgical treatments have an even worse track record.
Researchers who looked at hundreds of surgery clinical trials found that many set out to answer one question but ended up focusing on something else, leading to less reliable evidence being published. Others never bothered to state a goal for their study or published results while the trial was still going on.
"We know that this is a huge problem in medicine, and one that we've known about for three decades," Dr. Ben Goldacre, a UK doctor and founder of Alltrials.net told Reuters Health by email.
"It's a huge cultural blindspot for doctors, academics, industry, regulators, and policy makers, but it knocks the legs out from underneath evidence based medicine," said Goldacre, who was not involved in the study.
It's harder for doctors and patients to make informed decisions about which treatment or, in this case, surgical procedure, is best when the evidence from a study is modified or absent, he said.
"If you measure lots of different outcomes, and then allow yourself to select and report the one that flatters the treatment the most, then you're misusing this basic statistical tool," Goldacre said.
Gerjon Hannink led a team of researchers at the Radboud University Nijmegen Medical Center in Nijmegen, The Netherlands, comparing the results of 327 published surgical trials with the goals set out when the trials were registered.
Putting basic information about the aims of a trial into a clinical trial registry before recruiting participants is generally a prerequisite of publication. The International Committee of Medical Journal Editors began in 2005 requiring clinical trials to be entered into a registry before they begin, and many journals include that policy in their instructions to authors.
Despite that rule, the new study found that up to half of surgery clinical trials were still not registered correctly before their results were published, according to the report from Hannink's team in Annals of Surgery.
They found that out of 327 trials, fully a third (109 trials) had never been registered and another 48 were only registered after the study was completed. Of 152 trials that had been entered into a registry in advance with an end goal clearly defined, half changed that goal when the results were published, or left it out entirely.
That's a 50 percent rate of reporting bias in surgical trials, compared to estimates of 30 or 40 percent in general medical studies for drugs and other treatments, according to the authors.
"We expected the situation to be worse for surgical trials as compared to internal medicine," Hannink told Reuters Health by email. "However, the difference found was larger than we had expected."
Hannink acknowledged that it's harder to do the gold-standard, double-blind studies for surgeries than for pills because it's difficult to keep the surgeon or the patients from knowing whether they're involved in a new procedure or getting the usual treatment for comparison.
With drugs, pharmaceutical companies tightly control most testing of their medications and they know that regulatory agencies will scrutinize the results closely, he added, but there's no equivalent oversight for surgical procedures.
Incomplete or modified publication of results can have serious consequences, Hannink said. It can lead to the promotion of ineffective or harmful treatments or to expensive therapies seeming better than cheaper alternatives when in fact they are not, and that's "the worst possible situation for patients, health care professionals, and policy makers," he said.
"Sometimes this is about money, or reputation: people may not want to publish a trial that undermines their product, or the cherished theory that underpins their career," Goldacre said. "But equally, sometimes it's simply thoughtless, or what we might have to call incompetence."
"This problem has been well documented for three decades, and everything we have tried so far has failed, laws have been ignored, codes of conduct likewise," Goldacre said.
There's no clear solution, except to provide incentives for researchers to follow the rules and for journals and their editors to enforce those rules, experts said.
Goldacre and several partners launched a petition at Alltrials.net in January to campaign for full publication of all results of clinical trials in the future and all those going on now or already completed, mainly by urging regulators and governments to enact legislation requiring publication.
"Full publication of complete trial results is essential if clinicians, patients, policy makers and others are to make well-informed decisions about health care," Hannink said.
Stay up-to-date on the biggest health and wellness news with our weekly recap.