Most biomedical research cannot be replicated
New studies looking at the work of scientists in the biomedical field has found that their research is difficult if not impossible to replicate, partly because much of their raw data is never made available for other researchers to review.
But over the past several years, a growing contingent of scientists has begun to question the accepted veracity of published research—even after it’s cleared the hurdles of peer review and appears in widely respected journals. The problem is a pervasive inability to replicate a large proportion of the results across numerous disciplines.
In 2005, for instance, John Ioannidis, a professor of medicine at Stanford University, used several simulations to show that scientific claims are more likely to be false than true. And this past summer Brian Nosek, a professor of psychology at the University of Virginia, attempted to replicate the findings of 100 psychology studies and found that only 39 percent of the results held up under rigorous re-testing. “There are multiple lines of evidence, both theoretical and empirical, that have begun to bring the reproducibility of a substantial segment of scientific literature into question,” says Ioannidis. “We are getting millions of papers that go nowhere.”
There’s a lot more. Read it all. It appears that much if not all of biomedical research is suspect. Their conclusions might be correct, but their methods are questionable, at best.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
New studies looking at the work of scientists in the biomedical field has found that their research is difficult if not impossible to replicate, partly because much of their raw data is never made available for other researchers to review.
But over the past several years, a growing contingent of scientists has begun to question the accepted veracity of published research—even after it’s cleared the hurdles of peer review and appears in widely respected journals. The problem is a pervasive inability to replicate a large proportion of the results across numerous disciplines.
In 2005, for instance, John Ioannidis, a professor of medicine at Stanford University, used several simulations to show that scientific claims are more likely to be false than true. And this past summer Brian Nosek, a professor of psychology at the University of Virginia, attempted to replicate the findings of 100 psychology studies and found that only 39 percent of the results held up under rigorous re-testing. “There are multiple lines of evidence, both theoretical and empirical, that have begun to bring the reproducibility of a substantial segment of scientific literature into question,” says Ioannidis. “We are getting millions of papers that go nowhere.”
There’s a lot more. Read it all. It appears that much if not all of biomedical research is suspect. Their conclusions might be correct, but their methods are questionable, at best.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
No surprises here! I remember one chemistry proff indicating he tells his postgrad students not to believe until they can replicate. The bio-sciences and especially psychology is in worse shape.
The publish or perish theme leads to this state. Conferences are a way for researchers to meet others in the field face to face and cut deals to fast track reviews. The anthropogenic global warmists have shown how successful this can be. Research grants follow.
Basically, human nature or the human condition leads to this type of “Falsehood” in any field.
Phill O-
Good stuff.
My Daughter is in drug-discovery & screening (private industry), they have a whole department that does nothing but replicate target molecules. (They can’t just make-it-all-up.)
I’m in Human Services and you should see the psycho-babble that passes for “research.” You can literally say anything you want and few people will ever challenge you.
Personally, I have a strong (Skinnerian) Behavioral bent and spent a huge chunk of grad school replicating older animal studies and trying to overlay those onto human subjects. Our main goal was to functionally define ‘everything’ and avoid mentalistic causes, as much as is possible.
(Far too much of “Psych” in particular, is a jumble of explanatory-fictions and circular-reasoning.
Psychobabble
Alan Parsons Project
https://youtu.be/i7RbZNT5A2A
4:50
The book Rigor Mortis documents this very well along with lots of stories, many of the sources of error and what is being done (not much). It’s a great read. By an NPR reporter no less.
I was really amused when I was listening to Science Friday and they interviewed the author the same weekend the March for Science was happening. Once the host heard that most biomedical research is flawed, he dropped the author like hot rock.
I can understand the pressure to publish even though I am not in that type of field myself.
I have a friend who is a microbiologist. He worked for 4 years on a project that never worked out. Luckily he is not the type of person who has ever really felt the pressure to publish or parish as they say. He started in the field because it was fun and 30 years later is still in it for fun.
I believe its just as important to publish failures as it is to publish successes. Some people might not though and for personal reasons take it personally. Or they boast to colleagues and just can’t admit they failed or their ideas didn’t pan out.
pzatchok Absolutely!
The Wright Brothers had a problem when they discovered that the previously published data for wing lift and drag were unreliable.
They had built their second Kitty Hawk glider using that data, only to discover that their second glider flew worse than their first. Realizing that the other two experimenters did not have accurate data was quite a disappointment to them, and in addition to the time that they lost making a worse flyer due to the poor data, they had to build their own experimental apparatus and spend time getting accurate data in order to improve their designs.
Research that had been done before turned out to be unrepeatable by the Wright Brothers. In their case, they were able to find the right answers in order to make their famous advancement. But what happens when poor data and conclusions are not so easily recognized?