MRI software bug invalidates 40,000 research papers
The uncertainty of science: A bug just discovered in the computer software used by MRIs to measure brain activity could invalidate 15 years of research and 40,000 science papers.
They tested the three most popular fMRI software packages for fMRI analysis – SPM, FSL, and AFNI – and while they shouldn’t have found much difference across the groups, the software resulted in false-positive rates of up to 70 percent. And that’s a problem, because as Kate Lunau at Motherboard points out, not only did the team expect to see an average false positive rate of just 5 percent, it also suggests that some results were so inaccurate, they could be indicating brain activity where there was none.
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]
The research the article described is focused entirely on the problems the software causes for past research. It makes no mention of the problems this software bug might cause for actual medical diagnosis Was the treatment of any patients effected by this bug? It does not say.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
The uncertainty of science: A bug just discovered in the computer software used by MRIs to measure brain activity could invalidate 15 years of research and 40,000 science papers.
They tested the three most popular fMRI software packages for fMRI analysis – SPM, FSL, and AFNI – and while they shouldn’t have found much difference across the groups, the software resulted in false-positive rates of up to 70 percent. And that’s a problem, because as Kate Lunau at Motherboard points out, not only did the team expect to see an average false positive rate of just 5 percent, it also suggests that some results were so inaccurate, they could be indicating brain activity where there was none.
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]
The research the article described is focused entirely on the problems the software causes for past research. It makes no mention of the problems this software bug might cause for actual medical diagnosis Was the treatment of any patients effected by this bug? It does not say.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
Wow!… Faults positives up to 70%? I wonder if the brochure from the MRI salesman said that they guarantee up to 70% more patients?
Now we know why Canada has so few MRI machines. (I wonder if their software maker is the same company that runs our voting computers…)
Max–
in all seriousness however, the reason we have a lot of MRI machines in the USA, is/was primarily the free-market.
20 years ago it was perfectly legal for Doctors to buy/invest-in, their own MRI machines, load them on tractor-trailers, and run them almost 24/7. (often, in the parking lot of their office’s)
In the town where I lived at the time, two medical groups did exactly that & literally years before the local Hospital received “permission” to put one in.
Various “health-care reforms,” primarily in the last 8 years, made that illegal. (It’s a “conflict of interest,” apparently & we need to be “saved” from “exploiting doctors.”)
-Fortunately, we have a huge installed base of MRI machines (& CT scanners) already in place. I seriously doubt we would be in that position, if MRI machines were invented today, and controlled under the ACA.
As for the software errors; I know just enough Statistics to be dangerous, but that being said– if the error is consistent across time, software, & machines, are there not mathematical ways to extract that out, to a least some degree? and maybe not invalidate every single study??
Tangentially– personally, I’m highly suspect of the “active” real-time MRI scans they tout as being evidence for/against someone’s theory of brain-activity, but that’s a different topic above & beyond inherent software errors.
Interesting post. I agree with you on the last thing you mentioned in your comment above, regarding the “active real time MRI Scans”
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]