MRI software bug invalidates 40,000 research papers
The uncertainty of science: A bug just discovered in the computer software used by MRIs to measure brain activity could invalidate 15 years of research and 40,000 science papers.
They tested the three most popular fMRI software packages for fMRI analysis – SPM, FSL, and AFNI – and while they shouldn’t have found much difference across the groups, the software resulted in false-positive rates of up to 70 percent. And that’s a problem, because as Kate Lunau at Motherboard points out, not only did the team expect to see an average false positive rate of just 5 percent, it also suggests that some results were so inaccurate, they could be indicating brain activity where there was none.
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]
The research the article described is focused entirely on the problems the software causes for past research. It makes no mention of the problems this software bug might cause for actual medical diagnosis Was the treatment of any patients effected by this bug? It does not say.
Readers!
Please consider supporting my work here at Behind the Black. Your support allows me the freedom and ability to analyze objectively the ongoing renaissance in space, as well as the cultural changes -- for good or ill -- that are happening across America. Fourteen years ago I wrote that SLS and Orion were a bad ideas, a waste of money, would be years behind schedule, and better replaced by commercial private enterprise. Only now does it appear that Washington might finally recognize this reality.
In 2020 when the world panicked over COVID I wrote that the panic was unnecessary, that the virus was apparently simply a variation of the flu, that masks were not simply pointless but if worn incorrectly were a health threat, that the lockdowns were a disaster and did nothing to stop the spread of COVID. Only in the past year have some of our so-called experts in the health field have begun to recognize these facts.
Your help allows me to do this kind of intelligent analysis. I take no advertising or sponsors, so my reporting isn't influenced by donations by established space or drug companies. Instead, I rely entirely on donations and subscriptions from my readers, which gives me the freedom to write what I think, unencumbered by outside influences.
You can support me either by giving a one-time contribution or a regular subscription. There are four ways of doing so:
1. Zelle: This is the only internet method that charges no fees. All you have to do is use the Zelle link at your internet bank and give my name and email address (zimmerman at nasw dot org). What you donate is what I get.
2. Patreon: Go to my website there and pick one of five monthly subscription amounts, or by making a one-time donation.
3. A Paypal Donation or subscription:
4. Donate by check, payable to Robert Zimmerman and mailed to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage or shown in the menu above.
The uncertainty of science: A bug just discovered in the computer software used by MRIs to measure brain activity could invalidate 15 years of research and 40,000 science papers.
They tested the three most popular fMRI software packages for fMRI analysis – SPM, FSL, and AFNI – and while they shouldn’t have found much difference across the groups, the software resulted in false-positive rates of up to 70 percent. And that’s a problem, because as Kate Lunau at Motherboard points out, not only did the team expect to see an average false positive rate of just 5 percent, it also suggests that some results were so inaccurate, they could be indicating brain activity where there was none.
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]
The research the article described is focused entirely on the problems the software causes for past research. It makes no mention of the problems this software bug might cause for actual medical diagnosis Was the treatment of any patients effected by this bug? It does not say.
Readers!
Please consider supporting my work here at Behind the Black. Your support allows me the freedom and ability to analyze objectively the ongoing renaissance in space, as well as the cultural changes -- for good or ill -- that are happening across America. Fourteen years ago I wrote that SLS and Orion were a bad ideas, a waste of money, would be years behind schedule, and better replaced by commercial private enterprise. Only now does it appear that Washington might finally recognize this reality.
In 2020 when the world panicked over COVID I wrote that the panic was unnecessary, that the virus was apparently simply a variation of the flu, that masks were not simply pointless but if worn incorrectly were a health threat, that the lockdowns were a disaster and did nothing to stop the spread of COVID. Only in the past year have some of our so-called experts in the health field have begun to recognize these facts.
Your help allows me to do this kind of intelligent analysis. I take no advertising or sponsors, so my reporting isn't influenced by donations by established space or drug companies. Instead, I rely entirely on donations and subscriptions from my readers, which gives me the freedom to write what I think, unencumbered by outside influences.
You can support me either by giving a one-time contribution or a regular subscription. There are four ways of doing so:
1. Zelle: This is the only internet method that charges no fees. All you have to do is use the Zelle link at your internet bank and give my name and email address (zimmerman at nasw dot org). What you donate is what I get.
2. Patreon: Go to my website there and pick one of five monthly subscription amounts, or by making a one-time donation.
3. A Paypal Donation or subscription:
4. Donate by check, payable to Robert Zimmerman and mailed to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage or shown in the menu above.
Wow!… Faults positives up to 70%? I wonder if the brochure from the MRI salesman said that they guarantee up to 70% more patients?
Now we know why Canada has so few MRI machines. (I wonder if their software maker is the same company that runs our voting computers…)
Max–
in all seriousness however, the reason we have a lot of MRI machines in the USA, is/was primarily the free-market.
20 years ago it was perfectly legal for Doctors to buy/invest-in, their own MRI machines, load them on tractor-trailers, and run them almost 24/7. (often, in the parking lot of their office’s)
In the town where I lived at the time, two medical groups did exactly that & literally years before the local Hospital received “permission” to put one in.
Various “health-care reforms,” primarily in the last 8 years, made that illegal. (It’s a “conflict of interest,” apparently & we need to be “saved” from “exploiting doctors.”)
-Fortunately, we have a huge installed base of MRI machines (& CT scanners) already in place. I seriously doubt we would be in that position, if MRI machines were invented today, and controlled under the ACA.
As for the software errors; I know just enough Statistics to be dangerous, but that being said– if the error is consistent across time, software, & machines, are there not mathematical ways to extract that out, to a least some degree? and maybe not invalidate every single study??
Tangentially– personally, I’m highly suspect of the “active” real-time MRI scans they tout as being evidence for/against someone’s theory of brain-activity, but that’s a different topic above & beyond inherent software errors.
Interesting post. I agree with you on the last thing you mentioned in your comment above, regarding the “active real time MRI Scans”
“These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results,” the team writes in PNAS. The bad news here is that one of the bugs the team identified has been in the system for the past 15 years, which explains why so many papers could now be affected. [emphasis mine]