Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Readers!
For many reasons, mostly political but partly ethical, I do not use Google, Facebook, Twitter. They practice corrupt business policies, while targeting conservative websites for censoring, facts repeatedly confirmed by news stories and by my sense that Facebook has taken action to prevent my readers from recommending Behind the Black to their friends.
Thus, I must have your direct support to keep this webpage alive. Not only does the money pay the bills, it gives me the freedom to speak honestly about science and culture, instead of being forced to write it as others demand.
Please consider donating by giving either a one-time contribution or a regular subscription, as outlined in the tip jar below.
Regular readers can support Behind The Black with a contribution via paypal:
If Paypal doesn't work for you, you can support Behind The Black directly by sending your donation by check, payable to Robert Zimmerman, to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage. And if you buy the books through the ebookit links, I get a larger cut and I get it sooner.
Layers and layers of peer-review: Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and Electronic Engineers (IEEE), based in New York. Both publishers, which were privately informed by Labbé, say that they are now removing the papers. …
Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” (see ‘Computer conference welcomes gobbledegook paper’). A related program generates random physics manuscript titles on the satirical website arXiv vs. snarXiv. SCIgen is free to download and use, and it is unclear how many people have done so, or for what purposes. SCIgen’s output has occasionally popped up at conferences, when researchers have submitted nonsense papers and then revealed the trick.
The real story here is that many of these gibberish papers were peer-reviewed by actual scientists who are supposedly experts in their fields and should have spotted the fakery immediately. That they didn’t suggests another level of corruption. Either they don’t really bother to peer review the papers they are asked to peer review, or they knew what was going on and were part of the game.
That this kind of stuff happens repeatedly in many fields of science should make us all very skeptical of any controversial scientific claim that carries with it any political component. This doesn’t mean that all published material is fake, only that we must not take anything on faith. Controversial results had better be bomb-proof before we accept them willingly.
For many reasons, mostly political but partly ethical, I do not use Google, Facebook, Twitter. They practice corrupt business policies, while targeting conservative websites for censoring, facts repeatedly confirmed by news stories and by my sense that Facebook has taken action to prevent my readers from recommending Behind the Black to their friends.
Thus, I must have your direct support to keep this webpage alive. Not only does the money pay the bills, it gives me the freedom to speak honestly about science and culture, instead of being forced to write it as others demand.
Please consider donating by giving either a one-time contribution or a regular subscription, as outlined in the tip jar below.
Regular readers can support Behind The Black with a contribution via paypal:
If Paypal doesn't work for you, you can support Behind The Black directly by sending your donation by check, payable to Robert Zimmerman, to
Behind The Black
c/o Robert Zimmerman
P.O.Box 1262
Cortaro, AZ 85652
You can also support me by buying one of my books, as noted in the boxes interspersed throughout the webpage. And if you buy the books through the ebookit links, I get a larger cut and I get it sooner.
Layers and layers of peer-review: Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and Electronic Engineers (IEEE), based in New York. Both publishers, which were privately informed by Labbé, say that they are now removing the papers. …
Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” (see ‘Computer conference welcomes gobbledegook paper’). A related program generates random physics manuscript titles on the satirical website arXiv vs. snarXiv. SCIgen is free to download and use, and it is unclear how many people have done so, or for what purposes. SCIgen’s output has occasionally popped up at conferences, when researchers have submitted nonsense papers and then revealed the trick.
The real story here is that many of these gibberish papers were peer-reviewed by actual scientists who are supposedly experts in their fields and should have spotted the fakery immediately. That they didn’t suggests another level of corruption. Either they don’t really bother to peer review the papers they are asked to peer review, or they knew what was going on and were part of the game.
That this kind of stuff happens repeatedly in many fields of science should make us all very skeptical of any controversial scientific claim that carries with it any political component. This doesn’t mean that all published material is fake, only that we must not take anything on faith. Controversial results had better be bomb-proof before we accept them willingly.
Oy, what a mess.
It seems more and more obvious that peer review as presently performed is not improving the quality of published papers.
I suppose that wouldn’t be an issue if everyone agreed peer review adds no value, but the problem seems to stem from the supposed legitimacy garnered as a result of peer review. Authors often trumpet the fact that their papers have been ‘peer reviewed’ as though this in and of itself proves their conclusions.
So if peer review isn’t adding value, what would?
“So if peer review isn’t adding value, what would?”
Personal responsibility.
Hah! Indeed it would.
But there is much about the structure of the way science that gets done that incentives bad behavior. Funding structures that award money to those that already agree with the desired conclusion, journals whose peer review process is a joke and succesful careers based on publishing volume all seem to be problems.
A good system is one in which even the bad actors end up doing the right thing, modern science sometimes feels like the opposite. Given how much we are actually learning in spite of all that I would say that scientists as a group have pretty good personal responsibility.