Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Layers and layers of peer-review: Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and Electronic Engineers (IEEE), based in New York. Both publishers, which were privately informed by Labbé, say that they are now removing the papers. …
Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” (see ‘Computer conference welcomes gobbledegook paper’). A related program generates random physics manuscript titles on the satirical website arXiv vs. snarXiv. SCIgen is free to download and use, and it is unclear how many people have done so, or for what purposes. SCIgen’s output has occasionally popped up at conferences, when researchers have submitted nonsense papers and then revealed the trick.
The real story here is that many of these gibberish papers were peer-reviewed by actual scientists who are supposedly experts in their fields and should have spotted the fakery immediately. That they didn’t suggests another level of corruption. Either they don’t really bother to peer review the papers they are asked to peer review, or they knew what was going on and were part of the game.
That this kind of stuff happens repeatedly in many fields of science should make us all very skeptical of any controversial scientific claim that carries with it any political component. This doesn’t mean that all published material is fake, only that we must not take anything on faith. Controversial results had better be bomb-proof before we accept them willingly.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The print edition can be purchased at Amazon. from any other book seller, or direct from my ebook publisher, ebookit.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
Layers and layers of peer-review: Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.
Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and Electronic Engineers (IEEE), based in New York. Both publishers, which were privately informed by Labbé, say that they are now removing the papers. …
Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” (see ‘Computer conference welcomes gobbledegook paper’). A related program generates random physics manuscript titles on the satirical website arXiv vs. snarXiv. SCIgen is free to download and use, and it is unclear how many people have done so, or for what purposes. SCIgen’s output has occasionally popped up at conferences, when researchers have submitted nonsense papers and then revealed the trick.
The real story here is that many of these gibberish papers were peer-reviewed by actual scientists who are supposedly experts in their fields and should have spotted the fakery immediately. That they didn’t suggests another level of corruption. Either they don’t really bother to peer review the papers they are asked to peer review, or they knew what was going on and were part of the game.
That this kind of stuff happens repeatedly in many fields of science should make us all very skeptical of any controversial scientific claim that carries with it any political component. This doesn’t mean that all published material is fake, only that we must not take anything on faith. Controversial results had better be bomb-proof before we accept them willingly.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The print edition can be purchased at Amazon. from any other book seller, or direct from my ebook publisher, ebookit. The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
Oy, what a mess.
It seems more and more obvious that peer review as presently performed is not improving the quality of published papers.
I suppose that wouldn’t be an issue if everyone agreed peer review adds no value, but the problem seems to stem from the supposed legitimacy garnered as a result of peer review. Authors often trumpet the fact that their papers have been ‘peer reviewed’ as though this in and of itself proves their conclusions.
So if peer review isn’t adding value, what would?
“So if peer review isn’t adding value, what would?”
Personal responsibility.
Hah! Indeed it would.
But there is much about the structure of the way science that gets done that incentives bad behavior. Funding structures that award money to those that already agree with the desired conclusion, journals whose peer review process is a joke and succesful careers based on publishing volume all seem to be problems.
A good system is one in which even the bad actors end up doing the right thing, modern science sometimes feels like the opposite. Given how much we are actually learning in spite of all that I would say that scientists as a group have pretty good personal responsibility.