Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.

Layers and layers of peer-review: Two publishers of scientific journals have withdrawn 120 papers which they have discovered were nothing more than computer-generated gibberish.

Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and Electronic Engineers (IEEE), based in New York. Both publishers, which were privately informed by Labbé, say that they are now removing the papers. …

Labbé developed a way to automatically detect manuscripts composed by a piece of software called SCIgen, which randomly combines strings of words to produce fake computer-science papers. SCIgen was invented in 2005 by researchers at the Massachusetts Institute of Technology (MIT) in Cambridge to prove that conferences would accept meaningless papers — and, as they put it, “to maximize amusement” (see ‘Computer conference welcomes gobbledegook paper’). A related program generates random physics manuscript titles on the satirical website arXiv vs. snarXiv. SCIgen is free to download and use, and it is unclear how many people have done so, or for what purposes. SCIgen’s output has occasionally popped up at conferences, when researchers have submitted nonsense papers and then revealed the trick.

The real story here is that many of these gibberish papers were peer-reviewed by actual scientists who are supposedly experts in their fields and should have spotted the fakery immediately. That they didn’t suggests another level of corruption. Either they don’t really bother to peer review the papers they are asked to peer review, or they knew what was going on and were part of the game.

That this kind of stuff happens repeatedly in many fields of science should make us all very skeptical of any controversial scientific claim that carries with it any political component. This doesn’t mean that all published material is fake, only that we must not take anything on faith. Controversial results had better be bomb-proof before we accept them willingly.

FacebookTwitterGoogle+Share

3 comments

  • Patrick Ritchie

    Oy, what a mess.

    It seems more and more obvious that peer review as presently performed is not improving the quality of published papers.

    I suppose that wouldn’t be an issue if everyone agreed peer review adds no value, but the problem seems to stem from the supposed legitimacy garnered as a result of peer review. Authors often trumpet the fact that their papers have been ‘peer reviewed’ as though this in and of itself proves their conclusions.

    So if peer review isn’t adding value, what would?

    • “So if peer review isn’t adding value, what would?”

      Personal responsibility.

      • Patrick Ritchie

        Hah! Indeed it would.

        But there is much about the structure of the way science that gets done that incentives bad behavior. Funding structures that award money to those that already agree with the desired conclusion, journals whose peer review process is a joke and succesful careers based on publishing volume all seem to be problems.

        A good system is one in which even the bad actors end up doing the right thing, modern science sometimes feels like the opposite. Given how much we are actually learning in spite of all that I would say that scientists as a group have pretty good personal responsibility.

Leave a Reply

Your email address will not be published. Required fields are marked *