More than half of published psychology papers cannot be replicated

Please consider donating to Behind the Black, by giving either a one-time contribution or a regular subscription, as outlined in the tip jar to the right. Your support will allow me to continue covering science and culture as I have for the past twenty years, independent and free from any outside influence.

The uncertainty of science: An attempt to replicate 98 different psychological research studies has found that significantly less than half could be replicated.

In the biggest project of its kind, Brian Nosek, a social psychologist and head of the Center for Open Science in Charlottesville, Virginia, and 269 co-authors repeated work reported in 98 original papers from three psychology journals, to see if they independently came up with the same results. The studies they took on ranged from whether expressing insecurities perpetuates them to differences in how children and adults respond to fear stimuli, to effective ways to teach arithmetic.

According to the replicators’ qualitative assessments, as previously reported by Nature, only 39 of the 100 replication attempts were successful. … There is no way of knowing whether any individual paper is true or false from this work, says Nosek. Either the original or the replication work could be flawed, or crucial differences between the two might be unappreciated. Overall, however, the project points to widespread publication of work that does not stand up to scrutiny. [emphasis mine]

None of this surprises me. The focus of much science research, especially in the soft sciences like psychology, is statistical in nature and easily manipulated. In fact, most of it isn’t science at all, but an attempt to use mere statistics to prove a point. Science would instead try to find out why something happens, not just demonstrate through statistics that it does.


  • Carl

    The problem isn’t that any of the original works were published, the problem is that too few are replicated soon after publication. When researchers attempt to replicate results of studies and papers published by others, science proceeds in a self-correcting manner. Peer review is not designed to and cannot provide this correction. Only follow on work can provide this correction.

    Unfortunately, journals and research faculty too often view the admittedly less original work of replicating previous work as less worthy of attention. When papers replicating or contradicting results are as likely to be published as the original paper and when the faculty making tenure decisions respect it as much, science will be as self-correcting as it should be.

  • Phill o

    Even in chemistry, this phenomenon appears. The publish or perish syndrome. An overly aggressive professor can often get a paper published through shear intimidation. Having reviewed many papers myself, dealing with pesticide residues, I am familiar with the types of intimidation.

    There are a number of magazines where there is no peer review, notably Science, and Nature, and Scientific American. There are others but the ones listed tend to be believed without question. Even peer reviewed journals have the problem of collective review where buddies are chosen for reviewing. Conferences are the place for the formation of buddies. Consider the astronomical society and the demotion of Pluto. Through shear politics, scientist’s focus has been shifted from science. The study of Plutanoid planets might be an area for novel research, but people are distracted by the demotion issue.

    My question now is how does one educate (or reeducate) people who have been brainwashed into thinking that the university group are truthful. (Hey wait a minute, I was one of those people)

    Seriously think of the ozone scam. Technology was developed to detect the natural variations of ozone in the upper atmosphere. Note that the poles must have a lower ozone concentration due to the lack of ultraviolet light for about 6 months. UV light is needed for the production of ozone in the upper atmosphere. Ozone produced from lightning and welding etc, close to the earth’s surface can not reach the upper atmosphere and is considered a priority pollutant.

    I could go on about other environmental chemistry scams, but let me say that the universities do not deserve unsuspecting respect.

    Remember that some of the most transforming discoveries have been made by folk who thought outside the box, often being squelched by peer review. (Gallaleio, and also the MO theory versus valance bond theory for chemical bonding)

  • hondo

    I Remember the ozone issue. I was doing research for a grad paper back in the early ’90s, when I came across a very old science journal from the mid ’50s. It was an Air Force/science study of the Ozone layer above the Antarctic done at Mc Murdo. An interesting study of a formerly unknown natural phenomena.

    Science, like anything else, can be easily corrupted to suit one’s purposes. I believe we are simply moving thru a period of Lysenko Science.

Leave a Reply

Your email address will not be published. Required fields are marked *