A journalist and filmmaking team fake a study and the press buys it
The uncertainty of science journalism: How a science journalist and two documentary filmmakers fooled millions (and many science journalists) into thinking that eating chocolate will help you lose weight.
My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded.
He then describes step by step how they did it. They even got their “research” published in a journal that claimed it did peer review but did not change one word of their submitted manuscript and published it less than two weeks after submission and the receipt of their money (science journals traditionally require authors to pay them for the honor of publication). In the end,
We landed big fish before we even knew they were biting. Bild rushed their story out—”Those who eat chocolate stay slim!”—without contacting me at all. Soon we were in the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German and Indian site of the Huffington Post, and even television news in Texas and an Australian morning talk show.
The American women’s magazine Shape also fell for the fraud.
To me, the main reason the fraud worked was because none of the journalists involved ever bothered to actually read the science paper. They saw the press release, thought the story was cool, and simply rewrote the release. This is what happens repeatedly in the science field, and is why so much crap about climate science gets published. Too many reporters accept verbatim the claims of the scientists, doing no research to check the facts on which those claims were based.
I should mention also that John Bohannon, the man who played the scientist in this prank, also ran a sting operation against peer review journals in October 2013, creating a bogus paper and getting 157 journals to accept it for publication.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
The uncertainty of science journalism: How a science journalist and two documentary filmmakers fooled millions (and many science journalists) into thinking that eating chocolate will help you lose weight.
My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded.
He then describes step by step how they did it. They even got their “research” published in a journal that claimed it did peer review but did not change one word of their submitted manuscript and published it less than two weeks after submission and the receipt of their money (science journals traditionally require authors to pay them for the honor of publication). In the end,
We landed big fish before we even knew they were biting. Bild rushed their story out—”Those who eat chocolate stay slim!”—without contacting me at all. Soon we were in the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German and Indian site of the Huffington Post, and even television news in Texas and an Australian morning talk show.
The American women’s magazine Shape also fell for the fraud.
To me, the main reason the fraud worked was because none of the journalists involved ever bothered to actually read the science paper. They saw the press release, thought the story was cool, and simply rewrote the release. This is what happens repeatedly in the science field, and is why so much crap about climate science gets published. Too many reporters accept verbatim the claims of the scientists, doing no research to check the facts on which those claims were based.
I should mention also that John Bohannon, the man who played the scientist in this prank, also ran a sting operation against peer review journals in October 2013, creating a bogus paper and getting 157 journals to accept it for publication.
On Christmas Eve 1968 three Americans became the first humans to visit another world. What they did to celebrate was unexpected and profound, and will be remembered throughout all human history. Genesis: the Story of Apollo 8, Robert Zimmerman's classic history of humanity's first journey to another world, tells that story, and it is now available as both an ebook and an audiobook, both with a foreword by Valerie Anders and a new introduction by Robert Zimmerman.
The ebook is available everywhere for $5.99 (before discount) at amazon, or direct from my ebook publisher, ebookit. If you buy it from ebookit you don't support the big tech companies and the author gets a bigger cut much sooner.
The audiobook is also available at all these vendors, and is also free with a 30-day trial membership to Audible.
"Not simply about one mission, [Genesis] is also the history of America's quest for the moon... Zimmerman has done a masterful job of tying disparate events together into a solid account of one of America's greatest human triumphs."--San Antonio Express-News
I’d say a large factor in their success in pulling this off was the human desire to get something for nothing. It also illustrates the perniciousness of bias confirmation.
Wow,
My wife recently took a class in public speaking where she had to write papers and then present them to the class. This was one of the subjects that she chose. She even brought in samples of dark chocolate as visual aids to share with the class.
She got an “A” in the class.
She still eats 42 grams of chocolate every day.
She stands to my left as she has every day since we said our vows and she isn’t going to like it that Rush Limbaugh got it right when he said all fad diet and exercise plans are a load of bunk and the only way to lose weight is to just not eat so much.
She isn’t going to like it when I tell her that her weight loss is due to her limiting her calorie intake.
She’s still losing weight.
She’s looking really good.
Maybe I won’t tell her.
This reminds me of a case of good science being poorly reported.
A couple of decades ago, the morning drive-time crew on a radio station that I used to listen to gushed over a new study that they thought said that lots and lots of sex is good for you. An hour into the show, they got one of the authors on the phone to ask him about it, and he said that the study showed that sex was good in one specific area of health, and should not be taken as meaning that we all should have lots of sex. The morning crew toned down their report on the subject.
The noon news crew and the afternoon drive-time crew obviously didn’t ever listen to the station’s morning show, because they were gushing about how that same study said that lots and lots of sex is good for you.
Good science badly reported.
It is clear to me that journalists don’t bother to read science papers. But would they know what they are reading if they did read them? Scientific papers tend to be written in scientist-ese, a boring language with a lot of terms and phrases that look like English but need translation.
Not understanding the language and practice of science is one of the reasons why everyone uses the word “theory” when they refer to a hypothesis, an uneducated guess, or a political position.
If you are a good science reporter, you learn the jargon and can understand the papers. And if there is something in the paper you don’t understand that appears important, you call the scientists up and ask them about it. I have been doing this routinely now for more than 20 years, since I switched from filmmaking to science journalism in 1994.
The most important part however is to read the scientific paper. Sadly, too many journalists don’t do this, even though it is a prerequisite for doing their job right.
Excellent point. I suspect that most organizations do not bother to have a specialist reporter in the sciences. I also suspect that being a good reporter and asking for clarification is too much trouble for many journalists with deadlines.
I stand corrected. I should have been more careful about writing in a way that generalized all journalists. I respect the journalists at some technical news sources, such as Aviation Week and Space News (among others), and I respect some interviewers, such as John Batchelor, for knowing enough of the topic to know what questions to ask.
To paraphrase a phrase: you comment and you learn.
I will put that morning drive-time crew in the category of “good reporters,” because they bothered to call the scientist and ask the questions. Unfortunately, the noon and afternoon news crews did not even bother to find out what should have been common knowledge at the radio station.