Monday, October 07, 2013


"The quality of the science is abominable"

There are certain topics I have lined up ready to write about but don't for various reasons.  Sometimes its a lack of a hook - something to tie the topic to in order to make it current and meaningful to a reader.  Sometimes I just don't feel inspired.  And sometimes there's a topic I don't have the evidence or data on to support what I am sure is true but can't prove.
But as I've said before, if you wait long enough, the truth will float past you on the internet; you just have to be careful picking through the weeds.  For a while now I've been confident that peer review and scientific publication has been at least in part a highest bidder operation.  That you can get pretty much anything published, no matter how absurd or false, if you have the cash or an impressive sounding resume.
We've already seen hints of that with Center for Science in the Public Interest reports on how popcorn is killing us all and mother's milk is bad for babies.  And we've seen how an obscure lab by dubious scientists can spread panic and disease with the vaccination/autism scare.  But now I have more solid evidence.
Science Magazine did a sort of undercover sting operation to see just how far you could get an idea pushed if you had the cash.  So they came up with a plausible, impressive-sounding report with enough flaws in it to be unacceptable scientifically, and shopped it to various publications.  Richard Knox writes at NPR:
The final touch was to make the paper read as though it had been written by someone whose first language is not English. To do that, Bohannan used Google Translate to put it into French, then translated that version back into English.

In the end, the paper's fictitious authors got 157 acceptance letters and 98 rejections – a score of 61 percent. "That's way higher than I expected," Bohannan says. "I was expecting 10 or 15 percent, or worst case, a quarter accepted."

For the privilege of being published, the paper's authors were asked to send along a publishing fee of up to $3,100.
So that's too bad, publications are putting sloppy or even fake science up there.  Except its worse than it seems at first blush.  Being published in scientific journals gives work credibility, and even more problematic, its part of the process of "peer review."  Dan Vergano at National Geographic explains:
But in many cases, it appears the study wasn't peer-reviewed at all by the journals that responded to the spoof submission. Many of the reviews were just requests to format the study for publication. And of 106 journals that performed any review, 70 percent accepted the study.

"If a bogus paper is able to get through peer review, think about how many legitimate, but deeply flawed, papers must also get through," says Michael Eisen of the University of California, Berkeley, a founder of the Public Library of Science family of open-access journals.

One of those journals, PLOS One, was the only one of the 255 journals that received the spoof that noted its ethical flaws and "meticulously" reviewed the bogus study before rejecting it.
Peer Review is supposed to be the blue ribbon test for scientific work by having the scientific report examined by others in the field for flaws and proper method.  These papers were allegedly reviewed and published after this examination, which lends them credibility and prestige.
Science magazine its self has its problems.  A while back they published a study supposedly showing bacteria that "evolved" to survive and live in arsenic.  The National Geographic article above notes that this study was bogus, but it passed their peer review process and was given publication.   They set out to prove the online "open journals" were of poor quality, and ended up showing they make the same mistakes as Science magazine.
And here's the problem for you and I: we're not scientists, and we almost never get the real report to read.  What we get is a reporter's attempt to explain and write about the report with their high school science education.  Sure, he got a C in intro to chemistry, but that doesn't mean he's ready to explain a new lab report, or even know how to tell if its valid.
But the more exciting, new, and especially bias-confirming a report is, the more likely they are to report on it and give it a pass.  See, it was peer-reviewed, it was in these scientific journals, it must be true!  So we're flooded with trash science given credibility by the mere appearance in a newspaper.
And the problem is, people take science with the same level of seriousness and unquestioned trust that they did religious topics in the past.  Oat bran will save you from cancer!  Oat bran will kill you! That's what the paper said and scientists are better than ordinary people, they don't cheat, lie, or have bias, they are above political considerations!  When you strap on that lab coat, you are lifted up to transcend human pettiness.
So its easy to manipulate people, especially for a cause or idea that appeals to journalists.  This proves evolution!  This disproves evolution!  This shows Republicans are idiots/insane/monsters!  This demonstrates we're destroying the planet!  And if you can get a celebrity on your side to push the topic, well that's just icing on the cake.  Sure, Jenny McCarthy is a clueless actress and soft core porn model, but she's famous and pretty, she must be right!
And fighting this is a tough, uphill battle.  That's why I keep bashing science over and over in the Common Knowledge series.  Not because I dislike science, but because bad science is so incredibly destructive and difficult to counter.  People hear what they like and don't want to feel stupid because they were duped.  It becomes a matter of personal injury and ego to point out the error or mistake, because they become personally invested in the story. So they are either defensive and refuse to listen or believe it or begin to attack all science and be doubtful about all scientific statements.
Every time you see a scientific statement - especially an absolute statement like "always" or without qualifiers - its time to be skeptical.  It doesn't matter if it confirms what you figured to be true, or want to be.  Wait and see, and question what they say.  Because eventually, that truth will float by.  
Maybe it is true.  And maybe it isn't.  But unless we show more restraint and critical thought, we're just that much easier to manipulate.


Anonymous said...

Add to that the fact that governments use this flawed science to justify new laws and regulations and it becomes truly dangerous.

mushroom said...

You probably recall that one of Obama's promises back in 2008 was to restore science to its rightful place. You know, because the previous administration had governed by examining sheep entrails or something.

I'm all for the separation of science and state.