On reproducibility: Replicated errors and double standards from Anthony Watts at WUWT

The most recent article from Anthony Watts, who runs a climate conspiracy blog called wattsupwiththat, or WUWT, demonstrates once again that he doesn't understand the first thing about scientific research. It also provides another lesson in the telltale techniques of climate science denial, and the double standards of Anthony Watts. 

What the WUWT article (archived here) is about is an article in the New York Times (without a link), which in turn is an article about a paper in Science about the lack of reproducibility of many psychology papers. That is, Anthony published a "guest essay" about a New York Times article about a paper, topped and tailed with some irrelevant silly comments from Anthony himself. (Anthony doesn't write much himself these days. He's probably conscious that most of what he writes is too silly for public consumption so he copies and pastes press releases and denier blog articles from elsewhere, and relies on "guest essays" of dubious quality from his readers.)

If you skip to the end of the WUWT article, you'll notice an irony that suggests that Anthony is not only ignorant but lacking in critical thinking skills. Or maybe he regards his readers as lacking in intelligence. At the end of his copy and paste he added:

Related stories where Stephan Lewandowsky, Naomi Oreskes, and John Cook produce some questionable, and perhaps irreproducible psychological “science”:

Lewandowsky and Cook – back from the dead with another smear paper

A disturbance in the farce: Another hateful and pointless paper from Stephan Lewandowsky and Naomi Oreskes

And then there’s Rasmus Benstad’s recent laughable paper that five journals rejected, before he and his psyops crew found a journal that would publish what other the five journals rejected. Yeah, that’s what you might call “robusted” science.

The telltale technique Anthony used was Technique No. 2, the logical fallacy. He was claiming that a paper that argued that a large proportion of psychological studies were not reproducible, meant that two particular psych studies and one study showing that selected science papers weren't replicable, were "questionable, and perhaps irreproducible". That does not follow. It's a non sequitur, a logical fallacy.

Take Anthony's examples one by one.

Anthony Watts' first fail - a bout of recurrent fury

The first paper he refers to is Recurrent Fury, which you can read about here. Not only would that work most likely be reproducible, it was validated not once but arguably four times in the same paper! Anthony thinks it's a "smear" paper because it's about the evolution of conpiracy theories in the blogosphere. Something that you'll see daily at blogs like Anthony's WUWT. The paper reports how it:

• Used six criteria for conspiracy ideation from the literature, and added a seventh
• Brought in two independent researchers to validate the original research
• Validated the classifications through a blind study to recreate them (a form of replication) 
• Validated the original analysis through another blind study, comparing scholarly comments to conspiratorial comments.

If you read the paper, you'll also see how it cites other papers and builds on work that has been done before. A great example of how psychological studies can be validated and taken even further.

Anthony Watts' second fail - rejecting seepage of a "pause"?

The second paper that Anthony referred to was about seepage of denier memes into the scientific literature.  You can read about that paper here with links to more here. It would not be at all difficult to replicate that work. It validates itself in the scientific literature on climate. It makes you wonder if Anthony Watts is:

• Denying that there has been a "pause" in global surface temperature
• Accepting it was a slowdown, not a "hiatus"
• Rejecting the notion that deniers like him claimed that global warming has "stopped" or that it started cooling 16, 17, 18, 19 and 23 years ago!
• Decided his denial has absolutely no influence on climate science.

Anthony Watts' third fail is a spectacular replication of his mistakes

Anthony left his most spectacular fail to last. His evidence that many psychological studies can't be replicated is a paper demonstrating how specific climate science papers that disputed global warming couldn't be replicated! Let me put that another way. The paper that Anthony cites as "evidence" of non-replicability is a paper about non-replicability!

You can read about that paper at the Guardian in an article with the title:

Here’s what happens when you try to replicate climate contrarian papers

You can also read about it at realclimate.org - Let’s learn from mistakes, something that deniers are seemingly quite unable to do.

Reproducibility of science

The paper that was being talked about in today's WUWT article was about reproducibility of papers on psychology. (It builds on work that was flagged in another paper from a couple of years ago.) From the abstract:

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence. Even research of exemplary quality may have irreproducible empirical findings because of random or systematic error.

The study reported that it evaluated reproducibility using:

1. significance and P values, 
2. effect sizes, 
3. subjective assessments of replication teams, and 
4. meta-analysis of effect sizes. 

And the authors stated that although ninety-seven percent of original studies had statistically significant results, only thirty-six percent of replications had statistically significant results. The authors discussed scientific pressures for publication in terms of innovation vs replication. And how the pressure is on innovation, that simple replication isn't likely to result in publication. You'll need to read the paper to get a better feel for the work and what it signifies. 

I'll add that psychology is probably one of the most difficult sciences - unravelling the complexity of the human mind and human behaviour.

What Anthony claimed from the paper

At the top of his article, Anthony wrote:

Shades of Lewandowsky and Cook: When psychological science isn’t so sciency
Sound familiar? A study revealing a stunning lack of reproducibility in psychological science triggers another instance of reluctance to share data with any but friends, and an “adjustment” of  data to fit a theory.

First of all, the paper he wrote about didn't "trigger" any such thing. It was only published last week and the work that Anthony cited was published well before that. What it did trigger was another bout of conspiracy ideation at WUWT.

Nor does it signal a "reluctance to share data".  Could it possibly be that where Anthony Watts was complaining loud and long that a paper on conspiracy ideation did provide evidence to blog comments directly, now he's complaining that the replacement paper was anonymised? Surely not! (The data for Recurrent Fury is available to responsible researchers on application.)

Footnote about the WUWT author: one of Anthony's own "anonymous cowards"

Anthony's article was by "guest blogger" Frank Lee MeiDere. It didn't take much to discover that he is a WUWT reader and a blogger who says of himself:

I trust you'll realize that's not my real name, but I'm an English prof at a Toronto college and I figure a low profile is in order.

That same blogger on his own blog says: "while it’s possible that I may not like you, it’s far more probable that either as myself or as Frank Lee MeiDere, I just don’t give a damn."

So it turns out that "Frank Lee MeiDere" is one of Anthony Watts' pet anonymous cowards. Anthony denounces and despises pseudonyms, but only when they are used by pro-science people. I've no problem with pseudonyms, needless to say. What this episode does demonstrate once more are the double standards of Anthony Watts, and his inconsistency.

From the WUWT comments

Now since the blog owner, Anthony Watts, hasn't learnt anything at all from his mistakes, what are the chances that he attracts any readers who are capable of critical thinking or learning? Nil? Next to nil? You can see for yourself in the comments:

Pat Ch has a hypothesis which is not based on any research, let alone reproducible research. One might call it conspiracy ideation:

August 30, 2015 at 5:11 pm
Green energy, climate, sociology, anthropology, psychology. What do they all have in common? They are controlled by third rate scientists who are really left wing ideologues.

Chip Javert claims some expertise in the scientific method and declares all political, psychological and climate science null and void. Again, his hypothesis is not based on any research let alone reproducible research. Again, it is more properly compared to conspiracy ideation:

August 30, 2015 at 4:18 pm
Cogsys
Nice rant, but most of it simply is eye-wash.
Political science, psychology and “global warming” do not use the scientific method, and, thus, are not sciences (they are “fields of observation”).
The scientific method is not forgotten (popular internet material, written by non-scientifically trained journalists IS NOT SCIENCE). I would refer you to the recent search for the Higgs Boson – that definitely was not decided by a “consensus” vote; nor was the technology in your smart phone, PC, GPS…
If you choose to further argue this point, knowing your educational background would be interesting (mine: Ga Tech physics).

Allencic might be referring to the paper that Anthony cited, but I think not. It's more likely that he's got a (conspiracy) theory about mainstream science:

August 30, 2015 at 4:02 pm
Anyone who thinks replication of results doesn’t matter (such as most so-called “climate science”) needs to go back and re-read the story of “Cold Fusion.” Good intentions, crappy and false results. 

wickedwenchfan is either a long-standing Poe or the resident extremist at WUWT, so as to make the regular nutters appear not quite so extreme: 

August 30, 2015 at 3:34 pm
The world of physics is sadly no better. If it were you wouldn’t be hearing about absurd magic like Dark Matter, Dark Energy, Black Holes, Snowball Comets or Back Radiation!

MarkW doesn't "believe in" physics, engineering, biology, chemistry, economics, finance, ecology or any other field that uses computer models to explore and extend knowledge. And he doesn't "believe" that climate scientists go out in the field and collect and analyze actual data! I'm surprised that he owns a computer:

August 30, 2015 at 7:03 pm
Climate science is not one of the hard sciences. Never has been. Hard sciences go out into the field and collect and analyze actual data. They don’t use statistical “tricks” to compensate for missing data, or data that has been polluted by confounding affects.
And they certainly don’t use computer models for “proof” of anything. 

You'd think this would have been predicted (it was - pers comm). First deniers complain that evidence was made public in the supplement to Recursive Fury, and threatened the journal till it withdrew the paper. Now they complain that it's only made available to approved researchers on request - in Recurrent Fury, the anonymised version! Read about it at Shaping Tomorrow's World. Louis Hunt wrote:

August 30, 2015 at 2:11 pm
Lewandowsky’s original findings were replicated by whom? He refused to provide his data to researchers, so how could anyone replicate his findings unless he did it himself?

I'll finish with a comment from Anthony's "anonymous coward", Frank Lee MeiDere. He could be writing about any and almost everyone who posts their various random "thoughts' at WUWT - bald statements unsupported by evidence, with not even a reference:

August 30, 2015 at 1:27 pm
I taught at a city college for almost 10 years, and virtually every student started my class believing that a couple of references from second or third-level sources constituted proof. More disturbing, however, was their attitude that the references didn’t really matter; what counted was what they (the students) believed was right. 

References and further reading

Open Science Collaboration (Brian A. Nosek). "Estimating the reproducibility of psychological science." Science 28 August 2015: Vol. 349 no. 6251 DOI: 10.1126/science.aac4716 (subs req'd)

Open Science Collaboration (Brian A. Nosek). "An open, large-scale, collaborative effort to estimate the reproducibility of psychological science." Perspectives on Psychological Science 7, no. 6 (2012): 657-660. doi: 10.1177/1745691612462588 (open access)

Many Psychology Findings Not as Strong as Claimed, Study Says - article by Benedict Carey in The New York Times

Benestad, Rasmus E., Dana Nuccitelli, Stephan Lewandowsky, Katharine Hayhoe, Hans Olav Hygen, Rob van Dorland, and John Cook. "Learning from mistakes in climate research." Theoretical and Applied Climatology (2015): 1-5. doi:10.​1007/​s00704-015-1597-5 (open access)

• Let’s learn from mistakes - by Rasmus Benestad at realclimate.org
• Here’s what happens when you try to replicate climate contrarian papers - by Dana Nuccitelli at the Guardian

Lewandowsky, Stephan, John Cook, Klaus Oberauer, Scott Brophy, Elisabeth A. Lloyd, and Michael Marriott. (2015). "Recurrent fury: Conspiratorial discourse in the blogosphere triggered by research on the role of conspiracist ideation in climate denial." Journal of Social and Political Psychology, 3 (1). doi: 10.5964/jspp.v3i1.443. (open access)

Restoring Recurrent Fury - by Stefan Lewandowsky at Shaping Tomorrow's World

Stephan Lewandowsky, Naomi Oreskes, James S. Risbey, Ben R. Newell and Michael Smithson. "Seepage: Climate change denial and its effect on the scientific community." Global Environmental Change, 2015 doi:10.1016/j.gloenvcha.2015.02.013 (open access)

From the HotWhopper archives

• Curses! It's a conspiracy! The Fury is Back Thrice Over - HotWhopper, July 2015
• On Seeps and SCAMS Part I: Lessons for Climate Scientists - May 2015
• Galileo, fake authorities and adhominous cowards - July 2015