David Rose doubles down on #climate disinformation about NOAA. Let's get some perspective

I've had a post in train for a week now, after last week's ridiculously wrong article from David Rose in the Mail on Sunday. He based it on another ridiculous article, that one from John Bates on Judith Curry's climate disinformation blog. David Rose has followed it up with another ridiculously wrong article, which says nothing new and in which he's just trying to justify all the mistakes and lies in his first article.

John Bates is a meteorologist turned computer data person whose nose was out of joint because he didn't always get his way when he used to work at NOAA. It was a sour grapes whine from someone who wanted attention. That's all. However his attention-seeking moan had major repercussions through the deniosphere.

In his latest article, David Rose finishes with this:

We cannot allow such a vital issue for our future to be mired in half truths and deceptions.

Which raises the question - why is David Rose himself so mired in half truths and deceptions? His article doesn't just contain half truths, he's a bald-faced liar!

John Bates slagged off scientists, with no evidence, just to get some attention

John Bates slagged off scientists at Judith Curry's blog first, then to anti-science David Rose at the tabloid rag, Mail on Sunday. He walked back some of his worst defamatory comments in an interview at E&E News, reportedly specifying "he did not believe that they manipulated the data upon which the research relied in any way"

This was after making ludicrous claims at Judith Curry's blog, where he wrote, with no evidence whatsoever, and all the evidence showing otherwise:

So, in every aspect of the preparation and release of the datasets leading into K15, we find Tom Karl’s thumb on the scale pushing for, and often insisting on, decisions that maximize warming and minimize documentation. 

Then he went even further with David Rose, who quoted him lying that NOAA scientists threw out data. Of course they did nothing of the sort. John Bates also showed that he's not as clever as he probably likes to think he is. In fact he's as dumb as Pat Michaels (which is dumb, dumb.)

If you're wondering what John Bates wanted, a clue is in his paper, which was published four months after Karl15. It turns out that what he wanted would have involved something like a seven year wait from go to whoa for getting out climate research. I kid you not - see below.

There is almost no difference to speak of between NOAA and other data sets

In place of my original article (which probably won't ever get published now), here are some charts and tables that should make you wonder what the heck disinformers are going on about.

This first chart is a plot of five data sets. It includes: UK's HadCRUT4, NASA's GISTemp, Berkeley Earth, NOAA from Karl15, and NOAA "old" (prior to Karl15). The data goes from 1975 to 2014, which was where Karl15 reported to. I've chosen that period because the mid-1970s was the last time there was a change in the temperature trend, and I wanted to include meaningful trend lines.

Figure 1 | Global mean surface temperature, five data sets: UK's HadCRUT4, NASA's GISTemp, NOAA from Karl15 and NOAA pre-Karl15 . All  plots have been aligned to a 1981-2010 baseline. Data sources: HadCRUT4 | GISS NASA | Berkeley Earth | NOAA old and Karl15

If you have no confidence in your ability to read a chart, the point of this one is to show that there is very little difference between the different data sets, even though there are differences between the sources of data and the way they are processed. (See how close together all the lines are.)

If you thought the difference between the five sets of data was tricky to see, it's even harder to see when the lines are thickened - though probably not so much that they encompass 95% likelihood. You can see that the "old" NOAA data (green line) was a bit cooler in the most recent years, but that's probably the main difference.

Figure 2 | Global mean surface temperature, five data sets: UK's HadCRUT4, NASA's GISTemp, NOAA from Karl15 and NOAA pre-Karl15 . All  plots have been aligned to a 1981-2010 baseline. Data sources: HadCRUT4 | GISS NASA | Berkeley Earth | NOAA old and Karl15

So close they are almost identical

The linear rates of warming over that 1975 to 2014 period, from fastest to slowest are shown in the legend at the bottom of the charts. (Click the arrow on the bottom right to see all the trend numbers. Move the cursor over the chart to see the trends/temperatures at different times):

• GISTemp and HadCRUT4 and BEST and NOAA's Karl15 all have a trend of 1.7 C/century
• NOAA old version had a trend of 1.6 C/century.

See the odd one out? Yes, it's the old version from NOAA. In other words, the most recent updates just bring NOAA more into line with other data sets. Contrary to what disinformers try to make out, it's based on rigorous research from NOAA scientists - it's not a conspiracy, nor is there any "fudging".

Similarities are be expected in the case of GISTemp and Karl15, because they both use the Global Historical Climate Network (GHCN) for land and ERSST v4 for the ocean. I don't know if NASA and NOAA have switched over to the larger data set for land that was used in Karl15. In any case, they aren't identical. NASA also uses SCAR for Antarctica, for example. HadCRUT4 uses some GHCN but also gets a lot of data from individual countries and the World Meteorological Organisation (WMO). It does its own thing for sea surface temperature, though much of the raw data would be the same. The sources of Berkeley Earth data are described here.

Waiting seven whole years for climate research

John Bates made a complete fool of himself at Curry's place and to David Rose. He tried to walk it back when he spoke with the reporter from E&E News, but still came over as a whiny wuss, not a responsible level-headed scientist. The crux of his complaint is that the data from Karl15 was in the research stage and hadn't been through Bates' full and complete archiving process. But to do that takes a very long time. It can take five years just to get to what he calls the IOC phase, and a full seven years to get to what he calls "full operations". Bear in mind that these data have been up on the NOAA's ftp server since June 2015, the month Karl15 was published (h/t Nick Stokes). This is from his paper:

As of October 2015, the CDRP [Climate Data Record program] has transitioned 30 CDRs [Climate Data Records] from research to initial operations and 1 to full operations. The average time from initial grant award to a principal investigator to the IOC phase was approximately 60 months and about 84 months to full operations. The transition from initial to full operations has been difficult and variable, as detailed below.

Yep. Not just a seven year wait, but "difficult and variable". In fact, as John Bates confessed at Judith Curry's blog (see, I can use loaded words, just like David Rose):

I spent the last decade cajoling climate scientists to archive their data and fully document the datasets. 

His incredibly complex archiving system may have been suitable for some purposes, but it clearly was a thorn in the side of users. The diagrams in his paper show it as a very complex, long process involving umpteen steps and a multitude of different work groups at NOAA. I imagine the procedures manual could run to hundreds of pages. To what extent did he even involve or listen to users? Good data archiving procedures are important, particularly for climate data. I doubt anyone would dispute that. But what's the point of a system if it doesn't meet user needs? And why try to stop research being published when it's based on solid and well-tested data, just because it hasn't been through the full seven year archiving process?

Even David Rose admits that the formal process takes a very long time, though he peppered it with insinuations using the same word I just did (my emphasis):

Last night Mr Karl admitted the data had not been archived when the paper was published. Asked why he had not waited, he said: ‘John Bates is talking about a formal process that takes a long time.’ He denied he was rushing to get the paper out in time for Paris, saying: ‘There was no discussion about Paris.’

Look past David's trick word and read what Tom Karl actually said:

• John Bates is talking about a formal process that takes a long time.
• There was no discussion about Paris.

That's right. Despite all the lies and innuendo from David Rose, the paper had nothing to do with the UN talks in Paris, and the formal process was inconsequential to the paper itself.

David Rose is up to his old defamatory disinformation tricks

Now David Rose is no scientist or mathematician, that's obvious. He also knows that his fans are scientifically illiterate and works hard to keep them that way. David uses word tricks to gee up his denier fans into an apoplexy. Words like "claim" and "influence" and "manipulate" are used to manipulate his readers. The list below contains some of the words he used, or variations of them, in his first article, which was wrong, filled to the brim with disinformation and ridiculously wrong charts, and arguably defamatory:

Figure 3 | Manipulation words from David Rose at the Mail on Sunday, with frequency . Data source: David Rose's first article

More data to show up the lies of the climate disinformers

For the sake of completeness, here is a chart showing the five sets of data, from 1880 through to 2016. The "old" and "new" NOAA data sets only go to 2014, because they relate to Karl15.

Figure 4 | Global mean surface temperature, five data sets: UK's HadCRUT4, NASA's GISTemp, NOAA from Karl15 and NOAA pre-Karl15 . All  plots have been aligned to a 1981-2010 baseline. Data sources: HadCRUT4 | GISS NASA | Berkeley Earth | NOAA old and Karl15

As you'd expect, the earlier years are where the differences are greatest. That's largely because there are fewer observations the further you go back in time and the error margins (not shown) are larger.

Lots more that could be said

There's lots more that could be said about this disgusting behaviour from David Rose and John Bates. I haven't bothered with the stupid articles at WUWT, or Judith Curry, for example. Some people have said it already, and some have just pointed to the facts (see the references below).

I contacted people who would know him, and they behaved very professionally. They didn't want to impugn the character or guess at motives of John Bates. Pity that John Bates didn't show them the same respect.

I've taken the line that this behaviour cannot be tolerated and strong words are needed. The main message I've figured out from people who know him, and elsewhere, is that John Bates accepts climate science and is very clever. However he is not what one would call a "people person" and does not have a calm unflappable disposition. He has been known to "lose it" quite magnificently. Is that too harsh? I doubt it. Here is how he behaved just recently (read to the end).

For my part, I wonder what John Bates was thinking. Here he is, just embarking on a new career and presumably trying to win friends and influence people with his brilliance, and he goes and blows it all on a dumb dummy spit, with woefully wrong allegations. All he's done is let potential clients and professional colleagues know to be very wary of his behaviour. Not someone one with whom one would try to build a trusting working relationship.

David Rose calls Scott K. Johnson of Ars Technica a "snob"

A bonus - how David Rose lost his cool (did he ever have one?) and tweeted this to the excellent science writer at Ars Technica, Scott K. Johnson, who pointed out that the Mail is nothing more than a tabloid gossip rag:

@SJvatn Oo Scott, you are lucky to work for a right posh paper. You must be so brainy! But does anyone read it? But plebs are tiresome. SNOB

— David Rose (@DavidRoseUK) February 12, 2017

References and further reading

Karl, Thomas R., Anthony Arguez, Boyin Huang, Jay H. Lawrimore, James R. McMahon, Matthew J. Menne, Thomas C. Peterson, Russell S. Vose, and Huai-Min Zhang. "Possible artifacts of data biases in the recent global surface warming hiatus." Science 348, no. 6242 (2015): 1469-1472. DOI: 10.1126/science.aaa5632 (pdf here)

Huang, Boyin, Viva F. Banzon, Eric Freeman, Jay Lawrimore, Wei Liu, Thomas C. Peterson, Thomas M. Smith, Peter W. Thorne, Scott D. Woodruff, and Huai-Min Zhang. "Extended Reconstructed Sea Surface Temperature version 4 (ERSST. v4), Part I. Upgrades and Intercomparisons." Journal of Climate 2014 (2014). doi: http://dx.doi.org/10.1175/JCLI-D-14-00006.1 (pdf here)

Liu, Wei, Boyin Huang, Peter W. Thorne, Viva F. Banzon, Huai-Min Zhang, Eric Freeman, Jay Lawrimore, Thomas C. Peterson, Thomas M. Smith, and Scott D. Woodruff. "Extended reconstructed sea surface temperature version 4 (ERSST. v4): part II. Parametric and structural uncertainty estimations." Journal of Climate 28, no. 3 (2015): 931-951. doi: http://dx.doi.org/10.1175/JCLI-D-14-00007.1 (pdf here)

Peter W. Thorne, Kate M. Willett, Rob J. Allan, Stephan Bojinski, John R. Christy, Nigel Fox, Simon Gilbert, Ian Jolliffe, John J. Kennedy, Elizabeth Kent, Albert Klein Tank, Jay Lawrimore, David E. Parker, Nick Rayner, Adrian Simmons, Lianchun Song, Peter A. Stott, and Blair Trewin, 2011: "Guiding the Creation of A Comprehensive Surface Temperature Resource for Twenty-First-Century Climate Science." Bull. Amer. Meteor. Soc., 92, ES40–ES47. doi: http://dx.doi.org/10.1175/2011BAMS3124.1 (open access)

NEW: How an Interoffice Spat Erupted Into a Climate-Change Furor - A few weeks ago, on an obscure climate-change blog, a retired government scientist named John Bates blasted his former boss on an esoteric point having to do with archiving temperature data. - Article by Hiroko Tibuchi at the New York Times, 20 February 2017
Guest post: Why NOAA updates its sea surface temperature record - article by Peter Thorne at Carbon Brief, 10 February 2017

'Whistleblower' says protocol was breached but no data fraud - Article by Scott Waldman at E&E News, in which John Bates walks back on some of his previous monstrously wrong claims

How a culture clash at NOAA led to a flap over a high-profile warming pause study - article by Warren Cornwall and Paul Voosen at Science, 8 February 2017

Other blog articles about the atrocity from David Rose, John Bates and Judith Curry:

• Victor Venema at Variable Variability: David Rose's alternative reality in the Daily Mail
• Zeke Hausfather at Carbon Brief: Factcheck: Mail on Sunday’s ‘astonishing evidence’ about global temperature rise
• Peter Sinclair at Climate Crocks: Much Ado about a NOAA Thing
• ATTP: Expose: David Rose does not understand baselines
• Peter Thorne at Icarus (ERSSTv4 paper co-author): On the Mail on Sunday article on Karl et al., 2015
• Gavin Schmidt on Twitter: Hilarious screw up by @DavidRoseUK and #FailOnSunday
• Eli Rabett at Rabett Run: Boiling Bates Down
• Zeke Hausfather at ATTP: Guest post: On Baselines and Buoys 
• Gavin Schmidt at RealClimate: Serving up a NOAA-thing burger
• The Great White Con: Climategate 2 Falls at the First Hurdle? - and more here

From the HotWhopper archives:

• NOAA: No pause in the global surface temperature - June 2015, with references
• Tricks used by David Rose, denier "journalist", to deceive - January 2015
• The IPCC climate message is clear based on the evidence: The fundamental flaws of Hollin & Pearce - October 2015 - critique of an article written by a couple of denier apologists who tried to redeeem David Rose's bad science and trash the IPCC
• Evidence from media reports from journalists who asked a question at the IPCC press conference, September 2013 - relates to the previous article and shows that David Rose was the only journo who asked a question at the press conference who couldn't get the facts right
• Some history that led to Judith Curry hunkering down in the NOAA conspiracy theorists' bunker - November 2015