A couple of weeks ago I wrote about how John McLean thought he found an error in HadSST data, and how no-one else who looked could find the same error (except for some "missing data" notations that Zeke Hausfather noticed). Well it turns out there were some errors - in the files (not the data - see below). I found out from WUWT that there's an update on the HadSST website:
08/04/2016: An error in the format of some of the ascii files was brought to our attention by John McLean. Maps of numbers of observations and measurement and sampling uncertainties provided in ascii format ran from south to north rather than north to south as described in the data format. This has now been fixed. In some cases, the number of observations in a grid cell exceeded 9999 and were replaced by a series of asterisks in the ascii files. This too has been fixed with numbers of observations now represented as integers between 0, indicating no data, and 9,999,999, indicating lots of data.So if you've been using HadSST lately, you might want to check the data.
After someone said that Nick Stokes and Zeke Hausfather should apologise, Nick Stokes points out why no apology is warranted and has clarified what the error was and what in the original claim was wrong:
April 11, 2016 at 11:31 pm
“So lets see if Nick and Zeke would like to apologize and say they were wrong”
No, we were not wrong. The original claim of John McLean, on which Josh’s cartoon was based, was:
“1 – Files HadSST3-nh.dat and HadSST3-sh.dat are the wrong way around.
About 35% down web page https://crudata.uea.ac.uk/cru/data/temperature/ there’s a section for HadSST3. Click on the ‘NH’ label and you go to https://crudata.uea.ac.uk/cru/data/temperature/HadSST3-nh.dat, which has ‘nh’ in the file name. But based on the complete gridded dataset that data file is for the Southern Hemisphere, not the Northern. The two sets are swapped. The links to named files are correct but the content of those files is wrong, likely due to errors in the program that created these summary files from the SST3 gridded data.”
We both checked it. That claim was wrong. The two sets were not swapped. The contents of
and the SH version were correct, and have not been changed. The key link is
“But based on the complete gridded dataset…”
JM did a calc based on the ascii gridded set, interpreting the format as per the guide, got something different, and asserted that the .dat files were wrong. But they aren’t. His calc went wrong, not through his fault. The latitudes were listed in the opposite order to what was stated in the guide. HAD has corrected what made it go wrong.
The thing is that people commenting here mostly not only did not check themselves, but don’t know what the files being talked about are. The ascii grid output files which contained the error are obsolete. Hadley now produce a netCDF version which do not have these format issues, and are what serious researchers use. So JM’s claim
” the content of those files is wrong, likely due to errors in the program that created these summary files from the SST3 gridded data”
is wrong on two counts. The file content was not wrong, and there was no way that it would have been generated by someone reading the ascii files.
The issues (2 and 3) on observation counts (order and format overflow) were not disputed at any stage, and Hadley has corrected them. As JM said:
“The HadSST3 observation count problems won’t be used by many people, maybe I’m even the first if no-one else has hit the problems.”